Re(de)fining The Quality Start

by KerryWhisnant

In a recent Dugout Central article, John Bowen asked if the quality start is a bad statistic. A lively discussion ensued, and the general consensus was that it rewarded mediocre pitching, at least for those barely meeting the requirements. In this article I present data on how often a team wins given how long the starting pitcher lasts and how many earned runs he allows, so that people can judge for themselves what the criteria for a quality start should be. I also give my own opinion, and suggest two possible redefinitions of the quality start.

Using data from the last 5 years (2006-2010), taken from baseball-reference.com, the following table lists the winning percentage of a team if the starting pitcher goes at least the number of IP shown (decimals refer to thirds of innings) and he gives up no more than the number of ER shown.

IP 0ER <=1ER <=2ER <=3ER <=4ER <=5ER all
>=3.0 0.846 0.775 0.708 0.644 0.596 0.559 0.514
>=3.1 0.849 0.777 0.709 0.646 0.598 0.563 0.519
>=3.2 0.849 0.777 0.709 0.647 0.599 0.564 0.523
>=4.0 0.850 0.777 0.710 0.648 0.601 0.567 0.529
>=4.1 0.852 0.780 0.714 0.653 0.607 0.577 0.544
>=4.2 0.852 0.780 0.715 0.655 0.610 0.581 0.552
>=5.0 0.852 0.781 0.716 0.658 0.614 0.587 0.561
>=5.1 0.863 0.789 0.728 0.673 0.634 0.610 0.589
>=5.2 0.864 0.791 0.730 0.676 0.640 0.618 0.601
>=6.0 0.866 0.792 0.731 0.678 0.644 0.624 0.611
>=6.1 0.891 0.816 0.761 0.713 0.683 0.669 0.658
>=6.2 0.892 0.819 0.768 0.722 0.696 0.683 0.675
>=7.0 0.894 0.822 0.769 0.726 0.702 0.692 0.686
>=7.1 0.942 0.878 0.833 0.800 0.776 0.764 0.758
>=7.2 0.945 0.887 0.843 0.811 0.788 0.779 0.774
>=8.0 0.942 0.887 0.840 0.809 0.788 0.780 0.777
>=8.1 0.976 0.958 0.939 0.936 0.923 0.922 0.918
>=8.2 0.981 0.962 0.945 0.941 0.931 0.931 0.927
>=9.0 0.984 0.966 0.948 0.947 0.941 0.941 0.939

When a pitcher hurls a quality start (at least 6 IP and no more than 3 ER), his team wins 67.8% of the time. Other things we learn, for example, are that if a pitcher lasts at least 5 innings, on average his team wins 56.1% of the time, and if he lasts 5 innings and allows no earned runs his team wins 85.2% of the time.

But the cumulative totals don’t tell the whole story, since the 6 IP/3 ER qualification includes many performances much better than that (for example, a 9-inning shutout is also a quality start). To really determine if a particular start is worthy of being called “quality,” we must look at how the team did for exactly that number of IP and ER, shown in the following table. (Note that the numbers vary quite a bit for 0 ER, 5 or more ER, and more than 8 IP due to low statistics for those cases.)

IP 0ER 1ER 2ER 3ER 4ER 5ER 6+ER
3.0 0.500 0.538 0.474 0.364 0.316 0.153 0.175
3.1 1.000 0.750 0.500 0.286 0.342 0.204 0.111
3.2 0.500 0.545 0.300 0.370 0.275 0.156 0.079
4.0 0.591 0.569 0.301 0.254 0.279 0.175 0.091
4.1 0.600 0.727 0.474 0.347 0.294 0.210 0.116
4.2 1.000 0.500 0.508 0.246 0.290 0.216 0.101
5.0 0.724 0.673 0.527 0.441 0.308 0.250 0.160
5.1 0.786 0.704 0.595 0.523 0.307 0.265 0.129
5.2 0.750 0.762 0.655 0.510 0.405 0.286 0.103
6.0 0.781 0.691 0.590 0.483 0.384 0.283 0.164
6.1 0.857 0.723 0.583 0.498 0.354 0.302 0.113
6.2 0.851 0.726 0.726 0.560 0.409 0.266 0.089
7.0 0.847 0.735 0.615 0.481 0.405 0.325 0.179
7.1 0.911 0.727 0.703 0.645 0.425 0.261 0.000
7.2 0.968 0.830 0.828 0.623 0.455 0.438 0.000
8.0 0.912 0.780 0.638 0.467 0.320 0.258 0.333
8.1 0.867 0.929 0.813 0.909 0.500 0.500 ——-
8.2 0.900 0.875 0.933 0.750 0.250 1.000 0.000
9.0 0.984 0.936 0.821 0.905 0.500 ——- 0.000

Here we see that the minimum quality start, 3 ER in 6 IP, actually is a losing proposition for the team, which wins only 48.3% of the time with that performance by its starting pitcher. If you think a quality start should enhance a team’s chances beyond a coin flip, then 6 IP/3 ER is definitely not good enough.

It’s interesting that pitching 5 1/3 or 5 2/3 innings and allowing 3 ER is actually better for the team than pitching 6 innings with 3 ER. A similar pattern can be seen elsewhere in the table. I think this is because teams that are behind tend to pinch hit for the pitcher, which means that the starter will have an even number of  IP and the team will tend lose more often. Teams that are ahead tend to replace their pitcher mid-inning, to preserve the lead, and might tend to win more often.

Even extending the quality start qualifier to at least 7 IP still doesn’t assure even a 50% chance of winning (although 6 2/3 IP does, due to the fractional inning effect mentioned above). If you say that a quality start should give the team a 60% chance of winning (roughly three out of five), a pitcher allowing 3 ER should go at least 7 1/3 innings.

If you require a 67% probability of winning (about two out of three), then you would need 8 1/3 inning with 3 ER, maybe 6 2/3 IP for 2 ER, and 5 IP for less than 2 ER — fewer than 5 IP with less than 2 ER can also give the team a good chance to win, but a pitcher should at least qualify for a win to also earn a quality start.

It doesn’t seem feasible to have the same maximum number of ER for all IP in a new quality start requirement – at least if you want a quality start to have a uniform benefit to the team. If you want something that’s easy to remember, you could require that the number of IP would have to be at least 5 more than the number of ER, with a maximum of 3 ER. That would mean a quality start would give a team an 80% chance to win on average, and about a 70% chance if a pitcher meets the minimum requirement. The last two sets of criteria mentioned above are listed in the following table. On average, both give the team about a 75% chance of winning.

Criteria 1 Criteria 2
0 ER >= 5 IP >= 5 IP
1 ER >= 5 IP >= 6 IP
2 ER >= 6 2/3 IP >= 7 IP
3 ER >= 8 1/3 IP >= 8 IP

Under Criteria 1 there would have been 8552 quality starts in the years 2006-2010; for Criteria 2 there would have been 7893. This is compared to 11,874 standard quality starts and 24,296 total starts in those years.

For me, either one of these definitions for a quality start is much better than the current one. What do you think?

31 Responses to “Re(de)fining The Quality Start”

  1. Raul Says:

    This is awesome…and very interesting.

    You lost me on the Criteria 1 and Criteria 2.
    Criteria 1 is Earned Runs plus 5, subject to a maximum allowed of 3?

    I think there needs to be an easier way to look at this.

    I understand what you’re saying…it’s just…visually the 2 criteria aren’t appealing.

    There has to be a better way of explaining this “floating” quality start definition.

  2. Jim Says:

    Kerry, well thought out. But my criticism is that it is overly complicated, why not simply say something to the effect that a quality start is any start that exceeds the 60th percentile of all starts in the prior season for innings completed and ERA. The numbers would change slightly year to year, but would reflect the trend of performance in BB.

  3. KerryWhisnant Says:

    Raul, Criteria 2 is IP >= ER +5, with ER = ER + 5 with ER <= 3, as Raul said). The key is to get away from having the same ER criteria for all IP, since that does not give a consistent team winning %.

  4. KerryWhisnant Says:

    One more try.

    Raul, Criteria 2 is IP >= ER +5, with ER = ER + 5 with ER <= 3, as Raul said). The key is to get away from having the same ER criteria for all IP, since that does not give a consistent team winning %.

  5. KerryWhisnant Says:

    OK, I figured out what’s going on. Somehow it’s interpreting my greater than and less than signs as html code, and ignoring what’s between them. One more time…

    Raul, Criteria 2 is IP >= ER +5, with max ER of 3.

    Raul and Jim, Criteria 1 attempted to define it in terms of a team’s winning %, but that has problems, too, since it actually goes down as you go from 2/3 of an inning to the next full inning in many cases (the fractional inning effect I mentioned). So, for example, if you used 70% as your criteria, 1 ER and 5 1/3 or 5 2/3 IP would count, but 6 IP would not quite (!). Criteria 1 is an attempt to keep that philosophy but smooth it out.

    I think I like Criteria 2 better; it gives very similar results and the formula is fairly basic (IP >= ER + 5 with max of 3 ER, as Raul said). The key is to get away from having the same ER criteria for all IP, since that does not give a consistent team winning %.

  6. John Says:

    I maintain that the accepted version of a quality start was fine for the steroid era. It was actually an above average ERA for a bit, and definitely well above the median start.

    The recent downswing in offense (yay Mitchell Report!) has definitely changed that, although I was pretty happy with 6&3 from Brewer starters the last couple years :)

    Great work, Kerry! I find it interesting how just looking at the minimum requirement is tricky because, for reasons that I can’t comprehend, a 6&3 start has yielded a lower winning percentage than 5.2 or 5.1 IP and the exact same number of runs! Since 08, I saw that 6&3 yielded a better WP than 7&3! Doesn’t really make sense…maybe starts of 6&3 were on teams with better bullpens, meaning that managers had more faith pulling their guys after 6 – and those better bullpens were more successful at holding leads over 3 innings than the worse ones were at holding leads for 2. That might be the reason, I wonder how you could use the play index to gather that data though!

    I will say that I had to read over your formula twice to really get it – & I’m probably more technical-minded than the typical fan!

  7. Chuck Says:

    If I’m reading the chart correctly, the odds of pitching a shutout, or allowing one run, go down the more innings one pitches.

    But the odds of allowing three or more runs also goes down, which, again, if I’m reading correctly, DOES NOT surprise me.

    Like a marathoner, pitchers may hit a wall at some point during a game and lose command and have a rough inning, but as a group show a propensity to regain their stuff pretty quickly and settle down.

    80% of CC Sabathia is better than 100% of anyone in the Yankees bullpen for one inning..including Mo.

  8. Chuck Says:

    “http://www.baseball-reference.com/blog/archives/10350″

    Baseball Reference’s “new” HOF based on WAR.

  9. Chuck Says:

    Another interesting “stat” take by John Thorn.

    http://bleacherreport.com/articles/638724-baseball-and-america-how-our-changing-passions-have-shaped-the-game

  10. Jim Says:

    From Thorne’s article “Statistics are something of a fetish. Like a shrunken head, a stat is an encapsulation of a power once alive.”

    Should there be a minimum hat size requirement for posting articles and comments on this site?

  11. KerryWhisnant Says:

    @7, the chart gives the odds of the team winning, not the odds of that outcome occurring.

    “80% of CC Sabathia is better than 100% of anyone in the Yankees bullpen for one inning..including Mo.”

    I know you aren’t enamored with relief pitchers (and they are inferior overall), but I have to disagree with that statement. Now if you say 100% of CC is better than 100% of Mo, I could see that, even though Mo’s rate stats are more impressive.

    Mo is overpaid for the amount of work that he does, although his salary/WAR isn’t that much more than CC’s over their careers, 2.5M$/WAR to 1.7M$/WAR — I would have thought Mo was much more expensive than CC, but it was only 50% more (still no bargain). Compare that to ARod (2.6M$/WAR) and Pujols (1.1M$/WAR), just to take two examples. Of course Albert’s value per WAR is going to take a big hit with his new contract, wherever he ends up.

  12. John Says:

    100% of Mo is clearly better than 80% of CC Sabathia for one inning…or even two.

    1 inning of Mo gives you, on average, the best ERA and WHIP….ever.

    Chuck’s waiting for Rivera to finally, at like, the age of 45, have a bad season so he can be like, “see? The league figured him out!”

  13. John Says:

    As far as how overpaid Rivera is compared to Sabathia:

    Rivera’s typical inning is roughly twice as good as Sabathia’s typical inning.

    He pitches roughly 1/3 as many innings.

    He makes about 2/3 as much as much as CC.

    I dunno. Lots of rounding, but it doesn’t seem that extreme.

  14. Cameron Says:

    I took a look at the “Hall of wWar” article that Chuck linked to. Taking out the ineligible members (Pete Rose, Joe Jackson, and Eddie Cicotte), here’s the people that aren’t in the Hall of Fame that would make it.

    …Some of these “Hall of Famers” are pretty laughable.

    Dick Allen
    Kevin Appier
    Jeff Bagwell
    Sal Bando
    Buddy Bell
    Bobby Bonds
    Ken Boyer
    Roger Breshnan
    Kevin Brown
    Charlie Buffinton
    Cesar Cedeno
    Ron Cey
    Cupid Childs
    Will Clark
    David Cone
    Bill Dahlen
    Willie Davis
    Darrell Evans
    Dwight Evans
    Chuck Finley
    Bill Freehan
    Jack Glasscock
    George Gore
    Bobby Grich
    Stan Hack
    Keith Hernandez
    Orel Hershiser
    Larry Jackson
    Tommy John
    Silver King
    Jerry Koosman
    Barry Larkin
    Sherry Magee
    Edgar Martinez
    Jim McCormick
    Mark McGwire
    Minnie Minoso
    Thurman Munson
    Dale Murphy
    Graig Nettles
    John Olerud
    Rafael Palmeiro
    Billy Pierce
    Darrell Porter
    Tim Raines
    Willie Randolph
    Rick Reuschel
    Bret Saberhagen
    Ron Santo
    Ted Simmons
    Reggie Smith
    Dave Stieb
    Harry Stovey
    Frank Tanana
    Gene Tenace
    Luis Tiant
    Joe Torre
    Alan Trammell
    Robin Ventura
    Larry Walker
    Lou Whitaker
    Deacon White
    Wilbur Wood
    Jim Wynn

  15. Raul Says:

    I’m getting real tired of ESPN’s biases.

    The Royals have prospects up the ying yang who are doing interesting things this spring and there are no articles about this.

    Nothing about the Angels’ Trumbo being on fire this spring.

    Nothing about Pablo Sandoval hitting much better.

    I’m starting to wonder why I even bother with their website.

  16. Raul Says:

    I see no reason why Joe Jackson should not be in the Hall of Fame.
    In my opinion, it’s one of the unspoken stains on the game.

  17. Cameron Says:

    No Raul, the true stain on the game are fans who think Rick Reuschel’s a Hall of Famer.

  18. Raul Says:

    That’s not even worth discussing.

  19. Cameron Says:

    No, but it IS worth public mockery.

  20. Cameron Says:

    Oh man, this made my day.

    http://www.royalsreview.com/2011/3/7/2035256/royals-2011-world-series-champs

  21. John Says:

    Out of curiosity, have either of you ever seen Reuschel pitch?

  22. Chuck Says:

    “Out of curiosity, have either of you ever seen Reuschel pitch?”

    Yes.

  23. John Says:

    Well yes, you have. You also saw Al Spalding. Tough comparison.

    I can’t think of any eligible pitcher not in the hall who I truly believe deserves it. Kevin Brown was ridikulous, but not for quite long enough. Reuschel’s closer than I think you guys are giving him credit for, but only truly great a couple of years. Hossrex (where the fuck is he?) always mentioned how Hershiser deserved 3 straight cy youngs, and if he had won those, he wouldve been in.

    I don’t really think any of those guys listed were HOFers, but none of them would be even bottom ten among HOF pitchers.

  24. John Says:

    *none of the pitchers. I counted about 10 position players that I liked.

  25. Chuck Says:

    “Well yes, you have. You also saw Al Spalding. Tough comparison.”

    Hey, don’t knock Al..his fastball was better than Reuschel’s..and he threw underhand!

  26. Cameron Says:

    You want the guys I think are Hall-worthy? Not sure if this will shake my “big hall” label as much as it will or not. Anyway.

    Dick Allen
    Jeff Bagwell
    Barry Larkin
    Edgar Martinez
    Mark McGwire
    Dale Murphy
    Rafael Palmeiro*
    Tim Raines
    Ron Santo
    Luis Tiant
    Alan Trammell
    Jim Wynn

    * – I question how much of Raffy’s stuff was ability and how much is juice. Borderline.

    Most of these guys are either good or Very Good. …Not Hall of Famers.

  27. John Says:

    I think Dwight Evans is probably not quite there….but it baffles me how Jim Rice got so much more suuport. They played within an atbat of each other for their whole careers put up fairly similar offensive numbers, but Evans had an arm.

  28. John Says:

    Lol @ Chuck.

    I’ll miss our banter, be back next week!

  29. Mike Felber Says:

    There are a lot of borderline guys on that list, some worse. Some clearly deserving. Pitchers have tended to be treated a little too harshly in modern times. Partly because there were 4 historically great pitchers who started in the ’80ss. R.R. seems borderline, but was definitely underrated. Where can I find comparisons of FIP to ERA +? I would like to see who really deviates in this regard, that would help sort our the wheat from the chaff (though in reality it is a matter of degrees).

    A proportionately talented batter to Blyleven would not have waited so long as he did, & only make it with a concerted PR campaign. Brown (not considering steroids) Moose, Schilling,Smoltz, Glavine…hard to find one not worthy. Cone? He seems borderline to me. Would you put him in Chuck?

  30. Cameron Says:

    Cone was borderline in terms of pure talent, maybe, but I think he falls short of election. Out of those guys I listed, I think the most borderline is Jimmy Wynn, so for those of you who knock me I DO have standards. XP

  31. Chuck Says:

    “Would you put him in Chuck?”

    Cone?

    No.

    Although he was better than everyone you mentioned except Glavine.

Leave a Reply


RSS
Categories
Fan Duel
FanDuel - Daily Fantasy Baseball
YardBarker
Advertisement