What's Success Rate?

Sep 18, 2023

First off, some housekeeping. A lot of IlliniBoard renewals happen at this time of year. We have an auto-renew system (with a one-click "unsubscribe"), so unless you hit that unsubscribe button, your subscription will automatically renew to the credit card on file on your subscription date. Many of you have asked me "do I need to renew soon?" and my answer is always the same: as long as your credit card is current, the system will take care of your yearly renewal until the moment you've had enough of me.

The "as long as your credit card is current" part is why I'm writing this. Everyone's credit cards eventually expire. I've been doing this three years now, so I'm guessing 75% of the credit cards subscribers used originally have now expired (or been lost/stolen and replaced). Which means I'm getting a large number of "failed transaction - credit card has expired" emails these days.

When that happens, you get an email from me saying "if you'd like to stay subscribed to IlliniBoard, please enter updated credit card information." To prevent having to send some of those emails, I figured I'd just write about the process here. If you have 60 seconds right now (and you want to continue your subscription), please follow these four steps:

  1. Click on "My Account" above. If you're on your phone, you might have to turn it to landscape view to see the My Account tab at the top. (And you have to be logged in for this, so if you're logged out, you'll need to log back in.)
  2. Click on "Manage My Subscription". That will take you to the Stripe dashboard for your subscription. Stripe is the payment system we use.
  3. The last four digits of your credit card (and the expiration date) will be listed on that page. If it's an old card, or if it's expired, you can click on "add payment method" to enter a new card number.
  4. Then you'll need to make that card your default payment method, not the expired card (you can delete the expired card if you want). Now you're current and you won't be receiving an email from me on your renewal date.

And if you want to cancel, the button is right there on that page. "Cancel plan."

Thanks for taking care of that. Saves me hours of "customer service" emails.

On to the article.

I guess this is the year of Success Rate when I write these Monday (or sometimes Tuesday) NERDstat breakdowns. I've already referenced it here and on Twitter (and on Slack), so let's just keep running with it. It's just one metric of many metrics, but I think it tells the story of our season so far.

You might have seen the "Did we really get beat that bad?" graphic floating around Twitter. It's a graphic put together by a guy I've followed for a while (Parker Fleming, aka "@statsowar"). As an aside, it's crazy what choosing a good @ for Twitter will do. You have no idea how many people I meet who will reference me as "alioneye." Thanks, person I blatantly stole that from 14 years ago.

Success Rate isn't "his" stat. It's a way to evaluate football games that has been around for a while. I already put this cut-and-paste definition of Success Rate in the SOC for the Kansas game, so here it is again:

A play is defined as successful if:

  • It gains at least 50% of the yards required to move the chains on first down
  • 70% of yards to gain on second down
  • 100% of yards to gain on third or fourth down

To calculate success rate, simply divide the number of successful plays (as defined by down and distance above) by total plays.

I should also note what this stat is not. You hear me reference Bill Connelly's "Post-Game Win Expectancy" a lot. That one looks at a game's stat profile, looks back at the history of those stats in game in the past, and says "when a team does this, they generally win 67% of the time." That one factors in expected turnovers -- you should recover around 50% of fumbles, you should intercept around 20% of all passes defended -- and if you're grabbing every fumble and grabbing every pass that a defender gets a hand on, you're a bit fortunate (and your win expectancy will reflect that). Combine that with all of his predictive stats (explosiveness, efficiency, etc.) and it spits out your win expectancy.

Success Rate isn't trying to do that. It's not trying to say who "should" have won a game, at least not directly. It's very simple. Either the offense or the defense "wins" every single play. If the offense gains 6 yards on 1st and 10, they win. If the defense holds them to 4 yards on 1st and 10, they win. On 2nd down, the offense needs at least 7 yards (on 2nd and 10) or else the defense wins that play. On 3rd down, the defense needs to force 4th down or the offense wins.

Nothing else factors in. There's no category for turnovers, no category for penalties killing a drive. Game changing plays like a 99-yard fumble return or a punt return for a touchdown don't show up as anything. It's just trying to determine which team is winning more individual plays as the game goes along. Bill Connelly refers to Success Rate as the "on-base percentage" stat for college football and I think that's perfect.

It's a stat that reveals a lot, I think. Let me just link the charts that Parker Fleming has put out there the first three weeks.

Week One:

Maybe the best way to look at this one is to look at the games either side of Illinois-Toledo on a Net Success Rate chart. With similar advantages in Success Rate, Kentucky beat Ball State 44-14 and Florida State beat LSU 45-24. Illinois beat Toledo 30-28. What does that say? It says that we nearly let mistakes and penalties turn what should have been a comfy 38-20 win into a 28-27 loss. Especially against a MAC opponent.

It's also educational to look at each end of this chart. Minnesota had no business beating Nebraska but Nebraska did the late-game turnover thing again and handed them the game. And on the other end, yes, Virginia fans, it was just as bad as it looked and it's probably going to get worse. (It has gotten worse. So much worse.)

Week Two:

I don't think anyone would question that Kansas out-played us straight up. We're on the left side of the chart, so no, we didn't really "get beat that bad", but it was still one team beating the other fairly convincingly. As we discussed at the time, this wasn't the road losses at Virginia in 2021 or South Florida in 2017 or North Carolina in 2015, but we still got beat.

On the other end of the chart, Missouri squeaked one out over Middle Tennessee State but still dominated Success Rate (which, sight unseen, says they tried to give a game away when they were dominating most every drive). We then saw the positive result of them playing a cleaner game this past weekend (gross).

Week Three:

Remember, Success Rate doesn't look at turnovers. That interception at the 30 is just a single play won by the defense. The idea, at least the way I view it, is to erase all of that and simply say "are you getting beat up and down the field (2017 Illini) or are you holding your ground?

Let's just look at the games for the top-8 teams in the AP Poll since Penn State is #7 (and I'm going to 8 and not 10 because Notre Dame is 9). I'll list the opponent, the score, and the Net Success Rate (interpolating from the chart so I might miss one of these by a hundredth).

Georgia over South Carolina 24-14 - 0.19
Michigan over Bowling Green 31-6 - 0.33
Texas over Wyoming 31-10 - 0.13
Florida St. over Boston College 31-29 - 0.09
(USC bye week)
Ohio St. over WKU 63-10 - 0.19
Penn State over Illinois 30-13 - 0.01
Washington over Michigan St. 41-7 - 0.24

Now, that might say more about Penn State than it says about us. It's either saying that all of those other teams dominated inferior opponents or that the opponents weren't inferior. Maybe this just means that Penn State finishes the season #24, not #7.

Or perhaps it means that we're a little better than we've shown so far. And that's why I'm writing this article. This is just one metric among dozens of metrics. It's not to be pointed to as "see, here's the truth about this team." But it might - might - suggest that we'll be able to hang in every game from here on out. (I mean, not if we go -5 in turnovers every game, but you get the point.)

Winning through yards is sustainable. Winning through penalties and turnovers is not. If you're consistently "winning", say, 60% of the plays in a game (as defined by Success Rate), then you probably have a sustainable team. You can move the ball. You can stop the ball. If you're only "winning" 40% of the plays but you're still winning games because the other team is turning it over and killing drives with penalties, then no, that's probably not sustainable.

Here's where I'll peek at Connelly's PGWE real quick. This one is a different stat. This one is a "should you have won the game?" stat. This one looks at turnovers and tries to separate out the expected turnovers from the "OK now this is getting a bit ridiculous that the fumble bounced right to him" turnovers. The Post-Game Win Expectancy for the Penn State game: 24.1%. Play that game 100 times with the same statistics, Penn State wins 76 times and Illinois wins 24. Did you just tilt your head? I did.

You've heard me say before that not all turnover games are the same. Some meet the expected turnovers in the game, some far exceed it. I guess this will require a bit of a deep dive so buckle up.

Here's an example from Connelly's Twitter of him him arguing with a Michigan State fan back in 2018:

What he's saying there: On average, teams recover 50% of fumbles and grab around 20% of all passes defended (got a hand on it, including interceptions where they "got both hands on it", if you will.) In the game he's referencing, there were four fumbles and Michigan State recovered all four and there were 16 passes defended and Michigan State was only intercepted once. A game that statistically should have seen six Michigan State turnovers only saw one.

So let's go to our game from Saturday. There were two fumbles in the game. Penn State recovered both (and, not that it matters, but both one-hopped directly to a Penn State player). Penn State had 7 passes defended and intercepted 4 of them. Illinois had 6 passes defended and intercepted 0.

Before you start hyperventilating, no, those numbers don't result in "Illinois wins that game 95% of the time" or anything. Illinois still only wins that game 24% of the time. I'm not saying that Penn State got a bunch of lucky turnovers and won the game. Penn State did the football things better than we did and absolutely deserved to win.

I'm pointing to why this game was "Penn State wins that game three out of every four times". In early 2021, those four consecutive losses were not 76% / 24% losses. UTSA had an 87% chance to win. Virginia had a 100% chance to win. Maryland had a 96% chance to win. Purdue had a 83% chance to win. Even the close games were, statistically, games that our opponents should have won handily.

That's not the case so far this year. SP+ looks at the Penn State game and sees that the team with our stats should lose to the team with their stats by 2.9 points. We lost by 17. And one more time for the people in the back. When Bill Connelly's laptop "looks" at the box score it looks at passes defended and fumbles and projects the expected turnovers. It doesn't look at the actual turnovers from the game. It's looking for what typically happens, not what actually happened. Yards are sustainable. Having the defense close enough to get a hand on six passes but intercepting none is not.

That's enough PGWE. Let's wrap this up by going back to Success Rate.

I can't find firm Success Rate numbers from previous seasons, but just think about Illini teams in the past. When we got to 4-1 in 2015 with that offense that couldn't move the ball, I'm guessing the Success Rate numbers were screaming "yeah, this isn't sustainable at all." I'm guessing we weren't playing toe-to-toe football against anyone and were relying on inferior opponents to shoot themselves in the feet. You all know what watching that kind of offense feels like. So many 2nd and 9's it makes your head spin. 3rd and 3 feels impossible.

Or think back to the 2018 defense. You know what that feels like, too. I'm guessing that when we were 3-2 going into the Purdue game that year, the Success Rate numbers were screaming "this isn't in any way sustainable because the defense 'loses' so many plays." I remember the discussions that season about how we weren't even forcing third downs. You can get by on other stuff here and there and maybe win a few games, but if your Success Rate numbers are bad, you're eventually doomed. Like, lose at home to a just-OK Purdue team 46-7 doomed.

This season, we've played three solid opponents to start. I don't have a crystal ball but I believe that all three are bowl teams. So when we play solid opponents, I look to Success Rate to see where we stand. If it's Northwestern playing a horrific UTEP team, then no, Success Rate doesn't tell you much. But if you're playing decent opponents, you can look to see if you're holding strong or "really getting beat that bad."

We're not "really getting beat that bad." We're beating ourselves.


Ellisrt1031 on September 18, 2023 @ 09:44 PM

Agree..but we need to be better at offensive line pass protection and we need to play Reggie 75% plus snaps and consider using Feagin as second back. I am hopeful that FAU allows us to get our OL set for year. I don't think we have been good at pass protection for a long time. We used Tommy well last year. He could get it out quickly and accurately. Safety now an issue also. Punting again. The West is weak enough we have a decent chance to win 6 or 7.

uilaw71 on September 19, 2023 @ 07:11 AM

I was feeling better about the remaining games until the Bailey and Hill news hit. Our defensive success rate will be sorely tested with both out for the season.

Loyal2u on September 19, 2023 @ 10:34 AM

Was super easy to check my subscription status, thanks Robert!

Speak Your Mind

Please login or register to post comments on the IlliniBoard.

Post Preview