Basic Analytics (And How The Nfl Simply Doesn't Use Them)

[sdballis]

http://fivethirtyeight.com/features/nfl-coaches-are-getting-away-with-crimes-against-middle-school-math/

 

Semi-lengthy but lays out some very fundamental ideas that the statistics show the NFL as a whole should adopt immediately, but no one has had the balls to do so yet. Maybe our "cutting edge" front office and HC implement some of these ideas next season??

 

P.S. Sorry if the formatting is off. I know you guys like the story posted as well as the link. Thanks.

 

 

The NFL divisional playoff action once again demonstrated that NFL coaches are terrible at basic win-maximizing tactical decisions. In particular, there were two glaring spots where teams failed to go for two despite it being a fairly straightforward decision.

In the wake of Green Bay’s loss to Arizona — in which the Packers conceded a relatively routine winning touchdown on the first drive of overtime after spectacularly driving the length of the field (and then some) to tie the game in regulation — debate rages about whether the NFL needs to guarantee that both teams get a possession in OT. Whatever you think of the overtime rules, they are what they are, and under those rules, the Packers abandoned their best chance of winning by kicking the game-tying extra point at the end of regulation instead of going for it.

Much more quiet, but no less frustrating, was the call after Kansas City — which never seemed like much of a threat to New England — scored a touchdown with 1:13 left in the game that cut its deficit to 8. No one outside of sports nerd Twitter raised an eyebrow as the Chiefs kicked the extra point to draw within 7, and after they failed to recover their attempted onside kick, it’s likely that no one will ever care. But failing to go for two in this exact situation is one of the clearest and easiest-to-demonstrate mistakes in all of football, and how coaches continue to make this error virtually 100 percent of the time is a melancholy mystery.

Look, “advanced” stats can be opaque. In midfield situations, whether to punt or go for it on fourth down, for example, takes some confidence in expected points and expected win models that are statistically somewhat complicated and can sometimes get things a little wrong. Although I think those models are pretty good (or are at least good enough for most of the types of decisions they’re used to analyze), I can see how someone might find them foreign or overly abstracted. And yes, the statsy crowd can be preachy and overconfident. We probably don’t know as much as we think we do, and we often aren’t very good at explaining ourselves to skeptics.¹

But some decisions — like the ones faced by Green Bay and Kansas City — aren’t that complicated. Analyzing them requires no advanced statistical techniques, and solving them requires no more than grade-school-level math and an eye on win maximization.

First up: How Green Bay broke my spirit

Aaron Rodgers got utterly jobbed. To recap — not because you aren’t familiar, but because it’s not possible to relive this too much or too soon — at one point, the Packers’ last drive looked like this:

(4th and 20 from own 4; 54 seconds left)

Just 54 seconds of game time later — after Rodgers pulled the requisite pair of canonizing miracles out of his backside — all the Packers needed to win the game and knock off the second-seeded Cardinals was 2 more yards.

As the game went to commercial, I hoped against hope that Mike McCarthy would do the right thing and let that game live or die on Rodgers’s ability rather than try to send the game to overtime, on the road, against a superior opponent.

Was I being emotional about what I’d just witnessed? Sure. Even if the Packers ended up winning, it was depriving Rodgers — my sometimes muse— of the opportunity to complete what would have been probably the greatest drive in NFL history. But I also felt the passion of conviction — that this was the right choice — and the desperate hope that the professional NFL decision-maker would have arrived at the same conclusion.

In the first season of “Survivor,” the three final contestants were Richard Hatch, Kelly Wiglesworth and Rudy Boesch. Rudy was everyone’s favorite elderly curmudgeon. There would be one more challenge, and the winner would get an automatic berth in the final and the ability to vote off either of the two remaining opponents. Functionally, this meant they could choose their own opponent in the jury vote, in which eliminated contestants would choose the overall winner. Richard had a long-standing alliance with Rudy and was faced with an interesting strategic conundrum. If he won the final challenge and took Rudy with him, he would likely lose the voting to the lovable old homophobe. But if he betrayed Rudy, all that Rudy love would likely turn into Richard hate, and he would likely lose the jury vote to Kelly.

The “Survivor” forums were abuzz with strategic discussions,² and people saw this conundrum coming weeks before it came to fruition. The solution to Richard’s problem — indeed, his only option if he wanted to win — would be to lose his final challenge intentionally. Then if the third player (ultimately Kelly) won, she would be forced to eliminate Rudy herself. (If Rudy won, none of this was likely to matter, as he seemed likely to win the final vote regardless of who his opponent was.) The “will he or won’t he?” suspense when it came down to the final four was amazing. On the one hand, the logic was sound and it was clearly the right choice, but on the other hand, it’s asking a lot to expect someone to see past the seemingly lower-risk play of winning the challenge and guaranteeing his spot in the final two.

After whittling the field down to the final three, the anticipated situation finally came to be. The final challenge was called “hands on an idol” — each contestant had to stand on a small log while keeping one hand on a pole, and the last one to break contact would win. At first it looked like Richard was going to play the game straight-up. Hours passed (or so the show told us). Jeff Probst offered an orange to any contestant who wanted to join him. And then, finally, it happened:

With a twinkle in his eye, Richard wished the others luck and let go. It was one of the most satisfying moments of my televiewing life. It filled me with optimism that, yes, human beings are smart and capable creatures. Hatch went on to be the inaugural “Survivor” winner, claiming his $1 million prize (in addition to jail time for failing to pay taxes on it).

The opportunity for the Packers to cap off that already legendary drive with a counterintuitive but mathematically sound two-point attempt — whether successful or not — had the potential to be another such reason-affirming moment for me. But alas:

"As much as I loved that game and don't give a crap about who won, I think Packers not going for 2 is going to disappoint me my whole life."

Now, don’t get me wrong: That the Packers should have gone for two wasn’t obvious. But just because it wasn’t obvious doesn’t mean the call was difficult. This requires no advanced math and could literally be on a middle school homework assignment.

The question is: Which is greater, the chances of (1) Aaron Rodgers converting that 2-point conversion, or the chances that the Packers (2) make the extra point and (3) win in overtime? To make this comparison, we need to know or estimate three numbers.

Let’s start by looking at league averages:

1. Two-point conversion success rate: Since 2001,³ teams have converted47.2 percent of their 2-point tries from the 2-yard line (431 of 913).
2. Extra point success rate: Since the inception of the longer extra point this season, NFL kickers have made 94.3 percent of their attempts from the 15-yard line (1,131 of 1,199).
3. Expected winning percentage in overtime: Since 2001, the away team has won in overtime 45.5 percent of the time (110 of 242 overtimes that produced a winner).

With these numbers (which used only division), we can find our chances of winning for each option using — wait for it — multiplication.

• Go for two: With no time left, this is exactly equal to the estimated 2-point success rate: 47.2 percent.
• Send to overtime: Chances of making extra point multiplied by chances of winning in overtime. 94.3 percent * 45.5 percent = 42.9 percent.

There, we already have a baseline 4.3 percentage point advantage to going for two for a typical road team in the Packers’ position, using nothing but grade-school mathematics.

But those are just baselines, right? Everyone from coaches to media to fans will tell you that averages miss the hundreds of situation-specific factors at play. This is a technically true but often misleading rejoinder — and one that’s almost always used only to defend the status quo.

But in the spirit of accuracy and transparency, I’ve tried to refine the assumptions that go into that calculation above.

1. Two-point conversion success rate: Adjusting for team strength and refining the data to the most comparable situations boosts our estimate to 48.8 percent.
2. Extra point success rate: Adjusting for league trends and kicker Mason Crosby’s skill raises our estimate to 95.9 percent.
3. Expected winning percentage in overtime: Adjusting for the overtime rules changes and playoff dynamics lowers our estimate to 42.6 percent.

If you would like a little more detail about how I arrived at those estimates, here is a longish footnote.⁴

So here’s where we stand under our revised assumptions:

• Go for two: Equals estimated 2-point success rate: 48.8 percent.
• Send to overtime: Chances of making extra point multiplied by chances of winning in overtime. 95.9 percent * 42.6 percent = 40.9 percent.

Naturally, these educated guess assumptions could be off in various respects, but that 8 percentage point gap is hard to overcome. When people who argue that there’s too much uncertainty to buck the status quo actually list the variables they have in mind (unfortunately, they often don’t), they tend to overestimate the amount that situation-specific variables affect the balance of probabilities. And the variables cited often don’t even cut the way they think they do. For example: In this case, an oft-cited factor is that the Packers’ receiving corps was weakened by injuries, including the loss of Randall Cobb earlier in the game. But, as I discussed in the footnotes, anything that makes the Packers weaker relative to the Cardinals is likely to hurt their chances in overtime more than their chances of converting the 2-point try.

Thus, our best (and perhaps slightly conservative) estimate is that the Packers cost themselves about 7.9 percent of a win by kicking rather than going for two, and this whole thing could have been avoided if NFL coaches took the time to sit down and learn some basic percentages.

Kansas City fails to butter its toast

In Kansas City, we fast-forward past the long, grueling slog down the field that Andy Reid perpetrated on Chiefs fans. That was painful to watch, but it was a problem of tactics and execution, not arithmetic. Reid went so far as to say that right up until the onside kick, things went exactly as planned. Except the plan went off-rail one play before, when the Chiefs kicked the extra point — an error that is in some ways even more frustrating than the Packers’ because it’s so so simple and has been clear for so long: If you are down 14 and score a touchdown late in the game — where you are very likely to have only one more scoring opportunity, at most — you should go for two.⁵

This doesn’t require any modeling, it requires just a little thought and a little more grade-school math.

In a situation like the Chiefs’ — where there was only 1:13 remaining — it doesn’t matter that their chances of recovering an onside kick and scoring another touchdown are very, very small. We don’t forgo safety checks on airplanes because the odds of a crash are small, and you shouldn’t ignore basic win maximization just because it will only earn you the occasional extra win.

The key is to assume you get the second touchdown (and in K.C.’s case, recover the onside kick, but the logic is the same when you have time to kick off and go for a defensive stop) — because if you don’t, it doesn’t matter what you do now, you lose. Once that little leap is taken, this all flows from a little multiplication.

If you kick an extra point, you are essentially playing to make two extra points and win in overtime.⁶

As above, let’s use league averages. Assume a 94.3 percent chance of making each extra point and a 45.5 percent chance of winning on the road in in overtime.

• Your chances of winning this way are 94.3 percent * 94.3 percent * 45.5 percent = 40.4 percent.⁷ Add in the small chance that you’ll miss the extra point but then make up for it with a successful 2-point attempt, and your overall chances of winning with this strategy are 41.7 percent(of the times that you get the second touchdown).

Now, if you go for two and make it (which you should about 47.2 percent of the time), you can win by kicking an extra point (94.3 percent) after your second TD, or (much less frequently) by missing your extra point but still winning in overtime. If you go for two and don’t make it (which will happen 52.8 percent of the time), you can still win by going for two again (47.2 percent) and making it and then winning in overtime.

• Chances of making 2-point conversion and XP: 44.5 percent
• Chances of making 2-point conversion, missing XP and winning in OT anyway: 1.2 percent
• Chances of missing 2-point conversion, making second attempt and winning in OT anyway: 11.3 percent

• Combined chances of winning: 57.1 percent (of the times that you get the second touchdown)

I won’t bother going into detail trying to find perfect assumptions as I did above, because this calculation isn’t close enough for them to be necessary. Even if you assume a 50 percent chance of winning in overtime and a 100 percent chance of making your extra points, you still only need around a 38.5 percent chance of making each 2-point conversion (which would be absolutely terrible) to make going for it the better play.

Despite its obvious correctness, this is pretty much never done in the NFL. The only case of a coach going for two after a touchdown brought the team within 8 points in the fourth quarter since 2001 was Brian Billick with the Baltimore Ravens in 2001. Further back is murkier, though according toFootball Perspective, the only time a team trailing by 14 has ever scored a touchdown and a two-point conversion to cut the deficit to six in the modern era was the 1994 Cleveland Browns — coached by none other than Bill Belichick.

And finally

Another year, another year with NFL coaches not doing their jobs and not being taken to task for it. By now, coaches have no excuse for not having mastered basic decisions like these.

People say coaches are afraid of media criticism. But they’re professionals, among the handful of elite who are capable of doing what they do. If a coach cares what the media thinks, let him explain his logic.

I recall when the current replay system was implemented, there was concern that people wouldn’t be able to understand what it meant that there had to be “indisputable evidence” to overturn the ruling on the field — even though burdens of proof have been an essential part of our judicial systemsince this country’s founding.

It took a little while, but the refs, and then the media, and eventually fans all came around, and now you can go to your local dive bar and hear arguments in the form of, “It looked like a catch to me, but there obviously isn’t indisputable evidence to overturn the call. Sucka.”

It’s time to do the same with arithmetic.

[Tour2ma]

Love Nate Silver's work and site... there's not a bigger stat geek around.

 

But if we are ever in the position the Pack was, then it would be hard to accept not kicking the extra point... no matter what the stats say.

 

Good read...

[Opal312]

As for Kansas City; why didn't they call timeout? They let the clock tick away! I thought Andy Ried knew better by now.

[sdballis]

As for Kansas City; why didn't they call timeout? They let the clock tick away! I thought Andy Ried knew better by now.

 

Reid claimed he "didn't wanna score 15 points too quickly" cause then that'd give Brady and Co. time to march down the field and kick the game-winner.

Cause scoring 15 points too quickly makes ANY kind of sense...

[flyingfooldoug]

It makes a lot of sense. Why play for the tie? I get tired of seeing coaches playing not to lose. If it's winning, play to win!

[Mark O]

It makes a lot of sense. Why play for the tie? I get tired of seeing coaches playing not to lose. If it's winning, play to win!

 

But what he did made no sense. he's down 2 scores and he's wasting time and then tries to justify wasting the time. This isn't pickup basketball where if you score you get the ball. He wasted any chance of scoring and then playing defense and using his timeouts to scoring wihtout a lot of time and then having to try an onside kick.

[flyingfooldoug]

But what he did made no sense. he's down 2 scores and he's wasting time and then tries to justify wasting the time. This isn't pickup basketball where if you score you get the ball. He wasted any chance of scoring and then playing defense and using his timeouts to scoring wihtout a lot of time and then having to try an onside kick.

 

I agree, he fucked up on the clock. I was referring to the general idea of playing the odds that give you a better chance to win. Gb should have gone for the 2. I have never been a fan of playing to tie unless it's a FG to keep you in it.

[Mark O]

I agree, he fucked up on the clock. I was referring to the general idea of playing the odds that give you a better chance to win. Gb should have gone for the 2. I have never been a fan of playing to tie unless it's a FG to keep you in it.

 

 

I didn't think of GB doing that...but in hindsight and reading some stuff since the game...it would've made sense. They had all the momentum...why not go for the win.

[Unsympathetic]

Andy Reid's coaching career... Is it a performance art piece focused on innovating ways to lose in only the last 2 minutes of every half he coaches? So.. when he wins, he actually loses?

[flyingfooldoug]

I didn't think of GB doing that...but in hindsight and reading some stuff since the game...it would've made sense. They had all the momentum...why not go for the win.

 

Right, you're at the 2 and need 2 pts to win. Some of that playing for OT just doesn't make sense.

[TexasAg1969]

It makes a lot of sense. Why play for the tie? I get tired of seeing coaches playing not to lose. If it's winning, play to win!

 

If you clearly have the other team's D worn out or if your team's D is clearly worn out, then going for 2 and the win makes all the sense in the world. Baylor did that in the first overtime against A&M in 2005 for their first win ever in 20 years. A&M had scored something like 3 or 4 straight TDs including the game tier to get into overtime and had scored a TD in overtime. Baylor made the TD and went for broke and made it.

[OldBrownsFan]

 

If you clearly have the other team's D worn out or if your team's D is clearly worn out, then going for 2 and the win makes all the sense in the world. Baylor did that in the first overtime against A&M in 2005 for their first win ever in 20 years. A&M had scored something like 3 or 4 straight TDs including the game tier to get into overtime and had scored a TD in overtime. Baylor made the TD and went for broke and made it.

 

I agree with that. Looking ahead to the overtime you have to think about what gives you the better chance to win. If a defense is worn out or shown little ability to stop the other team's offense than it would be better to go for 2 and the win in regulation versus trying for the win in OT.

[The Gipper]

I need DePodesta to come and explain this to me.

[bbedward]

I need DePodesta to come and explain this to me.

 

I heard today on the radio that Sushi and DePodesta are having a press conference today at 12:30 (they said it will be on WTAM 1100 locally, probably on team website and stuff too I'd imagine)

 

Maybe your questions will be answered

[poplarbluffman]

John Urschel, a backup guard for the Ravens and likely the smartest man in the NFL, including players, coaches, GM's, did research on this topic and said all teams should go for the 2 pt conversion based on mathematical probability