Amazon is bringing a whole lot of AI to Thursday Night Football this season

This NFL season, if you watch a Thursday Night Football game on Amazon Prime Video, you’re likely to see all sorts of new on-screen stuff. If a team finds itself in a late-game drive with everything on the line, you might see a graphic telling you whether the team should go for it on fourth down. When the quarterback snaps the ball, the broadcast might automatically highlight the most open receivers down the field. And as the team marches down the field, you might see lines on the field not just showing a kicker’s field goal range but the exact spot from which he’s more than 50 percent likely to nail the game-winner.

For its second season as the official broadcast partner of Thursday Night Football, Amazon is leaning on AI tools and machine learning in a big way. Last year, its primary goal was to simply do a good broadcast — no buffering, no crappy halftime shows, no awkwardness between the announcers. It largely succeeded, and even the audience numbers were better than most people expected for a streaming-only football broadcast. Now, with a year of experience and plenty of confidence, the company is looking for ways to move past just showing the game.

The main thing Amazon heard from viewers, says Sam Schwartzstein, the company’s analytics expert for Thursday Night Football, was that they wanted to feel more involved with how teams play the game. “Their favorite thing is not the big hits, not the athletic plays, it’s the strategy,” Schwartzstein says. Amazon’s definitely still focused on big hits and athletic plays. There’s now an automatically generated highlight feed for every game so you can catch up, like watching one of those “last week on” montages in a TV show, but it’s also trying to bring people into the tiny nuances of football, too.

The Defensive Alerts feature is a good example. Amazon trained a machine learning model on 35,000 plays from the last few seasons in order to teach it to automatically suss out whether a defender is likely to blitz on any given play. Where they line up, how they move before the snap, their body language — all that stuff matters, and it’s the kind of thing quarterbacks and coaches spend countless hours poring over in film study so they can see it coming on game day. Now, when you’re watching a game, the Thursday Night Football stream might highlight the player who’s just about to take off. “What [the broadcast] is doing is saying, now you can watch the defense the same way the quarterback does,” Schwartzstein says.

AI is at the heart of a lot of Amazon’s new ideas. The fourth-down metric, Schwartzstein says, comes from a model that understands which players are on the field, who’s coaching, the situation, and more. If you put the best 11 players in the world on a team, I ask him, would the model tell them to go for it every time? He says probably not; it’s still a risk. “But if you put a middle school team on the other side of the field, it would.” From replays to field-goal percentages, a huge amount of the broadcast will come from these models. 

All of this data will be in Amazon’s “Prime Vision” stream of the game, which is a secondary broadcast designed for the data-hungry fan. That’s clearly where the company is focusing most of its efforts. But the beauty of streaming is that there can be more than one way to watch a game; Amazon’s default broadcast is still going to be a fairly straightforward football show. It’s also bringing back the Dude Perfect dudes and the Uninterrupted crew to do their own broadcasts. “I think the goal is going to be to adapt to fan preferences over time,” says Jared Stacy, Amazon’s director of live sports production. “Hopefully we develop some technology and features inside of Prime Vision that wind up making sense to deploy in the main broadcast, but the main thing is just serving the fan in the best way possible.” 

The other thing you’re likely to see a bit more of on this year’s version of Thursday Night Football is… Amazon. For the first-ever Black Friday game (which is as perfect an Amazon cross-promo event as you’re ever going to find), the company is planning to go all out with shopping integrations and is looking for every possible way to integrate football with other Amazon products.

“There’s a chance to really build a new tradition with the league,” Stacy says. “What can we do with music? What can we do with food? What can we do with shopping?” He emphasizes that it’s just one game, on one day, and that commentator Al Michaels is not going to start talking to Alexa or offering shopping advice during games anytime soon. But there’s obviously plenty more integration to come.

You’ll also potentially be able to see everything just a little bit better: Amazon’s streaming its games in HDR for the first time. “It’s going to be probably the most broadly distributed HDR feed,” Stacy says, “given that we’re not constrained by broadcast affiliates or set-top boxes.” If your TV supports HDR, you’ll get TNF in HDR, says Stacy. Alas, you won’t get 4K. When I ask Stacy about this, he doesn’t quite answer. “Yeah, you know, we think HDR has the biggest impact on fans, and it’s most noticeable to the most number of people.”

Football isn’t the only sport Amazon streams, and many of the things it’s building for TNF will show up for Premier League games and others. But NFL football is the most expensive and most important piece of content in the US, and Amazon knows it. In its second full season broadcasting games, it’s going to keep trying to do the job better — and to see what else it can do with the full power and breadth of Amazon behind it. From AI to Whole Foods to The Lord of The Rings, that road is long, and Amazon is still only at the very beginning.