Welcome to our first hands-on workshop! Today we are going to look at a routine in-game
decision that faces head coaches in football every week – what to do on fourth down. The
analytics community has a strong consensus that NFL coaches (and football coaches at virtually
every other level) are much too conservative on fourth down, punting or attempting field goals
when they should go for it. Some of the key analytics research on this topic is based on a model
developed by Brian Burke of Advanced Football Analytics (now with ESPN). This model was
the original basis for the NYT 4th Down Bot, an automated system run by the New York Times
that would live tweet every fourth down play in the NFL including the actual call made by the
head coach and what the bot thought the coach should have done. The NYT 4th Down Bot
operated between week 13 in the 2013 season and the end of the 2016 season (including
playoffs) with the Twitter account @NYT4thDownBot.
The basic concepts behind Burke’s model are described in his post 4th Down Study on Advanced
Football Analytics (I’ve uploaded a copy to Canvas for your convenience), while a high-level
overview of fourth down strategy is included in the New York Times article 4th Down: When to
Go for It and Why (also on Canvas). Your task in today’s workshop activity is to describe
how you would design a new and improved version of the NYT 4th Down Bot (beyond
simply restarting the computer that used to operate the original bot). While you do not have
to program your own bot from scratch in the class time dedicated to this activity (although that
would be quite impressive and almost certainly get full credit!), you should describe in detail
what data you would use, how you would analyze the data, and how you would present your
results.
For data that you propose incorporating into the model, the more specifically you describe the
data and where you will get it, the better. To get you started, there is freely-available play-by-
play data for every NFL game since 1999 (including preseason, regular season, and postseason
games) from the R package nflfastR, which also provides estimates for expected points and win
probability at any point in any game in the dataset. You can assume that this level and type of
data is easily accessible when designing your system. Note that estimates for expected points and
win probability are based on what NFL coaches actually do rather than what they should do in an
ideal world and do not account for differences in team strength (i.e. which two teams are
playing). You may want to incorporate additional information such as the quality of the two
teams (perhaps separately on offence and defense), quality of each team’s punter and
placekicker, and any other factors that you feel should influence coaches’ fourth down decisions.
Think about how you can determine these quality measures, if you choose to propose their
inclusion.
As you develop your analysis plan, consider carefully (and include in your write-up) factors that
influence the design of your model including how to weight recent vs. older data, combine data
from multiple teams (or why you might not want to do so), address situations where you have insufficient data (e.g., a rookie kicker or early weeks in the season), and game context (score,
down and distance, field location, time remaining). For factors such as field location, consider
what level of granularity is appropriate (exact yardage, five-yard bins, variable depending on
where on the field, etc.). Wherever possible, indicate how you can use data to help inform some
of these design decisions, rather than merely using data in building the actual model.
Your submission is due by 11:59PM on Friday, September 3rd
. That said, the assignment is
designed to be completable during class and you should feel comfortable submitting during class
without continuing to refine your write-up. Using the additional time is entirely discretionary, if
you feel that you would like to further improve your submission.
