Hey, everybody, and welcome to life is a gamble. This is episode
number eleven and my guest today is Frank Frigo, who is one of the
founders of a company called Zeus. And I believe you guys were the first
to start modeling NFL football, weren't you? Or well, we were one
of the early ones. And just for a small clarification, Zeus was the
original name of the simulation model that we developed. Chuck Bauer, a physicist,
and myself. The company that we found it was Edge Analytics and subsequently
Edged Sports, which is an analytics company that did a lot of work in
sports, but also in other industries like healthcare and education. Okay, so
when did you guys first come up with this idea and how did it come
about? Okay, So I think the way the best way to describe how
it started was me as a football fan watching games and being convinced without solid
proof, that coaches were risk averse in the NFL and were probably giving up
a lot of value. And I I knew as a backgammon player, as
a games player in general, that it's human nature to be you know,
it's a standard cognitive bias to be risk averse, and a lot of people
don't really think probabilistically and in terms of expected value, and that seemed to
be a parent with coaches, particularly on what we always referred to as directional
choices, you know, fourth down decisions, these binary types of things like
these, you know these critical moments when you're deciding between two very distinctive paths.
Do I try to retain the ball score additional points or give it up
on a punt or or subtle for a field goal? Do I go for
two or one on the two point conversion? Do I do an on site
kick or a regular kick? And it seemed like coaches were airing on the
side of being conservative, but didn't really have anything to prove it other than
just my general intuition. So it was a problem I had been mulling over
in my head for a while, and I was pretty familiar with programming achievements
neural nuts in backham and in particular and in other games, and was kicking
around the idea of how could we build a model to actually test this because
there were others out there that I think suspected that coaches were giving up value,
but nobody was really measuring it. And I was trying to think about
it in more of a commercial context, that wow, if you could actually
prove this and show mathematically that it's costing them, this is a commercial opportunity.
So I didn't quite knew how to go about it. I started,
I was doing various kind of back of the envelope calculations and so forth.
And Chuck Bauer, who's an experimental physicist and a good friend of mine and
also a very good backgamming player, was also familiar with a lot of the
work that had been done in modeling games, and we were talking about a
way to go about this, and to make a long story short, we
developed a simulation model. We basically put together the logic of the game,
the way teams typically behaved. We had a certain amount of data accessible to
us, so the early version of the model was a way to play the
game from start to finish, from the opening kickoff, running clock, penalties
and all for two generic league average teams. And what it allowed us to
do was through a Monta Carlo simulation. Once we had written out all of
the logic, we could set up any unique situation in the game and allow
it to play to conclude and the variables that define the game state where you
know, who possesses the ball, wears the ball on the field, down
distance, score, clock, timeouts, remaining, all these things, and
we could do it. We could set up a situation either opening kickoff,
any stage of the game and let it play a conclusion and estimate the win
probability for the teams. It was kind of a crude approach. It was,
you know, these were league average generic opponents, but we were able
to back test it against sort of known game states where there actually was a
lot of data. So if a team was leading by seven points at the
half or the end of the third quarter, we had some benchmarks that we
could say, how does that compare to what our model's producing through the simulation,
And if we're capturing those points reasonably well, then chances are we're probably
capturing some of the in between points pretty well too. So that kind of
got us started and many year Oh gosh, I was kicking this idea around
really in the late nineties, but I didn't engage Chuck until about two thousand
and one. I think we had talked about this idea. I told him
I was thinking about it, and I was living in Louisville. At the
time, he was living in Bloomington because he was working as a research physicist
at IU, and I knew Chuck knew how to do simulations as a physicist,
So I went up. We met at a coffee shop in Bloomington.
We sat down, I mapped out this idea that I had. He sounded
he was a little lukewarm on it. I think he thought, I go,
oh, God, this is going to like, you know, I'm
gonna go down the rabbit hole here. So I don't think he was sure,
which I was, yes, and I did, but I think he
was kind of curious about it. And we got off to a pretty slow
start. Thingsn't really happening. And then I remember I followed up with him
and said, are we going to do this thing or not? And his
response, I think at the time, was I'll tell you what, if
you write up the basic logic of coaching behavior, you know, down distance
and so forth, we'll take a shot at putting a simulation allel together.
So I did. I mean, I literally got paper and pencil out and
I came up with kind of a flow charts, and again, wasn't I
wasn't trying to construct logic of optimal behavior. I was trying to construct logic
of common behavior if a coach is in this type of a situation. These
are typically short passes some percentage of the time. I did have some data
that I was able to drop on, so I came up with some charts
for the basic logic of the game, and that was the original basis of
the of the Monte Carlo simulation. So we were able to get one off
the ground. I can't remember how much time that took and testing and retesting
and so forth, but we got to a point where we were pretty comfortable
with a simulation model that could actually play a game to conclusion. And I
think my original estimate might have been that coaches were giving up somewhere around a
half a game a season on average. I think that might have been my
original thought. Turned out the early version of the model revealed they were worse
than that. It was actually more likely to be closer to a game and
in a sixteen game, which was a sixteen game season at the time,
that's a lot. Yeah, we're looking at that going, wait a minute,
this is a multi billion dollar industry. You know, the lowest NFL
franchise is probably at the time was probably worth two billion dollars, and we
can mathematically show that with better decision making, they could get an extra game
a season. You know, if you're a nine to seventeen team and we
can get you to ten and six, it might get you into the playoffs.
Like they're going to roll out the red carpet for us. This is
really cool. And I came from a background as a commodity trader and the
energy markets, and you know, if I would have walked into a hedge
fund manager with this good of a of a value proposition, I mean,
you know, I just stood on the boardroom table and beat my chest right
and they'd have thrown money at me. So I'm thinking, this is what's
going to happen in the NFL. So you know, there's nothing like a
little naivete and ignorance to get you motivated to try to do something. So
I was very motivated to and I was working a day job at the time
to reach out to NFL teams. I mean, I will tell you it's
amazing. If you pick up the phone and call the front office and just
ask to speak to somebody, you'd be amazed. At how often you can
actually get a general manager, a head coach to call you back. Wow,
that is shocking. It happened way more than I thought. Maybe I
was a little more clever about it than the average person might have been in
terms of, you know, calling, trying different phone numbers and techniques.
But we were able to get audiences. Our first audience was with the Cincinnati
Bengals and Marvin Lewis the head coach. They had a player personnel was there,
the defensive and offensive coordinators. Now what year was this. I think
this was around two thousand and four. Maybe I'm not going to get all
my dates perfectly right here, but that was the first legitimate NFL meeting we
had. You know, I have to say I met you through my brother
Jake, who's also a backgamvid player, and I remember at the time Jake
telling me like, yeah, you know, Frank and Chuck they built this
model for NFL And that was really kind of the heyday of offshore internet gambling.
You know. It was before the UIGEA went and they were throwing money
at people and it was and I was like, oh my god, these
guys are going to be fortunate. He's like no, they don't want to
bet. They want to sell it to the to the teams. And I
was, you know, just I just was shocked, going, what are
they crazy? Like? But yeah, okay, so now you're expecting them
to just you know, bring out wheelbarrows full of money when you when when
you come to them. Yeah, what happened? You meet the Bengals first,
Yeah, we meet the Bengals first. And you know, there was
a there was a particular quote in that meeting that was a real eye opener
for me, which is which is at the end of the meeting, Marvin
Lewis, who was a longtime head coach of the Cincinnati Bengals. You know,
he was very nice. He said, this is super interesting and he
said, you know, and you guys may well be right. You might
be right that we're being too conservative in these situation and it's costing us.
He said, But you're not the guy that has to go talk to the
press on Monday morning. I am. And you know, you're not the
guy that has to look like the players in the eye. You're not the
guy that has to, you know, talk to the owner group about why
I did something that was counterintuitive and worked or didn't work, and so that
was the first time we had we were really able to sort of peer into
this culture, this curious culture of the NFL, which made us realize,
hmm, maybe it's not all about win optimization. Maybe there's other metrics at
play that are affecting these decisions. Well, more risk aversion and yeah,
and more protecting my job exactly exactly. I mean. So, one of
the guys that we one of the biggest proponents we had and supporters that we
had very early on was a guy by the name of Jim Schwartz. And
Jim Schwartz was at the time we met him, the defensive coordinator of the
Tennessee Titans. He had us come down to Nashville. He met with us
at the NFL combine. We sat in his hotel room and he was like,
this is so cool. I'm going to be a head coach someday I
will use this. I hope you guys don't sell it before I become a
head coach, because this is going to be amazing. And here's who you
need to talk to, and da, da da. He was like super
supportive. He claimed he was a chess player, you know. And then
when I asked him what his elo was. He's like, what's an elo?
And then I knew he wasn't really a chess player. But you know,
he's a fairly smart guy, and true to his word, he became
a head coach. He ended up getting the job, the head coaching job
at the Detroit Lions. And guess what, when he became a head coach
with the Detroit Lions, he wasn't the same guy that he was in that
hotel room because he had just signed his five year, multimillion dollar contract and
damn if he was going to go before the Lions fans and have to explain
why he did something counter intuitive and it worked or it didn't work, and
you could really see it, right, I mean, from his personal risk
management. It was very different than about just optimizing team wins. It was
about how do I make sure I've got enough. I'm good. You know,
I got enough in the kiddie for my life and I don't need to
shake that up. I finally got this job, and you could kind of
see. Now that being said, he did take our reports and we met
with him at the Lions and his staff and so forth, but he wasn't
quite the receptive guy we thought he was going to be, and interestingly enough,
before he got that, just to go back a little bit, before
he got that job with the Detroit Lions, he was one of the leading
candidates when an opening came about with the San Francisco forty nine ers, and
we were at the time in with the forty nine ers, you know,
with Terry Donnieu, the GM and a guy by the name of Paraguamate who
was an ex Baine consultant who was basically their head of analytics and player personnel,
who was a big proponent of what we were doing and literally wanted to
hire Schwartz because he felt Schwartz was going to subscribe to this kind of thinking
and this would be a great package deal. Turned out the York and the
Bartolow family to Bartolow family ended up hiring Oh, what the heck was the
guy's name, Nolan, I think was his name instead of Schwartz. But
I remember on the eve of the interview, Schwartz calling me saying, Hey,
I'm going to be talking to them all about zeus because he really wanted
to pitch himself as this analytics guru and the whole thing it was part of
his resume, and you know, we were trying to help out and prime
the whole thing up, and he ended up not getting the job, but
he later got the job with with Detroit. And what it made you realize
too, was that these coaches wanted to talk the game of analytics, but
not necessarily a ply it. You know, I sort of liken it to
the person that you know, has the fancy piece of fitness equipment in the
corner of the room because they want all their friends to think they work out
every day, but they never actually use it, you know. And I
we subsequently realized that that that was in play as well. They wanted to
check that analytics box and drop names, but they didn't necessarily want to,
you know, have the courage to actually take it on to the field.
So that was another thing that we were dealing with. I don't remember when
the whole moneyball thing was happening in baseball, was that. Do you remember
if that was pre this or after this, because it was the exact same
problems that you were facing. Yeah, I think so. You know,
Bill James was at this saber Metrics early on. He'd you know, he
was working out of his garage or basement or whatever doing his thing. And
then of course, you know the story with the movie and the Michael Lewis
novel about you know, how Paul D. Podesta and you know, Billy
Bean got together and really kind of shipped things up with the A's and so
forth. I don't know all of the exact years or exactly when the book
and movie came out. It over it definitely overlapped our time because after the
movie came out and Paul D. Podesta, who was he was really the
brains behind the thing. He subsequent you know, he subsequently got the GM
job with the Dodgers. You know, this young Harvard kid, and he
cut his teeth in football, by the way, I think he worked with
the Browns before he worked in baseball with the A's. We met with him
at Dodger Stadium, and I mean I can remember distinctly walking down the hallway
and getting looked up and down by Tommy Lesorda, giving Chuck and me a
look like, all right now, who's deep to talking to you now?
Because he was doing all these controversial things with the Dodgers, just like is
portrayed in the movie with the A's and the old school guys didn't like it.
You know, when things weren't going well, they wanted to blame the
analytics guy. And when things were going well, you know, some people
were crediting him. And he was a pretty humble guy. I mean he
told us behind closed doors. He's like, He's like, yeah, they
want to hang me. When you know things aren't going on, he goes,
and then I get credit. When things are going on, he goes.
And a lot of this might just be randomness. It doesn't happen that
fast, you know. But it was a fascinating discussion with him because he
did sort of make connections to moneyball in baseball and what we were doing in
football, and he was really impressed. We gave him a little demo.
He was very impressed with what we had done and was talking about the nuances
of football analytics versus baseball analytics, and there is some there are some really
clear differences. In baseball, you're basically trying to produce runs and keep your
opponent from producing runs, and that correlates really well with wind probability. In
football, it's much more contextual. You know, you get into the utility
of points, which is actually quite frankly a lot like playing tournament backham and
because the clock is decaying, because there's these different scores late in the game,
not all points have the same value, and football has the added piece
that you score in these different increments. So it creates some really difficult problems
about how does scoring and retaining the ball and you know, extending a drive
to try to score a touchdown versus taking a field goal or punting, how
does it affect win probability? Well, our simulation model was really well suited
for that, and so we were drawing comparisons with de Podesta about, you
know, what he had learned through the whole moneyball movement and baseball modeling.
And one really funny statement that he made at one point was he was encouraging
us to keep after but he said, he goes you know, he said,
but we he goes what we learned in baseball early on with Sabing metrics.
As he said, outs are bad. You know, basically you just
don't volunteer outs, you know, and you know, as a fair point,
so but you know, they they they were ahead of the curve in
baseball for sure. I mean they they got out there early on and baseball's
got one hundred and sixty two game season, so there's a little more time
to sort of test ideas and see subtle effects and so forth. But you
know, it's it's more of an abstract idea in football because you got a
shorter season and you're trying to convince somebody like, you know, you actually
win the game like ten to fifteen percent more often when you make this particular
play choice on a fourth down, but there's a lot of variance. You
know, you don't always see the result immediately, of course. Yeah,
and that's a that's a hard thing for some of those guys to embrace.
Yeah, so when did did who finally bought in and said, yes,
we get it, we're going to use it, and then and then actually
did so. The first people, the first people that actually wrote us a
check in the NFL where the Jacksonville Jaguars they had read they had read a
little bit because I used to do it. I used to publish some sports
analytics for different sports media. I wrote. I wrote regularly for The New
York Times. I used to do stuff for Fox Sports and a bunch of
others. So that the Jaguars had reached out to us, and they were
they were trying to put together some strategy around overtimes and they asked us if
we could help them with it. And it was a pretty pretty small paid
gig. The first substantial, really substantial client we had was the Eagles,
and they were particularly particularly open minded and they had a culture and it really
came down from Jeff Louri and Jeff Louri was like best buddies and neighbors with
Jeff Yazu's the founding partner of Susquehanna, which is an extremely innovative private partnership,
one of the pioneers of options trading and biggest equity options market maker in
the world at the time. And I used to work for Susquehanna, so
I knew Jeff, so I had a bit of a connection to get to
Jeff Lourie, the owner of the Eagles. And Jeff Lourie is a very
smart guy. I mean, he gets analytics and he was the first we
had. You know, we were knocking on lots of different doors, but
with the Eagles, I mean he sat in the meeting with us and Howie
Roseman, the GM was there and Alec Hallaby, the head of analytics was
there. Doug Peterson at the time was not there, who was the Super
Bowl coach in seventeen, but you know, Lourie was at the table asking
all the questions, and so what you saw there that was really important was
that you could get a you know, you could get a head coach to
do something if the GM had their back and if the owner had the GM's
back, and if the head of analytics was in the room. And the
Eagles I think took that approach because I think Laurie basically told Peterson, do
what's mathematically correct. I'm not going to second guess you. You know,
don't worry about the press on Monday morning, like Marvin Lewis had said,
just do the right thing. If it's if it's mathematically defensible, do it.
So they were really the first ones that kind of embraced it, but
even they I think slow rolled it a little bit. That Super Bowl season.
We worked really closely with them. We worked with them. We were
a paid client of the Eagles for multiple years, and we were very integrated
with their analytics staff. So we would because there were NFL rules restrictions,
I should make this point, you can't actually process information during the game and
run live simulations. So they would ask us to in advance run hypothetical scenarios
and create a book, a thick book of that they could navigate. You
know, oh, what's the score difference, what's what's the range of the
time, We're in the middle of the third quarter, We're in this section
of the field, and they could be it served as a guide, so
they could use that, and then they could have a guy on the headset
with the head coach or the offensive coordinator say, look, you know it's
first down. If we pick up seven yards in the next three downs,
we know we have a fourth down go. They already in advance knew that
from the model, and that allowed them to even dictate their earlier play.
And there was a lot of, you know, very strong game theoretical pieces
to this, because now they were operating a little differently than other teams.
You know, if it was third and eight and they knew fourth and two
was a go, was mathematically correct, they could go out and you know,
instead of trying to pick up the whole first down like conventional wisdom would
suggest, they might run a pass play that gets some six yards and then
immediately without calling a timeout, line up and go for it on fourth and
two. And that's what they were using very effectively. So not only were
they doing the right thing on fourth downs, and they and they were building
plays around that they knew go leading up to it that they could they could
use that to their advantage as well. So they were getting a lot of
value out of it. And did the media notice that they were doing things
differently? And and what did the media and the talking heads have to say
about what was going on? They did. And in fact, Peterson would
go to press conferences and he would get asked about some counterintuitive decision that he
made and he'd say, it's what the analytics told us to do, and
we're following it. You know, he was saying those kinds of things.
So, yes, the media was catching on to it. And the real
sort of crescendo of that was The New York Times ran a feature article on
us on Super Bowl Sunday when the Eagle the day the Eagles won the Super
Bowl, the New York Times ran a feature, full page article on you
know, how the Eagles are following the math or I can't remember the exact
name of it, but you could look it up. They were basically saying
that, you know, if you've noticed, if you've noticed that the Eagles
don't behave like every other NFL team and are doing you know, kind of
head scratching things and so forth, there's a reason for it. There's this
little company in Louisville, Kentucky that built this technology that the Eagles are using.
And then that night they go out play the Super Bowl win do all
these things that were very different, right, So you know, in the
game, they did all kinds of really interesting the kinds of things I'm describing.
And when we when we ran the analysis on that game, they followed
us to the t. I mean it was they they had like a zero
error ray by our judgment. I mean they were they just played it.
They played it spot on, and and these decisions we could measure them.
I mean they're not trivial. There are there were games where NFL teams would
give up a quarter a game of expected value, a half a game of
expected value, like it can happen, and they were they were spot on
in that game. So that was really like a key key moment for us.
We got this publicity, we got it in advance and they go out
and do exactly what the article talked about. So then all the teams showed
up with wheelbarrows full of months still didn't happen. We ended up, you
know, it was known those people talk. You know, it's a big
fraternity in the NFL. We ended up working with I think we ended up
with about twelve I think twelve paid clients in the NFL. Over time.
We had two Super Bowl champions, the Kansas City Chiefs, in both of
their Super Bowl wins used our model. I used to consult. I'd be
on the phone with their head of analytics sometimes on Saturdays, and he'd ask
me, if we encounter the following situation, how would I use the model
to kind of, you know, decide what we should do. You know.
And it wasn't just about fourth downs either. Sometimes it was about you
know, penalty acceptance or you know, what to do on a two point
conversion. You know, we've even we even had teams ask us about things
like when did you take an intentional safety? I mean, that's a that's
a strategy, right, and we could shed great light on that. Funny
enough, there's actually been a lot of talk in the NFL about changing they
have changed the overtime rules for the playoffs in the regular season, but there's
been some talk about a very creative solution to overtime called pick and choose,
where you flip the coin. One team gets to decide on possession and the
other team gets to decide on where they place the ball on the field,
which is a really kind of a unique idea. And we even had the
committee in the NFL, the Rules Committee, reach out to us and ask
us our opinion on the optimal optimal strategy for doing that and how they how
you would go about it. So we've been involved in a lot of different
aspects of the game, and but yeah, we ended up working with about
a third of the teams. One of the problems with the business model in
the NFL too was I mean, first of all, you know, it
wasn't a huge revenue generator for us. It was great for PR for edge,
but often what we would find is the teams would license our technology and
they were trying to build their own internal tools. So I think sometimes they
were using our model and then using it to calibrate a lot of their own
internal systems. Ah, you may use it for a year and go Hey,
thanks a lot. That was great. We're good to go now,
you know, And analytics has become so commonplace now in the NFL, or
at least what they call it analytics, and you know, every team's got
an analyst. You know a lot of times they hire some kid out of
an ivy league school. But even so, aren't they still playing really badly?
I mean, I shouldn't say badly, less than optimal by a fair
margin. There's still a lot of less than optimal going on. I haven't
measured it very closely this past year, but we did see an improvement.
I mean, there was kind of an inflection point a couple of years ago
where they really did start to get better at it. But I still think
I still think there's a way to go. I mean, I feel like
there's you know, the word analytics is used in a very broad sense.
Analytics is, you know, technology that wearables on the body. What's your
temperature and heart rate? And how do we keep you from getting a concussion
and build a better helmet? And analytics is you know, other player tracking
maximum speed, and analytics is how do we choose a player out of college
that has the highest probability of being a pro bowler in five years, and
what are the key characteristics that you can observe and tests before the draft that
are precursors to how they might perform down the road. Those are all kinds
of analytics. We were doing a very specialized part of analytics, which the
best way I would describe it is and I gave a talk on this at
the MIT Sloan Analytics Conference one year about win probability as a currency. So
this idea that, to use a poker analogy, win probability is like poker
chips. So if you think about it, at the start of a football
game, if you had two evenly matched teams, they both have fifty chips
of win probability, and it's a zero sum game. So at any moment
in time, they're making decisions, they're executing, they're trading those chips.
If one team gets to sixty, the other drops to forty. And of
course the goal at the end of the game is one one side is holding
one hundred chips and one side's holding zero, and that creates all of these
unique risk management decisions along the way where you're trading win probability. Hey,
if I go for it, on fourth down and I succeed, I get
this kind of a boost. If I fail, I lose this amount.
If I take the conservative action and punt, there's a benchmark. What's so
I stand to gain this much, I stand to risk this much? What's
the probability of my success? And it's far more complicated than that, because
there's a there's all kinds of distributions of outcomes. It's not just binary.
Sometimes you get tackle behind the line of scrimmage, sometimes you get a long
gainer, sometimes you just you just make the first down, but you're effectively
trading win probability chips, right, and uh, that's all well and good,
but you got to be able to measure them. And how do you
measure it? Well, you need a really good simulation model, and that's
what we had. So it's kind of a two part thing. I mean,
one, you needed the technology to a really good way to measure not
just generic teams but custom teams in a matchup in a given game state,
what's their win probability in different situations? And then and secondly, you know
you need to know how to use it in a game strategy context. So
you know, yeah, great, you can measure what wind probability is in
all these different situations and how much it gets impacted by different decisions and execution.
But how do you use that as a strategy to make better risk management
decisions? And so that was a big layer of what we were doing as
well. We weren't just handing people the model. We were saying, here
it is, and here's how you use it effectively. So there was a
lot of handholding as well. Sure, now you must have had a meeting
with Bill Belichick. We did. We did so little known is that Bill
Belichick has a right hand guy by the name of Ernie Adams. They went
to college together. Ernie's a very bright guy. He likes to stay out
of the spotlight. And I haven't spoken to him in years, so I
don't even know if for what he's up to these days, if anything,
But Ernie was the guy that was always kind of sniffing around what was going
on in the analytics world. And I can't remember how we first came into
contact with Ernie. We met him at the NFL combine. I had multiple
phone calls with him, and he was just a really curious guy about anybody
that was doing anything. Kind of on the cutting edge, and he would
do things like, you know, ask us to look at all of their
decisions from the Super Bowl, or you know, look at our two point
conversion chart, you know, that cheat sheet that the coaches use on the
side. They had one that was I think developed by a professor at Rutgers
that they used to use, and one time they went to us and Ernie
did and said, hey, what do you think of this? And I
remember my response was, yeah, I mean, we largely agree with it,
and the stuff we disagree with it is like morsels of value, you
know. I mean, the thing about two point conversions is it's the obvious
ones that are worth a lot, and the really complex ones that are harder
to figure out are worth like, you know, fractions of a percent typically
or something. But you know, at the same time, I would say
to him, the real value is in the fourth down decisions like that.
You know, that's where you got like ten somebody's ten fifteen percent of win
probability at stake and people don't even know it. No one even talks about
it. So he was very curious, and we get a call from him
saying, hey, Bill would like to meet with you guys, you know,
are you ever going to be up in New England and we like to
meet with you and come to Foxborough. And so at the time, you
know, we were, you know, sort of bright eyed. I don't
remember what year this was exactly, to be honest, but you know,
I had met Ernie before, but this is like, hey, this is
Bill Belichick. This guy's got a bunch of Super Bowl rings. We better
get on a plane and go meet with him. So Chuck and I went
up to Foxborough and met with them. We went into their you know,
their film room and met with Ernie and we were talking to him for a
while and he's like, well, Bill's going to come join us here in
a little bit. And you know, he comes walking in through the back,
just like he'd sort of expect him, you know, with the hoodie
and kind of shuffling along. He doesn't like to smile much, and you
know he, you know, he just had this attitude like he was going
to be in the room for ten minutes and then tell us to go go
to Hell or whatever turned out he was. He was very curious. He's
like, these guys came up here, I'm gonna get as much information as
I can't. We ended up spending four hours with him, I remember it
well, and they went over Super Bowl decisions and all kinds of different decisions,
and you know, I respected him as a coach. You know,
it's hard to argue with that many wins. I mean, obviously he had
some great players, but I was never I mean, I think at the
time we thought he was better than an average risk management decision maker in the
game, but that wasn't a very high bar at the time. And he
had done a very controversial decision in the playoffs against the Colts where he went
for it on a fourth and two in his own territory and failed and then
Peyton Manning scored a touchdown. He was he was barbecued over that. In
fact, I ended up writing a big feature article for The New York Times
on that decision. It was called the Anatomy of a fourth down decision at
the time, and I defended Belichick. Actually, so, you know,
we met with him, and you know, we were hopeful that, you
know, maybe something would come of this, and it became apparent that they
were you know, it was a big sort of fact finding sniffing session and
they were going to try to get as much information as they could. And
there were two particular moments in the meeting that really struck me. One that
I kind of said, hey, that's pretty impressive, and the other that
I said, wow, that's pretty disappointing. The impressive one was is that
at the time, this was before they moved the kickoff line, so before
they had all the concussion protocols in the NFL. You know, there used
to be a lot of runbacks on kickoffs and those are high speed collisions,
right. That became pretty controversial and they changed the rules and now it's like
every kickoffs a touchback almost, although that's starting to change a little bit.
But at the time, an average NFL kicker was probably only getting a touchback
rate. I want to say, it was like between fifteen and twenty percent
or something like that. And we had done an interesting study which was to
say, what if you had a kicker on your team, you know,
somebody was just a massive leg that wasn't very accurate for field goals, but
they could kick it in an unusually high touchback rate. So we did a
study with the model is an experiment to say, what if you had somebody
that could take you from a twenty percent touch back rate to one hundred percent
touchback rate. Imagine this person exists, where you could go find them or
train them or what have you. What would that be worth to an NFL
team in season expected wins? And we ran the analysis on this and it
was like, Wow, it was really something, and I think I think
at the time we had concluded I might be getting this just a little bit
off. But I think at the time we concluded it was worth about a
game and a quarter a season. Wow, that's cute. Yeah, if
this person could exist. And we posed the question to Belichick, and I
remember he kind of just, you know, sort of looked off into the
corner and was thinking about it, and he goes, I think he said,
so at the time he's he gave an answer that was within a quarter
of our answer. I think he said either one game or a game and
a half or something like that. And I was pretty impressed by that.
And that wasn't a number he pulled out of the air. He had clearly
thought about this problem and and actually probably put some pen to paper on it,
or had one of his staff or Ernie did it. So that was
pretty impressive, right, because those are the kinds of studies, those are
the kinds of hypotheticals we could do. We you know, we also did
one one time where we were like, what if you had a kicker that
could kick a fifty to fifty jump ball as an on site kick on every
kickoff, so you had a fifty percent chance of recovery. Yeah, that
was worth over a game the season. Right. So those are all kind
of really interesting little problems. But as much as Belichick's response to that was
impressive, a little while later, we were demoing the model and I was
giving just a very basic I was kind of setting up the framework of how
the model works, and I was giving a very basic example that no head
coach would get wrong, which is, it's the last play of the game,
you're on the opposing two yard line, you're trailing by two points.
You kick. Everybody knows, you kick a field goal. You could go
for the touchdown. You might get it half the time. When you succeed,
it doesn't mean you did something smart. And when you know and if
you miss the field goal one percent of the time or two percent of the
time, it doesn't mean you did something stupid. I was trying to give
the example you don't play results and how our model. You know, no
coach gets that wrong, but our model handles much more complex situations in the
same way. So I was I was using this as an illustration, and
he stops me, and he says, wait a minute. How do you
know that you get in fifty percent of the time if you tried to get
a touchdown from the two yard line, you know, like a two point
conversion rate. Basically, I said, well, coach, you know that's
just there's empirical data that supports that on league average. And I said,
I'm just using it as an example. We can use whatever number you want.
I said, I'm just using it as for illustrated purposes. And he's
like, yeah, he goes, you know, sometimes I'm one hundred percent,
sometimes I'm zero percent. I said, what I mean, I you
know, I'm thinking this is the guy that's supposed to be the genius of
the NFL. Did he really just say this? You know, I remember
leaving the meeting and thinking I really missed an opportunity for a zinger. You
know, I should have said to him, you know, hey, coach,
the next time you're one hundred percent you're gonna make it from the two
yard line. You know, you can lay me, you know, some
pretty good odds on that, and I'll be happy to you know, or
you can give me some pretty good odds on it. You know, I'll
be happy to take it. Yeah. So, anyway, that was an
interesting meeting. He was kind of what you expected. Nothing really ever came
of it commercially with those guys. We talked to them a bunch, but
they were not one of our twelve teams that ever paid us to do anything.
Our best clients were the Eagles and the Chiefs. Basically, you know,
we worked with Miami, we work with Carolina, we worked with Dallas,
We worked with Pittsburgh, Jacksonville, Detroit, Carolina, the Rams,
there were quite a few. Okay, well, I'm going to take a
reef break for a commercial, and then we're going to come right back and
talk about who I suspect we're actually your best clients, which is when the
gamblers came to call. We'll be right back. Okay, we're back,
and so so, yeah, you have some teams they're using it, but
then people who want to make real money with it. By the way,
before we even get to that. I just have to say, it seems
crazy to me that they wouldn't just buy this because it's not their money,
right, the owner is paying for it, So why wouldn't you have it?
Even if you're not going to listen to it, you could at least,
you know, look at it at the you know what it says.
There's a few things there. I mean, one is their cheap skates.
Two is they're very protective. They want to do everything in house. And
I'm sure at some point in time if I would is said hey, I'll
be your house analyst, they would have offered me some very modest salary to
go work in house for them, which I would have had zero interest in
doing. You know, also, there's a little bit of a fear of
what you don't understand, I mean thinking and thinking in expected value and understanding
variants it. That's a very foreign concept to a lot of people. And
you know is as Terry Donnaue, the GM of the forty nine ers,
once said to me, he's you know, your biggest challenge in dealing with
NFL teams is you're dealing with a bunch of pe majors. You know,
they don't necessarily that's a great line. They don't understand math, and I
think he's I think he's right. So, you know, there were a
lot of things going on there. I think also these these head coaches,
there's egos. There's a little concern they're going to get exposed, you know,
and they're going to relinquish some level of control. And we always tried
to come at that with saying, look, we're not here to be some
robotic replacement of what you do. There's what you do as an art form.
You know your players, you know their motivations. You not to design.
We're not telling you what plays to design in which situations. We're just
telling you that directionally when to take more or less aggressive actions pays big dividends.
And then once you've got the information, you know, do what you
please with it. But you at least ought to know, you ought to
at least be aware of it. So that was always kind of our pitch,
and I think it worked to some degree, iming teams you know,
would subscribe, but it wasn't. Again, it wasn't a big money maker.
So but then the gamblers found out about it. The gamblers found out
about us, they did and funny enough. I think some of them,
when they read or caught wind of what we were doing, had more confidence
in our abilities than we probably had. I mean, I think we always
believed we could shine a light on the relative merits of a play calling decision
and the difference in cost. But we were never really that confident that our
absolute numbers were better than market prices. I mean, let's face it,
you know these sports markets, the NFL, those are big markets. There's
a lot of money by a lot of sophisticated people being bet. Those prices
reached some kind of a fair market equilibrium, and that's a you know,
it's a really good representation of a wisdom of a crowd. And we we
just never thought, you know, with especially with Juice, that we could
ever really sort of overcome those numbers. We did think if there ever was
going to be an in running market live win probabilities trading, that we might
be onto something, because we were very good at all the nuances of the
of the in game game state, as the clock was changing, the ball
position was changing. But there really wasn't at the time a robust market for
that. There was a little bit in Europe and some groups dabbled in it
and used our information for it, but there wasn't a lot of liquidity.
But along the way there were some betting syndicates that said, hey, would
you guys be willing to just run reports for us? And our model,
quite frankly, had been getting a lot better. We were getting very good
at customizing teams on their attributes and how they had performed year to date.
So typically by about week five in the season, we could configure who we
thought that team was based on their prior performances. We could then take the
next game, make home field adjustments, and simulate from the opening kickoff based
on who we thought those two teams were. We could come up with a
distribution of outcomes that predicted not only what we thought the fair value of the
total and the points spread was and the money line, but also what the
distribution of that looked like the histogram of it. And this was very appealing
to betters, and with some encouragement, we were asked to produce reports.
And there was a time when we you know, there were a few different
people that would buy our reports, and there wasn't a lot of exclusivity to
it. But then when we got pretty serious about it. There were groups
that just wanted to basically said we'll buy it, but don't sell it to
anybody else. And this was before the repeal of PASPA, the Professional Amateur
Sports Protection Act, which you know, basically opened up the regulated market in
the US. So we were also treading very carefully because we were analysts,
we weren't sports betters. We weren't risking our own money. Quite frankly,
we knew that was illegal in the US, and we weren't going to do
that. And our attitude was, we're producing reports, we're simulating games.
People can use that for whatever purpose they want. It could be sports media,
it could be fantasy, it could be a team, it could be
a sports better It's their business what they want to do with the information if
they want to buy it from us. Terrific turned out that the of the
groups I just mentioned, some of the betting syndicates, saw the best value
proposition in that, and we're the most interested in having us do it and
lock us up. So we did have some very meaningful relationships along the way
where we would we would produce these reports, we would look at all the
games on the slate, typically beginning in week five, after we had a
little bit of a you know, a running average of some data for a
while, and we would simulate the games. We would simulate it under different
assumptions. I won't go into all the detail on that, but but you
know, one assumption might be going into this week, you know they're basically
well how they're how they're configured as a representation of how they've played in the
last four games. Later in the season, we might put a heavier weight
on recent performance, or we might make certain adjustments if a game was one
or lost on turnovers, you know, on key turnovers or something like that.
Then then also sometimes the clients would say, we're not sure if who
the starting quarterback is going to be. You know, run them as a
seventieth percentile passing offense, and run them as a fiftieth percentile passing offense.
Run two different scenarios. So we would run all these different simulated scenarios and
then compare it to the market price and basically say we think this is these
are the most mispriced games of the week based on these assumptions and these simulations.
Sometimes we might run six or seven different assumptions and every one of under
all of those assumption conditions, they all were off market, and that would
be a major flag. And then we'd look at that game and say,
okay, are we missing anything? Is there anything we don't know about?
No, hmm, this looks mispriced, you know, And that was those
were usually the ones that got the most tension. We didn't want to know
what they were doing with the data, how much they were betting, or
anything like that. It was basically we were just getting We were basically selling
a subscription. But it was a pretty handsome subscription. And you know that's
uh were you were you not tempted to be betting it yourself? Well,
of course, but but there's a there were a few issues there. One
is, there weren't any legal channels for us to do it unless we kind
of flew out and went to the bricks and mortars in Las Vegas. We
we recognized, you know, against traditional juice, based on what we thought
our edge might be, the amount of size we would have to get down
and risk we would take to generate that return, and you know, we
did the math on it. If there was somebody out there who had much
better who was much better at executing than us, on both sides and price,
and they were willing to pay us upfront guaranteed. I mean I did
the math on it. It just it was almost a no brainer just to
sell a subscription. Quite frankly and honestly, even if even if the market
was as it is today, I'm still not sure I would have done it
much differently because just you know, I know firsthand, I mean, even
in today's market, the challenge it is to execute for size, particularly in
the regulated market. So but yes, there was a temptation and you know,
to dabble there a little bit. But but I think it was a
much better structure the one we had. Yeah, but now since PASPAH,
you can pet and have access to other markets, So are you are you
out there making it better too? No, And I'm really not, not,
not with the process that we used to have. So one of I
guess the sort of advantages of our methodology and our technology was that we would
see in advance things that we thought were mispriced. And early on when we
were doing this, we would see somewhere I think like the early maybe it
was like eighty plus games out of maybe two sixty ish that we thought had
value in them. And we saw dwindle every year. You know, we
also thought there was mispricing between the money lines and the point spreads. There
was like there was conventional wisdom on those relationships. Our simulations were showing different
differently that conventional wisdom wasn't And then we saw that curve tighten up over a
couple of years. We could see it. We saw the number of plays
that we thought were mispriced start to dry up. So markets get smarter.
I mean, I know this from the energy markets, right, you can
find things that are you know that there's value and pricing. But it yeah,
we saw that, we saw the opportunities start to dry up over time.
Well, listen, I am running out of time, but I want
to thank you. This has been really interesting. I know my audience will
we'll find this all fascinating. So thanks Richard. I hope, I hope
we covered some good ground there. And yeah, it's been a lot of
fun. And you know, I continue to work in the analytics space and
in the sports betting space, so I've always kind of got my ear to
the ground and what's the latest going on in those areas. And it's been
a fun journey, all right. Well, that's it for this episode,
everybody. As always, you can reach me at Life is a Gamble Pod
at gmail dot com, or you can find me on Twitter at RWM twenty
one. Until next time at