Welcome to Fantasy Hockey Life, presented by fan Tracks. Here shus your source
of information and analysis to help you win your fantasy hockey league. Block off
hats, a step hit on, stay lock. Here's your host, Jesse
Sovier and Victor Nunyo. This see Hockey Live back once again. Jesse Severe
coming to you from the vantrack side and over there EP rings side zone.
Victor Nuno. How are you doing today, Victor? I'm doing awesome.
Jesse. I am finally not sick. I'm still dealing with a little remnants
of it, but overall feeling much better. I had my flu game.
Hopefully it went okay for people. Didn't sound too amazingly. But how you
doing, Buddy good Man? Good I am. It's December and that is
shocking to consider, but we are moving through the year. The hockey season
roll so on. You think you think of December as being relatively early in
the hockey season, but it's really not. We're quite a ways in at
this point, aren't we. Yeah, we're quite a ways in. And
at this point you're probably figuring out if your team is legit or not,
whether you are gonna be pushing your tips in or maybe retooling, so those
decisions probably either now or the next few matchups, are going to become a
parent, so I have to make those difficult decisions. But we are definitely
getting pretty close to Fantasy playoff time February ish, which is not that far
away. Yeah, absolutely, and that's that is that's coming up. Yeah,
we like to do our playoffs relatively early, so it is right around
the corner. You need things to be right around the corner. People don't
have attention spans. I don't have an attention span, man, Ever since
I got a phone, ever since you didn't have to wait in a line.
It's difficult for me to go through a six month season before playoffs or
five month season. Come on, man, let's move it on. Is
that, Victor? Have our brains been destroyed? Are we really going to
be able to keep playing something like fantasy sports on the long term if these
things happen. I sure hope so, because I enjoy it. But it
does make some other things difficult with some of the attungon span and some of
the other issues. But hopefully it's just more helpful than harmful. By the
way I throw these questions at Or, Victor has no idea what I'm about
to say, and even if it makes no sense, he reacts professionally and
with great insight all the time. You should appreciate that. About these intros,
We've got a cool topic to talk about today. We're going to get
into some data analysis, and unlike some of our conversations, they don't rely
on pictures. So we're gonna come back right after this. We're going to
blow your minds with a little bit of talk about NHL. Welcome you to
the show. A little bit of a change of pace. Chase thecullum of
EP ringside. He is a data genius. I don't use the word genius
casually. Maybe he wouldn't use it on himself, but I'm going to go
and call him a data genius who's going to have some great insights into maybe
some ways we need to change how we are thinking about NHL equivalenc Chase,
how you doing today? I'm doing absolutely fantastic. Great to be on here.
And how are you guys doing doing great? Doing great? I read
your substack. You have a substack. The substack has the explanation of something
called NHL Z that we want to talk about today and get into and I
guess this is an innovation of going beyond the NHL equivalency that Victor and I
frequently talk about here. Can you explain some of the problems that you're trying
to solve with your new model? Yeah? Absolutely? Should I explain NHL
E first, or do you think everybody will have be fairly well first?
Maybe? NHL E is just a very simple thing where it says, if
you can score this many points in the NHL or if you score the as
many points in the WHL, take it time as a factor. That's how
many points that would be worth in the NHL. Right, it's easypsy you
get a little factor. You guys have seen that chart before out there,
and so it's just a quick shorthand of if you're this good in this league,
how much do we have to water that down to understand how you'd be
in another league? Right? Yeah, yeah, exactly. NHL E I
think it's a genius concept. I've used it a ton in all of the
prospect the research I've done. But there's a couple different problems as it relates
to NHL E, depending on what you want to do with it. So
NHL E is a very good kind of base, right if you, like
you said, if you want to know, hey, this guy played in
the WHL, he scores one hundred points in the WHL, what do we
think he's going to score on the NHL next year. It's great for that.
It's as good as you're going to do. But something I wanted to
look into was league difficulty, which is subtly different than NHL E. E
measures how well points translate from league to league. So your hundred point WHL
player, but that is slightly difficult that they're different than actual league difficulty because
of a couple different things that I wanted to go through in the article.
And the first thing I noted was everything here is ERA adjusted, by the
way, So just to get that out of the way first. That'll come
into play later. But NHL E does not quite measure league difficulty because the
underlying distribution of scoring in the leagues will affect its NHL E translations. So
something like Patrick Bacon has really good NHL estimates and the Czech League ends up
as the third best league in the world and his estimates or second if you
don't count the NHL, and some people are surprised by that. The Chech
League is a good league. But one of the reasons why something like the
Czech League can look so good in NHL E compared to what people think is
people just don't score that much check league. And then in the inverse,
the AHL tends not to look very good because an individual HL point doesn't translate
particularly well, but scoring is really common in the AHL, so the points
don't translate as well because points are more abundant. So that was the first
thing I wanted to adjust for. And then the second thing is positional effects.
So a lot of NHL E models just treat everyone as the exact same.
There are presumably certain leagues where like a defender scoring a point per game
is going to probably translate differently than a forward. So those were a couple
of kids that's not been an NHL E doesn't account for the difference between a
defenseman and a forward in the scoring. It's just the multiplier factor won't necessarily
be equal for forwards as it would be for defenders. Is what you're saying
there right exactly. And then the final thing is NHL E. There's a
lot of math that goes it. It's a genius concept because of how simple
it is, but there's a lot of back end math basically an optimization problem.
But it takes a large sample for that math to work in its best
form. And that's great obviously, and data generally, the larger sample size
you get, the better. But something I also wanted to do was to
be able to look at different time trends. So the example that what all
inspired this was I got a comment when I was writing about Matt ve Mitchikov's
NHLE and people are saying, yeah, because of obviously the geopolitical situation in
Russia, a lot of guys aren't going there. It's like who otherwise would
Maybe some former NHL players and whatnot that are North American based would usually go
to the KHL that aren't so, Like the KHL is easier to score in
now than it used to be. So these time trends are existing, it's
going to really mess within a NHL E based model. So there's certain if
certain leagues are better or worse right now than they currently are, your NHL
E estimates are going to either over underrate the player's production based on that.
And the really difficult thing too, is there's both It's unlikely that leagues are
changing a ton in the short run, but it's also really unlikely that leagues
are completely static over time, and there's pretty much no way to intuit which
of those effects is winning out at any given time. So I wanted a
way to be very explicit about that, plus have error bars. So a
NHL E that estimate will tell you a WHL point is worth one point or
zero point one five NHL points, but there's no kind of error bars on
that, and anything statistical will have, whether you see it or not,
some sort of uncertainty measures. And with my measure, I wanted to be
very explicit about, Hey, these leagues are x close together, so the
difference is negligible. So I wanted to take a couple of the or a
couple of those problems with NHL E and try to solve for them with my
NHL Z. Yeah, so that there's a couple interesting things there. So
yeah, it's this is all intuitive. Leagues change over time. You can't
just take what happened in this year's league, this year's AHL and apply it
to next year's NHL. First of all, we don't know what it next
year's NHL is going to be but more to the point, Yeah, AHL
five years ago, when we're ten years ago, when some of our current
NHLers were in there, the AHL had a different score in environment probably than
it does now. So if you're working with how they translated, you're going
to have some problems. And so that makes sense to me, this business
of ranking league quality. I understand what you're saying is you can't just say
the WHL is worth point twenty seven. It's point two seven, it's twenty
seven percent as good as the NHL. It's that doesn't exactly work. But
I don't entirely understand what's happening here is the translation of points. The only
thing we're going for here, because when you're talking about league quality, I'm
trying to understand what the difference is between seeing this league is better than that
league versus this is the league where you can score more points being the equal
talent. Yeah. Absolutely, So it is still inferring league quality from scoring
rates. It's just taking a couple extra measures to actually to account for things
like those positional differences and which leagues just have like higher lower relative scoring environments,
and they can get pretty extreme. Actually, I think an average AHL
forward scores like fifty percent more than an average KHL forward. So when you're
looking how much the scoring tends to translate, scoring translates way better than the
KHL because points are just way more scarce. So a guy who scores twenty
points, those twenty points are just harder to come by. So if you
want to measure how good the league is, you want to account for how
good the player is within the context of his league. If that makes sense.
Yeah, But so another thing with that, like with a KHL and
I We're going to talk more specifically about the KHL later in the episode,
But any data model you have, you and I I've done enough data stuff
to know the biggest problem is what friggin data you can get and how clean
it is. And one of the big things with some of these overseas leagues
is the number of minutes these guys play and therefore the usage that they get.
How do you translate what somebody who's in the finish or the KHL league
gets as two minutes he gets on the ice as a fourth liner and apply
that to the type of scoring he would get if he was put in a
scoring line and in a higher level quote league. And then can you even
get the time on ice? I can ever find the daying time on ice
for some of these leagues, So I don't know how you're doing that.
But you can say somebody gets five points in the KHL, they might have
played a cumulative thirty minutes to get those five points as opposed to what they
get in a higher level leagues. How do you account for that type of
difference. Yeah, so you're absolutely right. You're entirely at the mercy of
the data you collect, which sucks, but it is what it is,
and I don't have time on ice. I use explicitly points per game,
but there's a number of different things that I think work to the advantage of
that, the first being it's not necessarily just a prospect base, so I
use I apply the model to prospects, but everyone who plays KHL games is
eligible for the model. Everyone who plays LEGA games is eligible for the model.
The small ice time will happen to some of them. But you can
also have the inverse problem, and we're tracking when players go up and down.
So if a guy goes from the LIGA to the NHL, that's in
there. If a guy goes from the Liga and he gets kicked down to
the Maytee I think is the second tier league in Finland that will also be
in there. So the idea is, since there's hundreds of thousands of these
transfers, that the kind of ice time airs will average out and we can
get a generally good idea of the league quality even without ice time. And
then the second thing is, I'm not one hundred percent sure accounting for ice
time would be the best idea even if you want, even if you had
it, because points per game is pretty highly correlated with like war measures or
any sort of total hockey rating value at the NHL level. It's presumably true
at every other level too, but it does obviously miss things, but time
on ice will help capture a lot of what it's missing. So the guy
just sucks defensively, coach isn't gonna play him as much. Stuff like that.
So if you the time on ice ends up, in my opinion,
not being as good of a defense as as possible, because there is signal
in that time on ice too. So the guys, oh, he went
to the KHL and he's not scoring as much because he's not playing as much.
Part of the reason he's not playing as much is because the league's way
better. When you're trying to measure league quality, I'm I don't know because
I don't have time ice, and this could just be me giving a very
self serving explanation of flyay a flaw in my own model is not a flaw,
but I'm not entirely sold accounting for time on ice would help that much
because they're signal in the changes of time on ice as well. The ability
to earn minutes is probably the most important thing, to be honest. So
this is so great, Jason. Obviously, as you're pointing out, there
are some issues with inferring lead quality, and I don't know that there is
a better measure. I think that it's helpful. So I'm wondering what kind
of we always have biases or blind spots. So, assuming we're gonna use
something like NHL E or Z for a league quality inference, even though that
may not be perfect, what kind of issues do you think what kind of
over underestimations might we be falling into? Because I think it's useful to some
extent to look at prospect When you're talking about Mitchikoff as your previous example,
I think you can clearly say that the playing in the KHL as a draft
eligible is certainly more difficult in playing and say the BHL or even the USHL
or probably the CHL. But at some point the line becomes a little bit
more blurred. But it is helpful to have some idea of that, even
if it's not perfect. But what kind of over under estimations or issues might
be running into if we play a little too fast and loose with that as
a comparable. Yeah, that's a very good question. And I think one
of the problems you can run into is these are all averages, and like
I've tested, the averages are predictive even at the individual level. But the
thing I say all the time is don't fire scouts, because if you're trying
to apply macro level predictions to micro level individuals, like there is a lot
of context there that is missing, and we do have to be aware of
that. Even though, like I said, average, I'm not sold that
ice time would be a huge help, it may matter in specific circumstances.
You're basically assuming, by using clients per game, the coaches know what they're
doing. I think that's a good assumption. On average, but that doesn't
mean, for example, every coach knows what they're doing, so there could
be a lot of basically niche circumstances at the individual level that can make some
of these inferences difficult. When guys do transfer over, would be the primary
thing I would look out for, if that makes sense. Are we factoring
agent of this because you can say that the league is better or worse,
But obviously you drop Patrick Kane into the HL, his translation is going to
be a little bit different than some guy who's coming up and maybe playing the
same minutes. You can't account for the fact that he's only nineteen coming up
there or something like that. So do you factor that into the model?
Yeah, I forgot to mention that at the beginning. That was another thing
that NHL models won't be accounting for that I do have in the model.
So I have an aging curve built into the model that tracks that allows for
younger players to be expected to get better, because that's another problem when you're
tracking scoring of OHL players who go to the NHL. But if you're twenty
turning twenty one and then you transfer, you will tend to get better just
because you're twenty turning twenty one, like the natural age progression of certain leagues
will help boy their NHL E translations, and a lot of those junior leagues,
like I said, because they're expected to get better when they leave.
Anyways, part of the reason may be something to do with the league,
but part of the reason will be the age. And I have age explicitly
controlled for, which is something that won't be done in anything NHL E based
as far as I'm aware. So I know we're going to get I keep
teasing it, but we're going to get into some specific leagues a little bit
later on whether they are improving, whether they're declining. But did do you
have any major finings otherwise leagues that we thought were better but under your new
model that appears to have been different leagues maybe that moved up in estimation or
moved down or in estimation dramatically based on your study. Yeah, the big
one in terms of moving up is the AHL. I have the AHL estimated
is the second best league in the world now over the entire sample going back
to two thousand. I don't believe the AHL has necessarily been the second best
league in the world, but on the year to year level of the AHL,
I have estimated as the second best league in the world and have for
half a decade now, which I think is a pretty big flipping of people's
perceptions. But as a lot of teams are starting to play their best prospects
in the AHL, Buffalo is doing it a lot. You can see it
in some of the organizations as they move their best players to the AHL,
as guys who are middling NHL players fighting for a roster spot aren't as well
to go overseas, specifically to the KHL for obvious reasons of not wanting to
go to Russia. Right now, I think it makes a lot of sense
that the AHL has been on the rise, and then the primary league from
a professional level that's been falling off has been the KHL. And then at
a junior level, there's a general rise in the American hockey pipeline, so
the NCAA, the USHL, the NDTPI is part of the USHL, but
there's been a general rise there relative to those CHL leagues. Yeah, for
sure. Yeah, And I'm just trying to understand the other thing. You
have to tell me whether this is already baked into the bread or whether this
is another piece of it is different leagues play by slightly different formats, slightly
different rules. European rinks are larger, penalties may be called different leagues or
maybe slightly different rules. Is that and so the scoring environment doesn't necessarily translate
to the talent I'm trying to I'm still trying to get through my thick skull
whether we are measuring how we can measure that the players are good relative to
a league if the league rules are slightly different. Is that what NHL E
does basically, or is that an additional factor that you have to account for?
So I guess we can go two ways to that it will be mostly
baked into the bread. Because say, whatever rule in the SHL is really
propping up these players scoring rights, then if they leave that environment, which
is what we're tracking anyways, and that rule changes, that rule's gone,
they're going to perform a lot worse, and the model's going to pick up
on the fact that, Hey, these guys who leave the SHL are always
not doing as well. The model won't tell you that it's the rule,
but if there is some rule propping up scoring or like the rink dimensions or
whatever, the model will be able to pick up on things like that.
But my answer to what could go wrong with these models bring up something that
I think is a better answer and something you should always keep in mind.
And this is true in my model and all sorts of models. Pretty much
everything will be regression based if you're aiming for interpretability, which will tell you
the average effects of a change in some variable. This is great in almost
all circumstances, even like medical journals and stuff are going to be using average
effects. But there are key interactions that may or may not exist as well.
So, for example, something I'm a little worried about that I don't
have an answer for is if you look at like Byron Bader's model, for
example, he measures NHL and he doesn't say this publicly, but he doesn't
explain how. But I've basically figured this out that his NHL estimates very closely
tracked what happens when you look when players jump from each league to the NHL.
So he has way higher estimates of the OHL than everyone else publicly.
Hey, and if you look at what happens when players go straight from the
OHL to the NHL, you happen to get an estimate that is basically the
exact same as OHL estimate, which tells me I'm fairly certain that's what's doing.
Whereas the other public models like Patrick Bacon's, Max's, Chateel's, all
of these will measure the web of interactions, so not just the OHL to
the NHL, but the OHL to the AHL, all of that and bake
that into the math. And what happens is if you look at estimates of
a lot of these, say the Russian leagues in particular as an example,
because they we can keep going on this theme is Bader's estimates are a lot
lower relatively on the Russian league's than the other public models. And I think
there's a potential that the Russian leagues feed into each other better than they feed
to the North American game because of the different ices and stuff like that.
So a model that is measuring the averages will be able to say that,
like, on average, these things are happening, but maybe when you're switching
from different ice surfaces you're scoring couldn't translate five percent worse or something like that
could be at play here, so that maybe you know, the KHL is
whatever best league in the world, but when the KHL is translating to the
NHL, it's actually five percent worse than it is on average overall, just
because of the way the ice surface translates or something like that. So I
think these kind of specific interactions are something that you would need specific scouting on
to know and could mess with any model that based around averages. This is
so interesting, and I just I want to reiterate that because I want to
make sure people understood what you just said, because there's such a huge difference
between guys like I don't know, say, Athan McKinnon, John Tavera,
Sidney Crosby jumping straight from their junior leagues to the OHL, and guys who
play two, three, four years in the OHL and then make the jump.
Those are obviously going to be different jumps. And I don't know that
it's fair to expect a similar jump and equivalency from even high end prospects that
might have played two three years in junior compared to those generational talents. So
it's not fair to assign the same probability, and especially guys with who play
in Russia who won't necessarily make an immediate jump because they have a contract so
they can't go next year to the NHL, so that might create some noise
or some effects that aren't necessarily correct or predictable. I think that's kind of
part of what you're saying too, right, Yeah, yeah, exactly,
because, like I said, there will be specific interactions even if you look
like compare first round picks to second round picks with similar NHL E values when
they do translate, the second round picks tend to perform worse. Why because
the scouts have correctly picked up on all sorts of these more subtle factors that
do matter but aren't measured in the model, right, which would be why
your Crosby's translate better than the Guard's going to translate better than Stankovin or whoever.
Like that kind of thing. Yeah, exactly. And I think there's
another thing you said earlier about the HL that I think is worth repeating or
digging into a little bit, and that is there's a lot of interesting things
when a player like Yuri Koulik is taken, right. I think he's a
perfect example because in the range that he's taken, and you compare him to
other players per se that have been that same we're taking in that same range,
but we're maybe taking out of the CHL and you know that they're going
to have to play, you know, a couple more seasons in the CHL
and aren't eligible to move over. So there's going to be a big difference
there. I think there's a lot that you can say about that, But
also the fact that you have the control of that player right you can,
like in Buffalo's example, you have basically the farm team right there close eye
you can help direct their development, as opposed to being in Russia for one
example, or even in the COCHL, where you don't have necessarily as much
control over what happens with that player. So just looking at that draft,
Cooliquent twenty eighth overall, and they are a bunch of players from the CHL
who went around there. In terms of other forwards and USNVP types Reach Schaefer,
Isaac Howard, Owen Beck, Jaggafercus. So you're not going to be
able to put those players in the AHL and give them a more difficult competition
level and also be able to have more say over their development run the same
system. So all of those are factors that are hard to measure right in
terms of having a large enough data set to be able to compare apples to
apples, but I think it's certainly important that is a big difference. I
think that that may be playing into some of these equivalencies as well. Then
you think all one hundred percent, and like I said, I think I
think back to the AHL being a lot better. Like I don't think it's
an accident that Buffalo got Coolich overseas as quickly as they possibly could, but
that will definitely end up playing a role. The tough thing from a developmental
perspective, why I'm not sold it'll ever be easily measurable is there's so much
selection bias too. You can probably measure it right now and say, hey,
guys that do make that jump to the AHL will perform better than guys
who are left in the CHL in their draft plus two year or whatever,
but part of the reason will also be, like, there is presumably developmental
effects there, absolutely, But if you ever go to measure statistically, the
big problem you're going to run into is that teams are going to bring the
better players over as best they can to play at these higher levels, So
your statistical models aren't going to be necessarily as valid as they should be because
there's just such a selection biases. Teams aren't bringing over random sample of players
like Buffalo is bringing over Coolich but not the next European player selected most likely
yet and the reason why it is just because cool itge is better. Right,
So it definitely does matter, but it's going to be very difficult to
ever find to measure that statistically. Yeah, so if I understand what's going
back and forth here, the guys who are held back in the CHL for
a certain amount of time tend to be if they're the ones who jump straight
to the NHL. A lot of times it's because they're ready more quickly,
and so the AHL guys who need to simmer for more years tend to not
be as big of an impact when they come up. So that's a problem,
and that's really a problem with NHL E if we are just looking at
one step to the next as the way to translate it. Okay, Okay,
I want to be fair to your model, because the greatest thing in
the world is to have somebody on and say, really cool system you've got
here. Now here's all the problems I've got with it, and it's broken,
and why are you been talking to me, and I don't want to
go there because to me, it seems like it's not that a model is
perfect or horrible, it's that your iteration can be more accurate than the NHL
lead. Everything steps up more and more in what you are accounting for is
presumably better and more predictive. But let me get down to a level and
you tell me if it's too granular to account for. So you distinguish between
Ford and d but there's so many other things that impact whether a player's game
is going to translate. You can be a forward going from the OHL,
the CHL to the NHL. Maybe you're in the queue and you can say
this is the scoring change. Some guys who are in the queue might be
thriving on just having open ice for days and being able to do things that
they can't do at a higher level, whereas other guys might be making that
scoring by the way that their game plays. That you don't see the difference
in the scoring at one level. You see it in the next level because
the way you they play, and that's going to end up coming out in
the sample, because some guys are going to perform higher, presumably and at
the low end of the model because of factors, but very difficult factors to
get data on. Is that a problem that there's any way to account for,
or something you think that somehow NHLZ is able to smooth out. So
I think it'll help smooth it out relative to an NHL model that just looks
at guys who jump to the NHL, because if you just look at the
guys to jump to the NHL, only the better of those two players you
named will probably make that jump, so it will measure both of those guys
the theoretical exact word I'm will leave, and if the league is propping up
players because of this open eyes, the ones who do leave and are propped
up by that will struggle, and the model will learn that and start to
pull its estimate of that league quality down because these guys will start to perform
worse than you would otherwise have expected when they do make that jump. But
again back to applying the predictions to an individual level, that is absolutely something
that can be an issue and one of the things I would love to have,
not necessarily time on ice. We talked about why that could be a
flaw, but what I think the easiest way to account for some of these
league merchants would be just getting power play versus even strength time. I think
a significant portion of these guys who have explosions could probably be explained by,
Oh, yeah, he's playing four and a half minutes on the power play
in the year before he was playing like twelve seconds or whatever. I think
that would probably be the most helpful contextual variable at which point you could measure
points relative to the distribution of their ice time rather than just the ice time.
Would be the primary way in which I think time on ice could help,
beyond just like the raw number, but actually knowing, hey, this
guy played four minutes on the power play, this guy played two minutes shorthand
instead, that's why the scoring differences might exist. Would be the primary way
which I would think you would want context there. Things like the power play
usage are huge if you can't get on the power play at your low level,
and especially at a man's league quote unquote, your translation is going to
happen very differently. But like you said, the survivor bias, the people
who actually make it from one level to another will take some of that out.
All right, let's take a quick part of it too. I think
that problem is getting worse actually, which isn't a good thing. As I
sit here with this model specifically for defenseman, which is why I want to
separate them, because as the world basically has followed the NHL into the four
forward one defenseman power play, it gets really hard to earn power play ice
time for defense Only one defenseman is earning meaningful power play ice time, which
is going to exaggerate that that'd be something we have to watch out for totally.
All right, there's so much more to talk about. Let's take a
quick break and come back some of the addition we talk we've got to do
here. We're back with Chase McCullum and one of the cool things here.
If some of you are glazing at this point, and even though it Chase
has done a great job of explaining it, and Victor and I have done
our best job to pull it out, maybe some of you are listening to
like, Okay, that's all fine, but what the heck am I ever
going to do with this? But you have written some articles on ep ringside
that really shows how to apply this very specifically to some contexts and really actionable
stuff. And one of them that caught Victor's eye. In my eye is
the article about Russian hockey in decline, And you've talked about that already a
little bit on the show, But can you expound on that. What were
you able to find about Russian hockey specifically in the data and the a plausible
explanation for it? Absolutely, So we talked a lot about the model's flaws,
which I think is really important to drill into, Like it's worth understanding
the classic saying that all models are rocks are useful, right, Like it's
very important to understand that I don't have everything I wish I had, and
there will always be imperfections, But I still think what has been measured here
is worth caring about. And the reason why is I took these predictions,
or I took our league quality estimates and found two very important things, and
then we can show how to apply them specifically. So, my league quality
estimates as measured in this model, if you take them and attach them to
your prospects, and you include my league quality estimates in your projection of how
likely a prospect is, to, say, make the NHL, you will
find that players who play in more difficult leagues as measured by my model have
been more likely to make the NHL, all else being equal, accounting for
things like their age, they're scoring, and whatnot that we're already accounted for.
And second, I mentioned time trends, so that leagues may be declining
or improving over time. I split out the effects just to prove a point
basically, and found that when the model recognizes leagues as in decline, players
who are in a declined version of the same league are less likely to make
the NHL, again holding scoring constant than players who are in the better version
of that league. So the example we're going to dive into is the KHL.
My model has KHL as getting significantly worse recently, which then if you
apply to predictions, it will say that, hey, a guy who scores
twenty points in the KHL is less likely as a first, say an eighteen
year old guy's eighteen and a half years old and he scores twenty two points
and thirty KHL games. If that guy did it ten years ago, you
would expect the odds of him to make the NHL are higher. And I've
tested that and was able to prove that. So that basically comes down to
the so what of the model. It's got all these flaws and yet it
ends up being useful anyways. So the big takeaway from this, I think
the most important thing is you're seeing a lot of Russian records fall. Mitchikov's
breaking them. Forget who one of the former NHL guys I used to break
the Russian scoring record. All sorts of these crazy things are happening. And
if you look at the changes in league quality estimates, some of the biggest
declining leagues are the NHL, the KHL, and the VHL. So it's
systemic to all of the major Russian leagues we would care about as it relates
to prospect or NHL level analysis have been declining. And then I compare it
to a sample of similar leagues. So if you take the KHL compared to
other pro leagues in recent times, the KHL has declined by these estimates significantly
more than the average of the other pro leagues, suggesting that we need to
change the way we think about KHL scoring. Not that it isn't good,
which is tempting to do when you see things like the KHL is not as
good as it used to be, but just that it is worse than it
was. So even once you account for this, somebody like Mitchkov, who
has I think the second best NHL E of the millennium, he's still a
good prospect. He's still first overall level scorer and all of those things,
but he's not the second best score of the millennium once you account for this
stuff, so you can track relative changes in prospect probabilities based on what the
model is saying. And I think the most important change in those probabilities is
that all else being equal to these Russian guys need to score more to be
as good as they were. So like you're seeing Mitchkov outscore Ovi, that
doesn't necessarily mean Mitchkov's a better prospect than Ovechkin was, because the league Ovechkin
played in was significantly better, even though it is the same league. If
that makes yeah, But it's Danilli You're Off as a guy I think about,
and Victor's hate for Danila You're Off in his draft here was epic.
He gave me the hardest time for liking Danilli You're Off. And Danili You're
Off played twenty one KHL games. He had zero points in those games.
He averaged under two minutes a game. Russia just screws around with these prospects.
So it makes it so difficult. In Mitchkoff, we get some kind
of a sample on him because at least they're using him and they're allowing to
do things. They're setting him up there and so he's running up video game
scores. But I just don't know how you can have the world where Mitchkoff
and you're Off are on the same wavelength, although you're off now he's got
twenty seven points in thirty four games this year. He's actually being used.
We can give some kind of a credit there two years past this draft year.
But yeah, that's what makes it so difficult to me to comprehend how
to do this kind of equivalency because of especially in Russia, it seems to
me the way that they screw around with these guys' deployment. Yeah, and
there's it's not just Russia too. Russia seems worse like the worst of them
all. I'm definitely with you there. And another problem with anything equivalency on
the pro guys is from a statistical perspective, you have to be less certain
about the pro guys because there's so many fewer events that matter at a pro
level. So Connor Bdard and mattve Mitchikov say their NHL ease are similar.
Connor Bdard had to do one hundred and twenty things to drive that NHL E
goals and assists over seventy games. Mitchkov only had to do twenty things over
twenty seven games to drive his NHL E. So there's a significant level of
uncertainty based on there as well that we have to account for. Even you
can see on my measure, I can't necessarily account for that, But the
pro leagues have wider air of bars because there's just more uncertainty underlying pro league
scoring because it's so much more rare too, which also plays into that and
makes it very difficult, which is something that needs to be accounted for and
set. And I think this is part of the reason why if you look
at the big NHL, you guys, the ones who fail almost all come
from pro leagues, because even though it's harder to score in a pro league,
it's a lot easier to luck into twenty points than it is to luck
into one hundred and twenty points like you would have to in the CHL To
be one of those really high scoring guys, if that makes sense, totally
sample size issues. We talk about these things all the time. There's going
to be way more variability in every direction when you have twenty games, twenty
examples, twenty incidents, and a lot of them just are noise and don't
show up as much, and so we just discount that as these guys aren't
very good and maybe there's some gems in there that we're not picking up on.
And then there's the guys who maybe lucked into a bunch of secondary assists
and all of a sudden they look like they're amazing, and I think this
is really good. Like trying to recalibrate some of our ideas and expectations.
I want to draw some attention specifically to last year's draft, because you also
looked at most recently this decline in KHL and there were there's obviously geopolitical concerns,
but we also saw some really interesting Russian players drafted in that. And
of course there was Mitchikov, which I know you wrote a whole thing about
and definitely people should go read that. But I also wanted to focus on
some of the other guys like Dmitri Simyshev and Dlo Boot. This was really
interesting because we had one team take to what some would consider big swings on
these two players. And one of them's defensemen obviously, so that's different.
But they're both big players and they're both in the KHL, and Samyshev didn't
have as much going I actually neither one of them did. But I'm just
wondering what your thoughts are on those in general, because there were people in
every direction, some people saying, Wow, this is great, I love
that they did this, and other people panning it, just saying, what
are you thinking? There's such better options, even passing on Mitchkov, although
there may have been other reasons why Arizona did that. But what are your
thoughts on specifically those two? Yeah? I thought those were the really bold
choices. I didn't like, necessarily hate them to the degree that most people
who make draft models did like. Our model was relatively high on Simashev and
relatively high on Daniel as far as I could tell, But I would have
definitely went a different direction even I would have based on our model and whatnot,
and based on the EP grades and everything I've heard from public scouting and
stuff. I've maybe Mitchcov didn't want to play in Arizona or something, but
if you look Mitchkov and Benson being the two guys that were drafted immediately after
their picks is a tough look. Now, with that being said, I
do respect the swing, especially on Simashev. I wouldn't have made them made
these swings. But if you think you have identified something and you have Simyshev,
this six' four physical defenseman who can skate like the win like he
has every physical tool in the book. As much as people publicly criticize picks
like Simashev for not swinging on upside because they weren't scoring and scoring is usually
highly related to upside at the NHL level the specific pick, I respect the
thought process of swinging for the player you've identified as potentially a complete game chain
like Simyshev. If he hits a ceiling, there won't be other players like
him in the league, which I respect the swing for. But I would
be really worried that pretty much everyone I saw and as best as I could
tell, other teams were not nearly as high on those players, even if
again they are good prospects, they maybe were just overdrafted, at which point
you start running into the scenario that you need to be right and everybody else
needs to be wrong, which is a very bad situation to put yourself in
when you have thirty one other teams all and the public scouts and the public
models. There is a lot of opposition to those picks. And any time
you put yourself on the opposite side of something that many smart people believe,
I think you should be a lot of red flags should be going off.
If that makes sense. Absolutely, you're digging your heels in positioning yourself opposite
everyone else. That's yeah, you better be right, because otherwise there's plenty
of people to point out, yeah, oh you were the outlier. There's
some other interesting things here. There's so many interesting things we could talk about.
I wanted to shift to another potential. What I view is sometimes a
somewhat similar situation. So in Europe there's other leagues. I always find it
interesting and sometimes give a little feather in the cap of players who are prospects
who are playing professionally at a young age, So that sometimes happens in Russia.
Obviously, there's Finnish examples or sweetiest examples. More recently the Czech league
has propped up a little bit. It is difficult, and I think especially
the Finnish League, which is known to be a very low scoring league,
and you already mentioned how that can make it difficult to assess because you're talking
about more rare events. So I wonder if you could just speak on those
a little bit. Is it still you think worthwhile to give special props to
players like that who are playing significant games in the top level in Sweden,
Finland you can include check if you want there, maybe even the NL,
there's some examples there. But I do find it's difficult, especially when the
player is and we see this a lot are playing fifteen to twenty twenty five
games in the SHL, but they have a couple of apples, and then
it's up great that they're playing, but they're not getting many points, and
then it's how reasonable is that with twelve minutes of time highce right, yeah,
exactly, I definitely agree. I think there is a significant amount of
value in just showing up in pro games. So I had mentioned our model
was a lot higher on the on SIMMISHEV than the other public models, and
that was the reason why, because our model will specifically account for the fact
that, hey, this guy played thirty KHL games or whatever it was as
an eighteen year old, even if you're not scoring. People who play at
these high levels, this means a professional coach has decided that you, at
eighteen years old, are worth playing because they give you a better chance to
win every night. That's a massive signal, even if you're not necessarily scoring
points. That's really important. And then it also builds into the fact,
so I think you have to look at them as two separate effects. When
they're in a pro league, Yes there is signal in and of itself,
and then it goes two different directions. So when you're in a pro league
and not scoring, you have to respect the fact that it's easier to be
good and not score in a pro league because those events are so much more
rare, so that the variance being against. You can't tank your NHL e
just by hitting two crossbars or something like that. And then yes, there
is value in and of itself even if you are scoring like a mitgejob,
but you will have to respect the fact that the variance goes the other way.
With these high scores where it's probably more likely that they can be inflated
by NHL e So it's a nonlinearity I think, where yes, there is
value in and of itself, but you need to be careful when talking about
the scoring of these prospects because it can affect the low scores and the high
scores differently, I think from an NHL perspect or, like from an equivalent
seed perspective, awesome. I want to go in as slightly different direction a
similar topic, and that is looking at some of the North American leagues,
and I think you mentioned and I would definitely say that the NC DOUBLEA is
different from not including the AHL, but like the other non professional leagues,
right, NCUBLEA, and you're talking USHL and CHL leagues WHL, OHL,
MHL, and you could maybe lump in the AHL, BCHL, but those
are so much lower I think, and I want to get your take on
this. There's a pretty big difference between those few players who play their draft
season in the NC DOUBLEA. You're talking like last year Fantilly there was Jack
Eigel, Paul Korea. You're talking this year maclen Celabrini. That is a
very different level and to me, should boost them up a lot. But
also there's the other rarity effect that you're talking about. These are not you
know, it's not the same as probably the generational talents that are like exceptional
status in the OHL, but some of them are as high. But they're
so you're talking about how do you compare these guys to someone who's a few
months younger. And that's sometimes the challenge with playing in college. But it's
not necessarily available to everybody, but it certainly can mean a lot if it
goes really well. And then the other flip side of that is sometimes it
goes okay, and then they just look like not the best prospect when somebody's
killing it in the Q or in the USHL. But it's such a lower
equivalent sye. So how do you marry it those thoughts in those discrepancies.
Yeah, I have a theory that I haven't tested yet, but I think
may help marry it. And I know I believe Jeremy Davis. I think
it was he had the job that I currently have an EP before I did.
I remember way back when he had different age adjustments for different leagues,
and if I remember correctly, it was that as an example, the pro
league guys got bigger age adjustments than the junior guy, so then applied to
a more the different junior leagues. I think there's an effect there where the
younger you are, there's probably larger effects the more difficult the league. So
like these guys who are in college performing all right, it could be more
impressive just because they're in A The NC DOUBLEA is significantly better than any of
the other junior leagues of USHL and the OHL and whatnot, So I think
they may be punished for being relatively young in the league more so than guys
who are playing in these weaker leagues. Is something that I think probably exists,
but I have not adjusted for yet. One of the interesting things about
the differences between these leagues NCAA and HL, I would think there's relative stability.
You tend not to get jerked between leagues. In those things, those
players will pretty much sit there first season. It could be traded. But
overseas leagues SAHL, there does seem to be some loney going back down KHL.
Man they could just apparently they could just do whatever. Between the VHL,
MHL and KHL. People are getting jerked around all the time, and
it's really got to change how you can translate between those leagues because those things
are varying in quality even throughout the year. It is that all does that
all come out in the wash as well, just because you could say all
that, but at the end of the day, when those guys come over
to the NHL, they all were in the same boat at the time.
Yeah, that's a tough one to account for from purely a statistical respect.
Then, like a presumably does matter. Evan Demodov has played MHL, VHL
and KHL games this year. Like nobody in the OHL is ever going to
have played in the I don't even Canada doesn't really have equivalents. But nobody
in North America would ever play in three leagues in the same year, unless
you're talking to like World Juniors and stuff. But that's different. So it
presumably does matter just at a personal level for these kids, but adjusting for
it in a model is going to be really difficult. I think that's the
point where you have to lean on your scouts, because some individuals may thrive
in that environment, some individuals may really struggle with that right. Some people
might take longer to adapt as they're changing back and forth. I don't know
how much it matters on average, but there are presumably small edges with the
individuals that you may be able to find there. Okay, yeah, Chase,
We've we've poked, we've prodded, we've been all around this thing.
Do you think there's anything else that people out there can learn that you want
to explain to people about your model? And I'm sure that the next question
everybody you have is can you just give me a spreadsheet with all the equivalencies
for these guys? Please? I do have a spreadsheet of all The One
really ian thing to go through with the model is that there's been a systemic
change recently in the non NHL league's quality relative to the NHL. And what
I have been seeing is that the NHL as a whole has been improving relative
to these other leagues. So on average, most leagues are worse relative to
the NHL than they currently are, which I think explains why you're seeing like
all sorts of guys are scoring at these near record paces from an NHL equivalency
perspective, And I think if these leagues are farther away from the NHL than
they used to be. I think that would explain a lot of it.
Also, even somebody like Jack Hughes who buy NHL equivalency, was like a
borderline generational prospect, but Dards the first rookie we've seen true rookie, not
you know, twenty year olds and whatnot coming over a lot of the top
picks right out of the draft. Everybody's rookie season has sucked since Matthews basically,
and the Matthews in line A, there's been a lot of very poor
rookie seasons, which lines up basically with my model. In twenty eighteen,
rest of the world as a whole started to get significantly worse compared to the
NHL. So I think there's a distancing of these leagues which could be driven
by a lot of things. The NHL is getting smarter, the NHL's expanding,
so the best players from a lot of these other leagues are leaving,
and there's forty extra NHL jobs or whatever it is on any given night compared
to what it used to be. So I think that's really important to recognize
that on average, a one hundred point OHL player or KHL player that don't
exist. But like you get the idea or BHL player or whatever is probably
farther away from the NHL than the same guy would have been if he was
doing that in twenty ten compared to somebody doing it right now. Wow.
Yeah, and that will jack up your sample, But yeah, you definitely
have that drift accounted for in some of those error bars. Yeah, and
that's why I posted a chart ware of the thirty best NHL seasons when you
don't account for these time trends, like nine of them have come from this
decade, despite the fact that we're only three years into this decade. This
decade had a pandemic where a lot of guys missed. Twenty twenty one,
specifically, a lot of guys didn't even get a shot at setting NHL ree
records, and twenty twenty two was universally recognized as one of the worst draft
classes you've ever seen. And yet we're still seeing elite seasons out twice the
rate this decade as we've seen previous decades, and mixing in the fact that
rookies have actually been worse despite this fact. So I think it explains a
lot of kind of things that are happening in the hockey world at the macro
level right now, this has been so great. I just wanted to follow
up with one last thing. I know that you've been brought into the EP
team and it's super exciting to have your insights there, and I know that
I'm pretty sure you were there for the first draft meeting, and yeah,
I'm just wondering, obviously your role is different and a lot of the other
scouts are really obviously relying on the eye test, in person, video scouting
and other things. What do you bring to that discussion. I'm wondering what
kind of specific insights you point out and things like that. What has been
your role there. Yeah, so I'm easing into it. I think I'll
have a better answer for you in six months than I do right now because
I've just been there. But for example, my favorite thing to do is
test things like I in grad school, my best class was econometrics, which
is just causal inference, So what's the average effect of X on Y?
So just basically testing scientific questions like that, and I think that's one of
the primary things where I can add value. And as an example, in
the last scouting meeting, we were talking about one of the prospects and the
scouts have brought up how this guy was dramatically outscoring his closest teammates, which
sounds like a good thing. I assume that was a good thing. I
made a variable for my draft model to account for guys who were doing that,
and I assume to because I assumed it was a good thing. And
what I actually found when I fed into the model was the model would actually
learn to punish players who are way out scoring their teammates. The model will
actually give more credit to guys who score on good teams than guys who score
on bad teams. The logic, I think being that points are a scarce
resource. There's only so many puck touches to go around, So if you're
beating out all stars for puck touches, you're probably that's probably more impressive than
beating out guys on the worst team in the league. So just being able
to as E gauge test things like test conventional wisdom, because a lot of
the time conventional wisdom is right, but it can also be wrong in all
sorts of different circumstances. And just having somebody with the ability to test whatever
hypotheses we think we have in the sport, I think and be really valuable,
and that's one of the things that I'm doing right now as well as
just like I wrote an article about statistically age adjusting their draft grades and things
like that, because mentally, it's going to be very difficult to say,
hey, how much more impressive does an eighteen year old have to be to
make up for the fact that he's eighteen relative to a seventeen and a half
year old. So I think there's just finding pockets like that, whereas statistical
analysis can really help you because it's hard to do mentally, And I hope
I'm able to help them out with all sorts of these things and probably find
more ways to contribute. As I stick around, you will find more ways
to contribute, and we will find more ways to consume Chase, because you've
got some great stuff going out there. I hope everybody will check it out,
just once more for posterity. Where should they go, Chase to see
all of your information? Yeah, so they can find if you just search
my name at hon Ei Prospects, anything that I've written should come up on
ep ring side and then I post everything to Twitter at CM Hockey sixty six
as well. As just random thoughts and whatnot. That's probably the best place
to find everything I'm doing. I also just tease all sorts of stuff I'm
working on whatnot there, So that's probably the optimal place to find outstanding everybody,
go out and do that. Thank you so much for coming on today,
Chase and broadening our minds as to new ways that we can find the
prospects who are going to be most successful in our line ay. A couple
things to mention before we get out of here. Our show is brought to
you by fan Tracks. Did you know that you should? Because their sponsor
of the show, you should move your leagues over to fan Tracks. Ask
them. They'll help you. You can start new ones, the most options
for scoring salaries, contracts, customizing your rookie eligibility, start up your leagues
the day after the season ends, anything you can think of. They even
have a pretty good chat feature while other platforms are pulling that one back.
You can look at the fan Tracks hqu side. I write there sometimes.
Actually I wrote most recently on baseball, but don't tell anybody, but there
are plenty of articles on fantasy hockey. That group has geared up like nobody's
business this year. We'd like to thank our entire FHL crew doing the work
over here. We got a posse. We've got a family who are doing
lots of work for us and participating heavily, including content curator Kevin Adams who's
been helping with show prep. Ryan Downey, the commissioner of the Tidy Legs,
affectionately known as the Tidy Admiral Net Guy. We keep him busy.
Brandon, website guru, doing a fantastic job over there. Jeremy Vee you've
heard him on the show. He's our lead scout and he keeps those scouting
reports coming. We got people really digging into the video. We've got kind
of the professional service that Victor has been able to get us into where you
can view a lot of video on these prospects, and Jeremy is coordinating a
lot of the scouting reports we pull out of that and we share with you.
Jason is helping with our aspect ranks victors, making those better and better
every day. Paul is assisting with some workflow and processes because there's too many
things going on. If you've got some skills you'd like to lend the show
to let us continue to build and build. Hit Victor up in the Discord,
email or Twitter. We are brought to you by Dabber Hockey and Daber
Prospects Victors and Editor over there. You can follow his work there, as
well as his other podcast, Daber Prospect Report, he does with Peter Harlane.
If you like this, you're probably gonna like that because they're talking a
lot about the prospects and the different leaks where the prospects play. I do
a solo show, Dynasty Sports Life. I talk four different Dynasty sports sometimes
at the same time. Sometimes we even cover some fantasy hockey, but usually
on that show, I don't know specific fantasy hockey episodes, but I talk
about dynasty playing theory. I think we might even have an upcoming show on
Dynasty ethics. We'll see about that. But this week it's going to be
about quarterbacks coming out in next year's NFL Draft with a great guest who knows
all about that stuff. Follow us on x at fan Hockey Life at Victor
Newno. Twelve, Rate and review us on Apple Podcasts, Spotify, wherever
else you like to get pods. There's so many good places that you can
get this stuff, So check those things out, listen to us, We
appreciate you, and until next time, keep living that fantasy hockey life.