|
For well over a year now, I've been telling you that stats-based draft rankings can produce results at least as good as traditional scouting-based drafting, at least when it comes to the major sources of new hockey talent. Some of you might still be skeptical, of course. Don't worry, I don't take any offense. I'm a very skeptical person by nature, and I realize that it's always correct to respond “prove it” when presented with an assertion. I've provided some of the evidence as the Projectinator has developed over time, so it's not an unsupported assertion in this case. But this week I thought I'd delve into a couple of conceptual issues, particularly as they relate to one possible reason for skepticism about claims of stats-based drafting: the famous 2002 Moneyball draft in Major League Baseball.
In Michael Lewis' bestseller Moneyball, the approach of Oakland A's General Manager Billy Beane at the 2002 draft was covered in great detail. Beane was clear in his intent to draft based on statistics, not subjective scouting reports. If you read this site regularly, there's a good chance you've read the book as well. It's a great read, but when it comes to the draft coverage it does tend to descend into cheerleading rather than reporting. That can happen when you're trying to evaluate a draft before the players have had time to show what they can do on a professional level, but the point of the A's drafting strategy was to see if it could be done, rather than assuming it could be done, and therefore winning the draft.
Looking back at that draft now, one can see that the A's did come away with some very good players in Nick Swisher, Joe Blanton and Mark Teahen, all taken within their first seven picks. A skeptic might point out that the team's third, fourth and sixth picks (John McCurdy, Ben Fritz, and Steve Obenchain) did nothing at the major-league level, and that their vaunted choice of Jeremy Brown seemed promising at first, but he played only five games for the A's. Moreover, all of these players save Brown were rated among Baseball America's top 250 available players at the draft, so their selections, while earlier in the draft than some expected, could hardly be considered surprising. That is, if they were using traditional scouting methods, they might well have drafted these players anyway.
As such, Brown and the team's later picks are more informative. One, John Baker, has had some success at the major-league level. But other than that, the lower picks have been disappointments at best, and complete busts at worst. Steve Stanley and Brant Colamarino (of whom it was said in Moneyball that he might be the best hitter available) both failed to reach the majors, or even to play a full season at AAA. From this perspective, you could sat that Oakland's draft experiment suggests that pure stats-based drafting does not work well. I'd be pressed to argue with that.
So what makes me think that such an approach could work in hockey, when in baseball (with its apparently superior statistical information) it has failed? It's easy for me to sit here with my spreadsheets and my unhealthy fixation on junior player's numbers, but in the real world, it's not so simple. Well, besides the numerical evidence I've already provided here at Hockey Prospectus, there are two essential and crucial differences between drafting baseball players and hockey players that allow this type of analysis to work for the great winter game.
One reason is the spread of available players, or more to the point, the spread of the quality of competition that these players play under. In their draft, Oakland management restricted their selections to college players only. Well, there are about 300 Division I baseball programs, spread over 30+ NCAA conferences. The sheer number of players is staggering, but more importantly each of these conferences presents its own quality of competition, and when analyzing amateur stats the quality of competition is absolutely critical. Add to this all of the high school players available from all over the US, and you have quite a morass to deal with. The A's list of their 20 most wanted players included players from 10 NCAA conferences, plus one Canadian university.
An NHL team does not have nearly that number of players, or that number of qualities of competition to deal with. Even if you limited yourself to the OHL, WHL and QMJHL (where the Projectinator started), you have a huge chunk of the available players covered. Since each of these league has many more teams than the typical NCAA conference, the quality of competition is much more even for this large group of players. That is, a much greater proportion of the desirable players are playing under the same conditions, which makes comparing them to each other much easier and more reliable. Anything you add to that, such as the USHL of European leagues, adds many more players but relatively few levels of competition. This makes amateur hockey stats more reliable when using them to predict future performance.
The second big advantage that hockey has when it comes to drafting is the relatively small distance, chronologically, between a player being drafted and the player contributing to an NHL team. Hockey is a young man's game. It is not uncommon for an 18-year-old to step into the NHL and contribute, or even star, without having played a single minor-league game. Indeed, the top players each year are expected to earn an NHL job immediately. Some do spend a year or three in junior or the minors, but Sidney Crosby is far from unique in going from junior to superstar in a single year.
This just doesn't happen in baseball. Even the best college baseball players, already four years older than the typical junior hockey draftee, typically need a few years in the minor leagues before they are able to produce at the major-league level. This shorter time frame means that hockey players are closer to their “final” form when being drafted, and this shorter distance between draft and prime means that a player's future prospects can be more reliably predicted.
So, despite the fact that baseball numbers are superficially more informative than hockey stats, and that there are many more people working on the baseball side, and yet stats-based drafting in that sport has failed, it seems that hockey drafting by the numbers can work. This is not due to the superiority in the numbers or in the analysts, but to at least two inherent differences in the natures of the amateur games. It's more than a pencil-neck pipe dream. It can be a reality. |
Up and Coming (10/19)
0 comments have been left for this article. 
