October 15, 2009
Up and Coming
Projecting Goaltenders

by Iain Fyffe

When it comes to projecting goaltenders, I was never convinced that I would be able to develop a system good enough to compete with professional scouts. Drafting goalies has always seemed to me to be something of a crapshoot, with an abundance of high-profile misses and misleading junior stats.

To make matters worse, the single most telling statistic for goaltenders (save percentage) was not readily available for many of the players included in my database. As such I had to use goals-against average (GAA), which is a distinctly inferior measure of a goaltender's ability.

For consistency I used GAA to develop the 10Y scores for goaltenders, which are designed to approximate the distribution of 10Y scores for forwards and defensemen. I'm not entirely satisfied with the ordering of the best netminders using the 10Y score, but of course that's not what the score is designed to do. It's designed to allow the 10YE score to be built off of it. So while having Martin Brodeur at the top of the list with a 10Y of 1.12 is okay, having Jocelyn Thibault tied for second at 1.03 is less so. Though taken as a whole, the top goalies as indicated by the 10Y score are very good, with Chris Osgood (1.03), Felix Potvin (.87), Roberto Luongo (.85), Trevor Kidd (.85), Martin Biron (.84), Jamie Storr (.73), Jose Theodore (.71) and J-S Giguere (.65) rounding out the top 10.

Some of the usual factors went into the 10YE scores for goalies. Age is quite important, and height has a small degree of importance as well (with the same caveats I noted last week). Obviously GAA (compared to the league average) is very important in the projection. However, there are two factors considered for goaltenders that were not used for either forwards or defensemen: games played, and team winning percentage.

Why are games played important for goalies, but not skaters? On any team there is typically only one goalie who plays in a game, while there are usually 12 forwards and six defensemen. So a goaltender has to be the best netminder on his team in order to play in the majority of games. Conversely, a team's sixth-best defenseman might play all of his team's games because there are more slots available in each game. So the simple fact that a goalie is playing games at this level at this age does, in fact, say somethingabout his ability. It's a principal you will see applied in the future when I develop the Projectinator system for European players.

What about team winning percentage? What does that have to do with evaluating goalies? Well, really it has less to do with goaltenders and more to do with the statistics. Being restricted to using GAA, I found that including an adjustment for team winning percentage improved the estimate. I believe this is because bad teams tend to give up quite a few shots, so this adjustment moves the GAA (which is being used) closer to the save percentage (which would ideally by used).

Let's have a look at the top 20 goaltenders by 10YE scores for the players in our database (first draft-eligible year from 1989-1998, played in the CHL), along with their actual 10Y scores. The Pro GP column is the player's professional games played in the 10 seasons after his first draft-eligible year.

Player	            Pro GP  10Y	   10YE
Biron, Martin	    351	    .84	   1.04
Kidd, Trevor	    336	    .85	   1.04
Theodore, Jose	    421	    .71	    .85
Thibault, Jocelyn   526	   1.03	    .83
Chouinard, Mathieu  142	    .25	    .82
Luongo, Roberto	    447	    .85	    .81
Potvin, Felix	    409	    .87	    .75
Giguere, J-S	    377	    .65	    .73
Hillier, Craig	    137	    .24	    .71
Lindsay, Evan	    250	    .21	    .71
Garon, Mathieu	    352	    .53	    .67
MacDonald, Aaron     75	    .21	    .67
Gagnon, Pierre	    147	    .21	    .66
DesRochers, Patrick 264	    .30	    .66
Lenarduzzi, Mike    313	    .23	    .64
Wilkinson, Derek    256	    .28	    .63
Fichaud, Eric	    258	    .48	    .62
Tanner, John	    191	    .28	    .62
Lamothe, Marc	    279	    .29	    .62
Boucher, Brian	    263	    .55	    .60

While the very top of the list looks fairly good, there are quite a few big misses here, and some are goalies you've probably never heard of. You might think this is a big blow to the Projectinator as it applies to goaltending, but I think you'd be mistaken. Taken as a whole, the scouts really liked this group as players as well, with very few exceptions. Only two players (Potvin and Gagnon) were not drafted in their first draft-eligible year (the 10YE projection for Potvin is very good, for Gagnon very bad). Of the remaining 20 players, 10 were drafted in the first 30 spots in their drafts, and only Wilkinson and Lamothe were below #57. The scouts saw the great majority of these players as top-two-rounds players. DesRochers went #14 in 1998, followed by Chouinard at #15, and Hillier was drafted #23 overall in 1996. So while the Projectinator misses on a good number here, so did the scouts, and the system picked Potvin, while the scouts passed him over.

Of course, there are also players that the scouts saw better than the system. These are the goalies with a 10Y score of .55 or more that the system did not do well on:

Player	         Pro GP	 10Y	10YE
Brodeur, Martin	 479	1.12	.49
Osgood, Chris	 440	1.03	.53
Storr, Jamie	 335	 .73	.53
Denis, Marc	 344	 .63	.47
Fernandez, Manny 365	 .60	.42
Brathwaite, Fred 346	 .59	.32
Legace, Manny	 310	 .58	.22
Dafoe, Byron	 420	 .56	.47

Maybe we should put in an adjustment for being named Manny? Some of these aren't too far off their projections, but some are. All except two of these goalies were drafted in the top 55 picks of their respective drafts, with the highest being Storr at #7 in 1994. Brathwaite and Legace were not drafted, so we can't really give the scouts credit for them.

Moving on to our final group of players, we'll take a look at goalies drafted in the first 50 picks of their drafts who had a low 10Y score and a projection that pegged their value much higher than their draft ranking did.

Player	        Pro GP	10Y	10YE
Damphousse, J-F	196	.29	.33
Moss, Tyler	298	.37	.39
Langkow, Scott	346	.29	.40
Sauve, Philippe	284	.30	.36
Weekes, Kevin	367	.34	.24
Cruz, Jomar	 60	.21	.45
Adams, Andy	 25	.21	.15

Damphousse was #24 in 1997, Moss #29, Langkow #31 and Weekes #41 in 1993, Sauve #38 and Cruz #49 in 1998, and Adams was #50 in 1994. The Projectinator could have protected against these unwise (in retrospect) picks.

What about the 2009 draftees? In this case there are no unwise picks to warn against, since the top North American goaltender in the draft was Matt Hackett, taken by Minnesota at #77. Only three others were taken in the first four rounds of the draft. Their 10YEs are as follows:

Player	         Team	Drafted	 10YE
Hackett, Matt	 Min	#77	 .53
Morrison, Adam	 Phi	#81	 .30
Berube, J-F	 LAK	#95	 .48
Pasquale, Edward Atl	#117	 .52

None of these young goalies seem to have a particularly bright NHL future, and the scouts agree on that point. It was not a banner year for North American netminders at the draft, though all but Morrison are projected to have decent professional careers, though probably in the minors for the most part.

Drafting goaltenders seems to be a difficult proposition, both for the pros and the hacks like me. This is hardly surprising given the volatility of a goalie's performance over the years. Jose Theodore went from the winner of the Vezina and Hart trophies in 2002 to a save percentage of .882 and a ticket out of town in 2006, and he's hardly the only example of such wide variations in play between the pipes. In the end, I think the Projectinator does about as well as the scouts have done in selecting goalies, perhaps a smidgen better, or perhaps a sliver worse. Only further analysis will say for sure.