New-Look HP! Unfiltered Articles Stats Glossary Contact Us
Hockey Prospectus home

New Look Hockey Prospectus is available with new Premium Content!
Limited time offer: $9.99 for an entire year!

<< Previous Article
From Daigle To Datsyuk (05/14)
<< Previous Column
Numbers On Ice (05/07)
Next Column >>
Numbers On Ice (06/01)
Next Article >>
NHL Playoffs, Third Ro... (05/16)

May 14, 2010
Numbers On Ice
Ranking The Goalies

by Tom Awad


Following up on last week’s article, I determined that if we can extract the amount of skill in the distribution of goaltending across the league, we can also perform an estimate of the skill of each individual goaltender in the league. Today I will attempt to do just that.

First of all, the methodology: I have used the same 3 seasons of data, 2007-08 to 2009-10, that I used in last week’s article. I decided to perform an estimate for the 40 goaltenders who have seen the most shots at 5-on-5 and 4-on-5, so the cutoff was Alexander Auld’s 2,296 shots. I also assumed that each goaltender had the same “true” talent for the 3 years, and that is what I’m estimating, so I’m not taking into account potential injuries, changes due to personal circumstances (Jean-Sebastien Giguere) or aging (Tim Thomas, Chris Osgood).

I took the distribution of skill from last week’s analysis. The league average save percentage was 0.910, and the amount of skill observed in the actual distribution of Goals Versus Average (GVA) implied that the standard deviation was 0.005. I then calculated, based on this distribution of skill, what was the most likely true talent for each goaltender, and what was the uncertainty of our estimate. The more shots a goaltender faced, the more certain we are of his actual skill level. Because we are looking especially at #1 goaltenders, more than half will be above average.

Note for Math Geeks: I performed a Bayesian analysis to do this. As the a priori distribution, I took a normal distribution with an average of 0.910 and a standard deviation of 0.005. I know the distribution of goaltender talent, especially at the NHL level, is not normal, and certainly not symmetrical, but it made the math easier and didn’t change the results much. I then used a binomial model to calculate what were the odds that each initial save percentage could have yielded the observed results.

One last thing: all I am looking at is shot-blocking ability. Nowhere have I attempted to account for other goaltender skills, such as puck handling or rebound control.

First, the results:

Rank	Goaltender	       True Talent    Rank    Goaltender        True Talent
1	TOMAS VOKOUN	       0.9176	      21      CRISTOBAL HUET	0.9108
2	TIMOTHY THOMAS	       0.9172	      22      MANNY LEGACE	0.9107
3	JONAS HILLER	       0.9167	      23      JASON LABARBERA	0.9106
4	JAROSLAV HALAK	       0.9157	      24      CAREY PRICE	0.9104
5	HENRIK LUNDQVIST       0.9156	      25      JONATHAN QUICK	0.9102
6	MARTIN BRODEUR	       0.9144	      26      ILJA BRYZGALOV	0.9101
7	CRAIG ANDERSON	       0.9137	      27      MARTIN BIRON	0.9098
8	TY CONKLIN	       0.9135	      28      ANTERO NIITTYMAKI 0.9097
9	ROBERTO LUONGO	       0.9134	      29      JOSE THEODORE	0.9096
10	CAM WARD	       0.9133	      30      MIIKKA KIPRUSOFF	0.9092
11	RYAN MILLER	       0.9131	      31      DWAYNE ROLOSON	0.9088
12	PEKKA RINNE	       0.9129	      32      ALEX AULD 	0.9083
13	NIKLAS BACKSTROM       0.9128	      33      STEVE MASON	0.9081
14	NIKOLAI KHABIBULIN     0.9126	      34      MATHIEU GARON	0.9076
15	CHRIS MASON	       0.9121	      35      MIKE SMITH	0.9070
16	MARC-ANDRE FLEURY      0.9121	      36      CHRIS OSGOOD	0.9069
17	KARI LEHTONEN	       0.9121	      37      MARTY TURCO	0.9066
18	EVGENI NABOKOV	       0.9112	      38      JOHAN HEDBERG	0.9053
19	JEAN-SEBASTIEN GIGUERE 0.9109	      39      VESA TOSKALA	0.9048
20	DAN ELLIS	       0.9108	      40      PETER BUDAJ	0.9044

My colleague Rob Vollman calculated the rankings of save percentage since the lockout, and Tomas Vokoun came out 1st by a pretty big margin, so it’s no surprise to find him at #1 here. In the top 5, we see that 2009 playoff hero Jonas Hiller and 2010 playoff hero Jaroslav Halak are not complete flukes, especially given that this is only regular-season data. The top 5 surprise is Tim Thomas, who most people wrote off as a fluke after his numbers came back to Earth this year, but he had 2 excellent seasons (2007-08 and 2008-09) as well as multiple excellent seasons in Europe. During the current “disaster” season, his save percentage was 0.915; we should all be so lucky. It’s quite possible that age is catching up to him, as he is already 36, but his track record is rock solid.

In the following graphics, I have listed every goaltender, as well as the 50% and 90% confidence intervals for the true talent of each. The thick rectangle represents the 50% confidence interval: there is 1 chance out of 2 that a goaltender's true talent lies within that rectangle. The thin extremities are the 90% confidence interval: there are 9 chances out of 10 that a goalie's true talent lies within those points.



You can see that the confidence bars are not the same length for everybody, although since I restricted myself to goaltenders with 2,300 shots against they’re all reasonable. For goaltenders who have seen a lot of shots (Vokoun, Lundqvist, Kiprusoff) we have a pretty good idea of their talent range, while for Halak, Conklin and Auld we’re less certain.

What we see is that, for almost three-quarters of the goaltenders in the NHL, their performance is almost indistinguishable from one another. We know for sure Tomas Vokoun is better than Peter Budaj and Vesa Toskala, but that’s about it. This explains why the NHL goaltender leaderboard changes so much from one season to another, and why nobody agrees whether goalie X is really good or just overrated. When fans were debating whether team Canada should have brought in Marc-Andre Fleury or Cam Ward, it turns out that our best guess is that they’re equivalent.

It also shows that public perception of goaltenders is very much affected by team success and track record and less by actual talent and performance. Of the upcoming goaltender free-agent class, the four currently highest-paid goalies are Evgeni Nabokov, Marty Turco, Jose Theodore and Vesa Toskala. Nabokov is seen as a blue-chip goaltender and Turco as a dependable veteran (granted, Turco also has puck handling skills not quantified here), while the perception of Theodore has much to do with him being pulled in Game 2 against Montreal. But none of these goaltenders is any better than Chris Mason, a less touted name, and they’re not demonstrably better than Dan Ellis either.

In practice, the assumption that there is little skill difference between most NHL goaltenders is correct. You can rarely count on your goaltending to win you a game or a series because Brian Boucher can outplay Martin Brodeur over 7 (okay, 5) games, but there is still some difference. Next week I’ll attempt to establish how much you should pay for goaltending.

Tom Awad is an author of Hockey Prospectus. You can contact Tom by clicking here or click here to see Tom's other articles.

0 comments have been left for this article.

<< Previous Article
From Daigle To Datsyuk (05/14)
<< Previous Column
Numbers On Ice (05/07)
Next Column >>
Numbers On Ice (06/01)
Next Article >>
NHL Playoffs, Third Ro... (05/16)

Annoucements: Where Are The New Articles?
Zamboni Tracks: Who's That Guy? Special Edmo...
Hall Of Fame: My 2014 HHOF Inductees
On The Horizon: Four Nations And Junior "A" ...
A Closer Look: MacArthur-Turris-Ryan Keeping...

MORE FROM MAY 14, 2010
From Daigle To Datsyuk: Memorial Cup

2010-06-01 - Numbers On Ice: The Blackhawks' Main Advanta...
2010-05-20 - Conn Smythe Watch: Toews Is Number One
2010-05-16 - NHL Playoffs, Third Round: Philadelphia Flye...
2010-05-14 - Numbers On Ice: Ranking The Goalies
2010-05-12 - Conn Smythe Watch: Halak's To Lose
2010-05-07 - Numbers On Ice: How Much Skill Is There In G...
2010-05-06 - The Playoff Value Of Grinders: Are They Impo...

2010-09-29 - Numbers On Ice: What Makes Good Players Good...
2010-09-28 - Numbers On Ice: What Makes Good Players Good...
2010-06-01 - Numbers On Ice: The Blackhawks' Main Advanta...
2010-05-14 - Numbers On Ice: Ranking The Goalies
2010-05-07 - Numbers On Ice: How Much Skill Is There In G...
2010-04-23 - Numbers On Ice: Does Shot Quality Exist?
2010-04-08 - Numbers On Ice: Give Crosby The Hart