New-Look HP! Unfiltered Articles Stats Glossary Contact Us
Hockey Prospectus home

New Look Hockey Prospectus is available with new Premium Content!
Limited time offer: $9.99 for an entire year!

<< Previous Article
Up and Coming (11/12)
<< Previous Column
Numbers On Ice (11/02)
Next Column >>
Numbers On Ice (11/20)
Next Article >>
Howe and Why (11/13)

November 12, 2009
Numbers On Ice
A Castle Built on Sand

by Tom Awad

Printer-
friendly
Contact
Author

The field of serious hockey analysis has been open for roughly a decade now, with detailed stats such as shot distance, takeaways, drawn penalties and hits now available for puckmetricians to delve into. Unfortunately, one of the serious flaws with these stats is that every arena seems to have its own definition of everything. 40 feet is actually 30 feet if you’re in Madison Square Garden, while a missed shot will almost never be recorded as such in Chicago. It seems like the only reliable statistics are goals, penalties and shots, and one of these may not be as reliable as it seems.

With data provided to me for every team in the NHL by Puck Prospectus contributor Gabriel Desjardins, I have compared the number of shots recorded in every team's home games with the number of shots recorded in every team's road games for the four seasons from 2005-06 to 2008-09. The result does not inspire confidence:

Total shots recorded by both teams, 2005-2009

	
Team             Home	 Road	 Difference
New Jersey	 9171	 9963	-792
St. Louis	 8746	 9516	-770
Minnesota	 9094	 9689	-595
Dallas	         8557	 9108	-551
Chicago	         9281	 9731	-450
Pittsburgh	 9536	 9901	-365
Vancouver	 9179	 9535	-356
Carolina	 9826	10038	-212
Washington	 9986	10174	-188
Los Angeles	 9593	 9731	-138
Philadelphia	 9858	 9961	-103
Edmonton	 9308	 9378	 -70
New York I	 9949	 9992	 -43
Atlanta	         9955	 9905	  50
Phoenix	         9728	 9661	  67
Toronto	         9880	 9800	  80
Detroit	         9863	 9781	  82
Calgary	         9461	 9363	  98
Buffalo	         9863	 9754	 109
Montreal	10054	 9906	 148
Tampa Bay	 9662	 9497	 165
San Jose	 9343	 9147	 196
Boston	        10111	 9906	 205
New York R	 9793	 9580	 213
Columbus	 9535	 9243	 292
Florida	        10745	10316	 429
Anaheim	         9866	 9371	 495
Ottawa	        10182	 9609	 573
Colorado	 9825	 9236	 589
Nashville	10068	 9226	 842

Given a total of roughly 19,000 shots, the home-road difference could be expected to be about 140 on luck alone, so differences of 800 are clearly due to some kind of bias. There are clearly arenas in which more and less shots are systematically recorded. This leads me to wonder: are the shots simply being underreported, or are there arenas (or teams) which are simply conducive to less shots?

As an aside, for those who think that the number of shots is a firm, objective number, I invite you to view this video.

How many shots do you count on this play? Watch it again. The game log, from Florida, says 4. In St. Louis, this would have been 2 shots.

To figure out whether the number of shots was actually being undercounted, I figured I’d look at goals. Surely, all things being equal, if less shots were truly being taken, less goals would be scored, right?

Here is the sum of stats for the 8 arenas with a strong “less shots” bias:

	 Home	 Road	H/R Ratio
Shots	73390	77481	94.7%
Goals	 7238	 7325	98.8%

Here is the sum of stats for the 8 arenas with a strong “more shots” bias:

	 Home	 Road	H/R Ratio
Shots	80125	76487	104.8%
Goals	 7346	 7285	100.8%

In both cases, roughly 5% more or less shots are recorded, but the goal difference is only 1%. This leaves only two possibilities: either some arenas are recording less shots, or somehow the arenas with less shots simultaneously feature more dangerous shots on average. While I find the second possibility remote, it is a distinction without a difference: in both cases shot data is skewed, resulting in biased shooting percentages and save percentages.

Who Does This Affect?

The good news is that, since shots at both ends of the ice are affected, shot differential metrics, such as Corsi, are almost untouched. Player shooting percentages and shots on goal, while interesting statistics, are rarely ever used to judge players, with good reason: because a player controls both how many shots he takes and how well he takes them, it is rarely useful to rank players by these metrics, except when looking for outliers whose stats are likely to mean regress. Where this bias will have the most impact is on goaltender save percentages.

In the “more shots” category, I was not surprised to find Nashville and Florida. I’ve always found something fishy about Florida’s inflated shot totals, both during the old Luongo days and more recently with Vokoun. Likewise, Nashville always seems to have a goaltender with a 0.920 save percentage (Tomas Vokoun, Chris Mason, Dan Ellis and Pekka Rinne). Though the most remarkable bias comes on the “less shots” side and is found, unsurprisingly, in New Jersey.

Whenever any hockey analyst makes any breakthrough, it always seems like it applies to the New Jersey Devils and Martin Brodeur. The Devils are the team that everyone loves to hate, everyone loves to redefine and analyze. Statistical analysts have made it their job to deconstruct Martin Brodeur’s reputation as one of the all-time elite goaltenders; one blogger even named his site after him. I am about to become the first statistician to attempt to enhance Brodeur’s reputation.

One well-known and impressive statistical nugget about the Devils is how well, historically, they have managed to keep shots against to a minimum; the last season where the Devils allowed more shots than the league average was 1989-90. Since then, over the course of 18 seasons, they have averaged about 330 shots against less than the league average per season, despite going through different coaches and massive personnel changes (Brodeur notwithstanding). I believe we now have one explanation for this phenomenon: by my calculations, New Jersey’s home shots against have been underreported by about 100 a year. This would still leave them as a defensive juggernaut (preventing 230 shots against per season is no joke) without making them such a statistical outlier.

If this is true, it could further legitimize Brodeur’s claim of being one of the all-time greats. Brodeur’s career all-time shots against (25,481 as of this writing) could be too low by as much as 1,000 shots, which would boost his career save percentage from 0.914 to 0.918. This would boost his career GVT by roughly 80, which would place him #3 or #4 on the all-time list, neck-and-neck with Jacques Plante and only trailing Dominik Hasek and Patrick Roy.

I would love to have somebody else validate my analysis, if possible using data from other seasons. Until then, I will maintain a healthy skepticism on the reported shot totals in the NHL. As we continue to push back the frontiers of hockey analysis, I realize that we are building our edifice on sand.

Tom Awad is an author of Hockey Prospectus. You can contact Tom by clicking here or click here to see Tom's other articles.

0 comments have been left for this article.

<< Previous Article
Up and Coming (11/12)
<< Previous Column
Numbers On Ice (11/02)
Next Column >>
Numbers On Ice (11/20)
Next Article >>
Howe and Why (11/13)

RECENTLY AT HOCKEY PROSPECTUS
Annoucements: Where Are The New Articles?
Zamboni Tracks: Who's That Guy? Special Edmo...
Hall Of Fame: My 2014 HHOF Inductees
On The Horizon: Four Nations And Junior "A" ...
A Closer Look: MacArthur-Turris-Ryan Keeping...

MORE FROM NOVEMBER 12, 2009
Up and Coming: The USHL

MORE BY TOM AWAD
2009-11-24 - Numbers On Ice: Strange But True, Crosby Hur...
2009-11-20 - Numbers On Ice: Fatigue Worse Than Playing o...
2009-11-17 - Player Power Rankings: Miller's Time
2009-11-12 - Numbers On Ice: A Castle Built on Sand
2009-11-10 - Player Power Rankings: 'A' for Anderson
2009-11-02 - Numbers On Ice: Better Lucky Than Good?
2009-10-29 - Numbers On Ice: The Good and The Great, Refl...
More...

MORE NUMBERS ON ICE
2009-12-03 - Numbers On Ice: Week 9 Power Rankings
2009-11-24 - Numbers On Ice: Strange But True, Crosby Hur...
2009-11-20 - Numbers On Ice: Fatigue Worse Than Playing o...
2009-11-12 - Numbers On Ice: A Castle Built on Sand
2009-11-02 - Numbers On Ice: Better Lucky Than Good?
2009-10-29 - Numbers On Ice: The Good and The Great, Refl...
2009-10-06 - Numbers On Ice: Slump? More Like 'Sophomore ...
More...