Don’t mock my title. The entire objective of the statistical analysis of sports is to determine which players contribute to victory – in effect, who is good. There are many metrics to quantify player value: some go with the traditional goals and assists, which capture offensive contribution in the roughest way. The next step is often plus/minus, which is brilliant in theory but hopelessly flawed in practice. GVT is an improvement over all of these, but it is coarse and has its limitations. Now, in recent years, we have started measuring players by Corsi, ZoneStarts, QualComp and Delta… and still, the fundamental question remains: what skills do good players bring to the table that make them valuable? Or, to ask my initial question once again: What makes good players good?
(Note: There are a lot of numbers in this article. I promise I will try to explain them all clearly, and the payoff should be worth it if you stick with me. Data used in this article can be found here.
1. The Data
When performing statistical analysis, our biggest enemy is sample size. It is often extremely hard to figure out if a player has a particular skill, be it shooting, drawing penalties, or stopping shots at 4v5, because we don’t have sufficient data. Therefore, I decided I would look at Good Players (henceforth GPs) as a group. If I aggregate the data of all GPs, I’ll be able to see what characteristics distinguish them from their peers.
That’s all fine and good, but how do we identify GPs? The more naïve would simply take the top 50 scorers in the league and baptize this as our group of GPs. Obviously, this is unacceptable: scoring points involves a great deal of luck, and I would in effect be selecting the answer ahead of time. What distinguishes GPs? Well, they score a lot of points! Instead, I decided to go with a usage metric: even-strength ice time. Even strength ice time is a great filter because it is not results-based; it mainly reflects your coach’s opinion of you. While there may be individual cases where players are receiving the “wrong” amount of ice time (due to injuries or force majeure), they should be rare.
I will begin with forwards. I took all 582 forwards who appeared in at least one NHL game last season, and ranked them by even-strength ice time per game, from Ryan Getzlaf’s 961 seconds per game (s/GP) to Steve MacIntyre’s 168 s/GP. I then summed them into 4 tiers: the 1st tier contained 25% of ice time played by the 83 players with the highest ice time, the 2nd tier contained the next 25% of ice time (95 players), and so on. I then aggregated the results of all of these players.
2. On-Ice goals and shots for and against
First, let’s go with the basic events that happened while they were on the ice:
GP: Games Played
TOI: Time on Ice, indicated seconds or minutes.
GF: Goals For while on ice
GA: Goals Against while on ice
EGF: Expected Goals For while on ice (shots for weighted by shot quality)
EGA: Expected Goals Against while on ice (shots against weighted by shot quality)
Delta: EGF – EGA. A player’s expected plus/minus,
taking into account all the shots for and against while
he was on the ice and their odds of going in.
For an explanation of Delta and its derivatives,
see here and
here. For Delta data for the last 3 seasons,
2009-10 aggregate results
GP TOI(min) GF GA EGF EGA Corsi +/- Delta Exp +/- (Corsi)
1st tier 5916 84296 4013 3435 3588 3403 1688 578 185 135
2nd tier 6546 84607 3577 3385 3493 3333 1666 192 160 133
3rd tier 7204 85057 3149 3326 3286 3313 -304 -177 -27 -24
4th tier 9479 85320 2567 3087 2942 3208 -2507 -520 -265 -201
2009-10 per-60 min results
TOI(s) GF GA EGF EGA Corsi +/- Delta Exp +/- (Corsi)
1st tier 855 2.86 2.44 2.55 2.42 1.20 0.41 0.13 0.10
2nd tier 775 2.54 2.40 2.48 2.36 1.18 0.14 0.11 0.09
3rd tier 708 2.22 2.35 2.32 2.34 -0.21 -0.12 -0.02 -0.02
4th tier 540 1.81 2.17 2.07 2.26 -1.76 -0.37 -0.19 -0.14
The results are more or less what we would expect. GPs were cumulatively +578, or +0.41 per 60 minutes; no one is surprised that they are, overall, plus players. What’s most interesting about this number is how it was achieved. GPs had a cumulative Corsi of +1688, which is good but not exceptional: since the average shooting percentage at even-strength is about 8%, a Corsi of +1688 only justifies a total +/- of 1688 * 8% = +135. Where does the rest come from?
Surely shot quality has something to do with it? After all, we expect good players to be fast skaters and deft stickhandlers, able to deke both defensemen before moving in on the goalie, seeing the fear in his eyes point blank before roofing the puck and skating off to the cries of the adoring crowd. Well, partly. Factoring in shot quality, we see that GPs’ expected goal differential (also known as Delta) is +185, 50 goals better than the Corsi-based +/-. So GPs do indeed manage better shots on average than their opponents, but the effect is minor.
Where does the rest come from? Shooting percentage. The on-ice shooting percentage while GPs were on the ice was 9.1%, a whole percentage point more than the 8.1% achieved by their opponents. As we saw earlier, a small fraction of that difference, about 0.1%, is due to shot quality, but the rest is due to either ability or luck. When looking at individual players over the course of a single season, luck is often, correctly, invoked as the biggest reason for high or low shooting percentages. But over multiple seasons, a players' shooting percentage converges on his true talent. In the same way, when looking at 83 players, luck has been mostly filtered out of the equation. We can see that GPs were on ice for 4013 goals for versus 3588 expected GF, which means they exceeded expectations by 425 goals. Since the standard deviation of this value should be sqrt(3588) = 60, statistical variance cannot be the explanation. GPs really are better at getting the puck in the net than their less-talented bretheren. It’s not just that the best players distinguish themselves: there is a continuum of talents, from the top all the way to the bottom. The drop in shooting percentage from 1st tier to 4th tier players is more or less what we would expect.
PDO: On-ice shooting percentage + on-ice save percentage.
Group Shooting % for Shooting % against PDO
1st tier 9.1% 8.1% 1010
2nd tier 8.3% 8.1% 1001
3rd tier 7.7% 8.0% 996
4th tier 6.8% 7.7% 991
For those not familiar with PDO, it is a statistic developed by Vic Ferrari (explained by Tyler Dellow here that is simply the sum of a team or player’s on-ice shooting percentage and save percentage. The idea is that PDOs significantly above (or below) 1000 are due to good (bad) luck and will regress to 1000 in the long term. While this is true, PDO will actually regress to a player’s “true talent”, which is slightly north of 1000 for a good player and below it for a mediocre one. In other words, good players will manage to sustain slightly above-average shooting percentages year-after-year; however, player percentages vary by far more than 1% year-to-year, so bounces will still dominate. And obviously, since we are sorting by ice time, there is no way here to figure out if a team as a whole has the ability to maintain high percentages, though some do.
Still not convinced? We can add extra results to our study, by compiling these players’ results for 2008-09 as well. As players’ skills don’t change dramatically from one year to the next, the results from 2008-09 are probably still quite representative of our groups’ underlying talents. Granted, some of the 1st tier players in our sample, like Steven Stamkos or Tomas Plekanec, didn’t get 1st tier minutes in 2008-09, but for the most part this is still a good addition to our data. We now have a fifth tier, consisting of players who played in 2008-09 but didn’t get a single minute in 2009-10. We would expect these players to be significantly below-average, and indeed they were exactly that:
2008-09 aggregate results
GP TOI GF GA EGF EGA Corsi +/- Delta Exp +/- (Corsi)
1st tier 5804 77796 3569 3055 3328 3016 3314 514 312 265
2nd tier 5719 70571 2999 2722 2862 2714 1429 277 148 114
3rd tier 5908 68280 2643 2613 2593 2684 -1387 30 -91 -111
4th tier 8231 77566 2410 2933 2699 2938 -2418 -523 -239 -193
DNP 09-10 3339 33963 1119 1334 1254 1325 -412 -215 -71 -33
2008-09 per-60 min results
TOI(s) GF GA EGF EGA Corsi +/- Delta Exp +/- (Corsi)
1st tier 804 2.75 2.36 2.57 2.33 2.56 0.40 0.24 0.20
2nd tier 740 2.55 2.31 2.43 2.31 1.21 0.24 0.13 0.10
3rd tier 693 2.32 2.30 2.28 2.36 -1.22 0.03 -0.08 -0.10
4th tier 565 1.86 2.27 2.09 2.27 -1.87 -0.40 -0.18 -0.15
DNP 09-10 610 1.98 2.36 2.22 2.34 -0.73 -0.38 -0.13 -0.06
First things first: We see that the ice time order of our tiers has remained intact. 1st, 2nd and 3rd tier players from 2009-10 all got less ice time the previous season, while 4th tier players got a bit more (Note that there was about 3% less total even-strength ice time in 2008-09 than in 2009-10 due to there being more power plays called). So our good players were still considered good the previous year, and our grinders were still grinders. No surprises.
How about their results? Once again, GPs were significantly above average, being +0.40 per 60 minutes, almost exactly the same rate they achieved the following season. Their Corsi was much more dominant this time: they were +2.56 per 60 minutes of play, as opposed to only +1.20 in 2009-10, which yields an expected goal differential of +0.20. Once again, we see that shot quality has a small but measurable effect, as GPs Delta exceeded their shot differential by 47 goals, or 0.04 goals per 60 minutes.
Here are the 2-year results for the 4 tiers of players, broken down into the three main sources of advantage:
Group +/- due to finishing +/- due to shot quality +/- due to outshooting
1st tier 0.22 0.04 0.15
2nd tier 0.07 0.02 0.10
3rd tier 0.00 0.01 -0.06
4th tier -0.20 -0.04 -0.15
The unmistakable conclusions from this table? Outshooting, out-qualitying and out-finishing all contribute to why Good Players dominate their opponents. Shot Quality only represents a small fraction of this advantage; outshooting and outfinishing are the largest contributors to good players’ +/-. This means that judging players uniquely by Corsi or Delta will be flawed: some good players are good puck controllers but poor finishers (Ryan Clowe, Scott Gomez), while others are good finishers but poor puck controllers (Ilya Kovalchuk, Nathan Horton). Needless to say, some will excel at both (Alexander Ovechkin, Daniel Sedin, Corey Perry). This is not to bash Corsi and Delta: puck possession remains a fundamental skill for winning hockey games. It’s just not the only skill.
Obviously, the problem with measuring finishing ability remains. Because goals are in short supply in the NHL but shots are in ample supply, it is far easier to measure if a player is good at controlling possession than it is to measure if he is capable of finishing his chances. We can add ice time as an extra proxy for finishing ability if we lack sufficient data otherwise, since the coach who is distributing ice time has probably eyeballed the player’s finishing skill.
One last thing we see is that GPs show no ability to reduce their opponents’ shooting percentages. This is not very surprising, given how you would expect opponents’ shooting percentages to be driven especially by the opponents’ shooting ability and your own goaltender. Also, we are dealing with forwards here, who are even less likely to be able to influence opponents’ shots than defensemen. Vic Ferrari did this analysis on defensemen and came to the same conclusion here.
In Part 2, I’ll take a look at what situations good players are used in, and how they contribute to special teams. See you soon!
Tom Awad is an author of Hockey Prospectus.
You can contact Tom by clicking here or click here to see Tom's other articles.