When projections go wrong: The Minnesota Wild
How do you predict where a team will finish in the standings?
The question came in ragey, bullying form via Twitter…and as frustrating as that may be to deal with…a good question is a good question.
The example? The Minnesota Wild. Hockey Prospectus’s VUKOTA projection system was not a fan. The system, based on past statistical performances – not human instincts or hunches or even things we might infer such as Player X will get top line minutes in Minnesota when he received fourth-line minutes in New York.
Now, the HP write-ups in the book cover these things. They point out the issues with the projection and what might go differently from what the numbers say. But some can’t get past the number.
So here is what happened with the Wild:
VUKOTA knew Minnesota could not score goals. It projected them to finish 29th in goal scoring. They are currently 26th. Why? There were very few proven goal scorers present outside of Jason Pominville and Zack Parise. The projection does not favor aging players like Dany Heatley and Mikko Koivu because players past their prime tend to see a reduction in points. In this case, Koivu beat the clock. Most players do not.
The Wild also relied on young, unproven players. Ones that made projecting them statistically even more difficult. For example, after scoring one NHL goal in 55 games in 2011-12 and spending 2012-13 in the minors, Nino Niederreiter has posted 34 points. He spent the most ice time with Granlund, Pominville and even 122 minutes with Parise. Maybe if you covered the team from training camp on, you would have known from Day 1 he was going to get chances, but by statistical measures, even factoring for AHL production and age, the numbers would never say a John Scott-like year would turn into 34 points.
Is it a shortcoming? Of course. Statistics done a month before the season can not predict circumstances. There are just far too many options. It was just as likely Nino would score 13 goals as he would end up back in the AHL. A projection system designed to gave credit to players like that would be put together illogically. In that case, you’d have a lot of 4th liners projecting to be big time producers. Instead, you deal with the outlier.
Mikael Granlund also out-performed his projection, despite VUKOTA factoring his age and coming out with a double-up in production from last season. Same with Charlie Coyle.
Historically, players take a step from age 20 to 21. These two took leaps. And good for them. But if the numbers say that most players only take a step, how would you ever guess these two fellows would leap? Only by trying to arbitrarily weigh the numbers, which would be messy and involve bias.
There were defenseman who out-performed their past too, namely Marco Scandella, who was an AHL player for all but six games in 2013, yet has turned into a very solid defenseman. Last time he was in the NHL full time, he was minus-22, this year he is plus-10. (Yes, we know about the shortcomings of plus-minus). The point is, that makes four players who out-performed their past by a mile.
Trading for Matt Moulson is super obvious. A projection system can not guess trades that will happen in March. Moulson is an impact player and has scored big goals for the playoff-bound Wild. If you want a system to guess trades, there are other places on the internet for that.
And the biggest factor: Goaltending.
Based on the past numbers of the Wild netminders, VUKOTA said they would end up 20th in the NHL in goals against. Instead, they are fifth.
Why? One is a trade for Ilya Bryzgalov. The other is that Josh Harding performed far, far above his career statistics.
Harding has a .933 save percentage, while his career save percentage is .918. The system takes a big sample to make a prediction. If it randomly predicted an above average goalie to score higher than most Vezina Trophy winners, something would be seriously wrong.
And .933 and .918 are no small difference. The Wild are plus-3 in goal differential. Over 1,000 shots, Harding’s hot streak would be a gap of 15 goals. Slip him back to his career average and – BAM – you have the Wild at minus-12…or almost exactly the same as Winnipeg, who is in last place in the Central.
It’s not just Harding, either. Bryz has a .923 save percentage – far above what you would expect from a backup – and Darcy Kuemper was above average at .915. Would any system project an AHL goalie from the year before to be above average? Sure, it happens, but you wouldn’t expect it.
Should we have known that some of these things would be different than the statistical projections. Yes. Oh, wait, we did!
From the book:
“Minnesota is a bubble team and a healthy Josh Harding might just be what ultimately makes the difference,” – Robert Vollman.
Harding wasn’t healthy the whole time, but he was a big difference maker. And they are a wild card team…which sounds like a bubble team to me.
So, what we have here is a perfect storm for a projection to short sell a team because of past performances and goaltending. VUKOTA’s 2014 projections had several teams out-perform the stats it spit out and others land right in the same ballpark.
The numbers should give you an idea of what might go right and what might go wrong. If you think they are a be-all, end-all, set-in-stone, bet-your-house prediction, you might be a little naïve or using the numbers to take unnecessary shots to make yourself feel smarter than you are.
As always, Hockey Prospectus says take the full picture. That’s why the projections include 1,000-word write-ups. If there were no other factors, it would be a lot easier to write.
And, good job Minnesota Wild. Could be an upset team. I’d predict it, but that might just be asking for more bullying.
Matthew Coller is Managing Editor of Hockey Prospectus. He is the long-time host of Hockey Prospectus Radio, producer of the Howard Simon Show on Buffalo’s WGR550 and their Rochester Amerks reporter, and a multi-sport play-by-play announcer.
Follow Matthew on Twitter at @matthewWGR.