Wisdom of the crowds is winning

Editor’s disclaimer: online gambling is illegal in Washington state. Don’t do it. This is for fun.

Just this season, Fangraphs.com launched its Game Odds on the scoreboard page. These Game odds show the daily probabilities of each team winning, adjusted for the team’s lineup and those players’ ZiPS and Steamer projections. I have collected more than 500 such games of information, recording the pre-game Fangraphs probabilities along with the results of the game.

It’s impossible to predict game results in sports without quite a bit of error. Games like Germany’s 7-1 win over Brazil in the World Cup and the Mariners 13-3 smashing of the White Sox last Thursday are always going to be unexpected outcomes. In baseball, even a good projection system will still be off by an average of three or four runs in projecting game run differentials. It should come as no surprise that a statistically significant relationship exists between the Fangraphs pre-game probabilities and the eventual run differentials. So even though the standard deviation of Fangraphs’ error was 4.1 runs, the pre-game probabilities still held some predictive ability.

The actual win-loss results of the games are important, too, I hear. A logistic regression suggests that for a 1.5-times-increase of projected odds of winning, the actual odds of winning go up by about 1.25. That was also statistically significant. For people not comfortable with odds, here’s a chart of how often teams actually won when the Fangraphs pre-game probabilities had them projected in the given ranges.*

Projected	Games	Wins	Pct
25 – 35%	37	13	35.1%
35 – 45%	108	47	43.5%
45 – 55%	219	97	44.3%
55 – 65%	147	83	56.5%
65 – 75%	39	21	53.8%
75 – 85%	2	2	100.0%

*Only one outcome from each game was recorded (say, Mariners win 4-2) since the opposite outcome (Braves lose 2-4) would be repeated information. 

It’s not a perfect correlation by any means, but you can still see it. For the most part, if the Fangraphs pre-game probability is greater for a collection of teams, then they do indeed tend to win more. However, the sample size is too small at this point to confirm or reject Fangraphs’ precision.

What’s more interesting to me is that sports book bettors to this point have been doing…better.** For each game in the data set, I also collected the betting lines from Sportsbook.com so I could compare the statistical model to the opinions of bettors. While neither the bettors nor the Fangraphs pre-game projections were able to break a standard error of four runs when predicting run differentials, the bettors produced a slightly smaller error, as well as being able to predict a greater percentage of the variance in run differentials.

**Ugh. I walked right into that one.

The Sportsbook betting lines are just another version of what is often called wisdom of the crowd. When a bunch of somewhat educated people make conjectures about the true value or outcome of something, the average of all their conjectures is often quite close to actuality. Granted, my sample size of 551 games is not enough to say definitively that Sportsbook betting lines were more precise than the Fangraphs’ statistical model. But the fact that I am unable say that a highly educated statistical model is performing better than the average of a group of somewhat educated people is kind of cool!

I don’t think this should be seen as a slam on Fangraphs–especially since there is not a large enough sample size to even be sure its models are worse that the bettors’ collective opinions. Rather, I think it’s just another example of how powerful a group of appropriately motivated people can collectively know more than a well-constructed statistical model.

While I have made an effort to confirm starting pitchers, I admit that I was not always able to double check at game time. I did make sure that the starting pitchers on the Fangraphs page matched those on Sportsbook’s website.