The Baseball Analogy

Baseball has already undergone its statistics revolution. While we still have some stubbornly ignorant fans that cling to RBIs and wins, most have accepted that these have huge failings in terms of actually determining how good a player is. Everybody realizes that run support has nothing to do with the pitcher (ok, a little bit in the NL), and most can now see that the distribution of which games the pitcher allows his runs in can be chalked up to luck. In fact, what we see now for pitchers is that all they really control is their strikeout rate, their walk rate, and their ground ball rate. This leads us to some examples of the two main kinds of luck that falsely and disturbingly influence our opinions of players. The first is the "pure randomness" kind of luck. Twenty percent of non-home-run fly balls go for hits, so if a pitcher has the misfortune of allowing a .300 average on in-the-park flies, we can all acknowledge that he just had really bad luck. But this is usually only counted against a pitcher when people just plain ignore advanced statistics (either by laziness or by principle). On the other hand, we have the "clutch" kind of luck. A pitcher has the skill to strike out 20% of the batters he faces. His luck is WHICH twenty percent of the batters he strikes out. We hate admitting that this is luck. We want to say, "He rises to the pressure," that "He came to play." But these are just our feelings getting in the way. He's always ready to play. He's a professional athlete who has practiced this particular skill for his whole life. If he really could just muster up all his focus and strength to get that one strikeout with the bases loaded, he would do it every time. But he can't. And statistical studies have confirmed this. There aren't pitchers who are consistently "clutch."

Now let's look at an almost identical example in football. A quarterback's skill is to get the ball within a certain range of where his receiver can reach. Let's say he hits this spot 60% of the time. It's luck WHICH 60% of the time he does. If he happens to do it on third down, or if he happens to do it in the red zone, he's just luckier. That's all. And that's my biggest qualm with the more advanced statistical measures that I look at. They all want to attribute this luck to a quarterback's skill. Even Football Outsiders' DVOA, the best team ranking metric I've seen, not only falls into this trap, but is proud of it. They base the success of a play (at least somewhat) on the percentage of the distance-to-go that it gained for the down it was run on. While there is some value in this (both the offense and defense will use a very different play call on 3rd-and-short than on 2nd-and-long), most of this adjustment just gives importance to random noise. It's too complicated for its own good (On the other side, an oversimplification such as points or wins will "subconsciously" make this same adjustment for luck, and do it even worse than DVOA does).

And that's why I brought in an algorithm that finds a middle ground. I plan to publish the full team "power" rankings on Tuesday. However, while I think that even in the beginning they absolutely do give value beyond an intuition-based-ranking (like what you would find on nfl.com), until week 3 or so, that intuition-based-ranking is going to be more accurate than my power rankings, simply because 1 or even 2 weeks just isn't enough of a sample size, even with the algorithm, to judge team performance or to adjust for the skill of opponents better than a combination of watching the games and considering performance the previous season (which is really a subconscious Bayesian adjustment, so good for them).