Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Friday, September 19, 2008

A thin correlation between love and hate

WARNING: Reading the following article will make you an asshole. Not quite a just-finished-reading-Atlas-Shrugged asshole, more like a guy-at-the-office-who-explains-why-you-paid-too-much-for-your-new-car asshole. So, if you don't want your friends to hate you because of your new found proclivity for bloviating about statistical misconceptions, you may want to skip this week's article.

Consider yourself warned.

(Yeah, I just said bloviating. That just happened).

I love the start of the Football season. The pageantry of the college game, the intensity of the NFL, and, most of all, the idiocy of football commentators. It would seem, the only qualifications for discussing football on TV, radio or in print are a lukewarm knowledge of the game, a below average intellect, and a penchant for making controversial statements. (Yes, I do realize the inherent irony of that statement coming from me.) Furthermore, since so many people discuss football, everyone tries to come up with new, original insights, no matter how absurd, just so they can claim the idea as their own.

If Carson Palmer completes 73.275% of his passes, the Bengals will win tonight.

If Georgia turns the ball over less then three times, scores more then 26 and three quarters points and limits USC to 316 passing yards or less, they will definitely win.

As long as Jon Gruden finishes his grilled cheese sandwich by half time, Tampa Bay should win on Sunday.

I'm confident that if the University of Hawaii scores more points then San Jose State, they'll have a good chance to win.

I find this unendingly funny, because football is, inherently, a simple game. The team that controls the line of scrimmage and protects the ball usually wins. Other then that, you're splitting hairs and over analyzing a game between twenty-somethings with a ball that doesn't bounce straight.

That's not to say that there aren't valid statistical indicators. How well a team runs the football, how well a team stops the run, and a team's turnover margin, all provide quality insight about a team. But, outside of those, which, as you may have noticed, simply indicate how well a team controls the line of scrimmage and protects the ball, most other statistics are simply flashy numbers that commentators misinterpret to prove their own preconceived notions.

My favorite statistic, bar none, that commentators routinely misinterpret is the correlation between how many rushing yards a team gets and whether or not they win.

Today, while predicting the outcome of various college football games on his radio program, Colin Cowherd said the key to the Florida-Tennessee match up, is who does a better job running the football. Colin came to this conclusion because the team that rushed for more yardage won the last six match ups.

Wow.

That's amazing.

Unbelievable, really.

I think my mind exploded.

It's brilliant, except, of course, for the fact that it's not at all true.

What Colin failed to realize is the distinction between correlation and causation. Correlation says whether or not two things are related, like rushing for more yards and winning football games or watching sports on my couch drinking beer and having the day off. Causation, on the other hand, says whether or not one thing causes the other. You see, just because I'm typically watching sports on my couch drinking beer on my days off, does not mean that watching sports on my couch drinking beer causes me to have the day off. (Believe me. I've tried.) Simply that the two occur simultaneously with relative frequency. Similarly, that fact that teams that rush for more yards typically win football games, does not mean that rushing for more yards then the other team causes teams to win football games. In fact, it's the other way around.

When teams have the lead, they typically rush the football to take time off the clock and protect the ball. Conversely, when teams are behind, they typically throw the football to try and score as fast as possible. Because of this, teams that win, typically have more rushing plays, ergo, bingo bango, more rushing yards. Rushing more then your opponent doesn't cause you to win. In general, winning causes you to rush for more then your opponents.

I sincerely apologize for bringing this to your attention. From now on, you will officially be "that guy", whenever someone claims tells you that their team needs to get their running game going. But, hey, don't blame me. I tried to warn you.

Now if you'll excuse me, I've got a date with my couch, a Cubs game and a six pack of Heineken. Because, you know, never hurts to double check...

Friday, November 02, 2007

Soldiers of Fortune

Aloha, friends and J-Bors. Let's start things off with a few college football questions:

  1. Is the University of Hawaii one of the top twelve football teams in NCAA Division I football?

  2. Is Ohio State the best team in the country?

  3. Is LSU better than Oregon?

  4. Does Kansas deserve a shot at the national championship?

  5. Is Colt Brennan a better Quarterback then me?

Hint: Each question has the same answer.

Hint: No, is not the answer. (Mea culpa for the double negative)

Hint: Yes, is not the answer either.

Confused?

The answer to each question is that you have an insufficient sample size.
No one can definitively say whether Hawaii is one of the top twelve teams in the country, because they haven't played any team ranked in the Top 25. Similarly, no once can definitively say whether the best team in the country is Ohio State, LSU, Oregon or Kansas because none of the teams have played each other. Lastly, no one can definitively say whether Colt Brennan is a better Quarterback then me, because they haven’t seen me play. (Seriously, if you saw my deep ball… My cell phone is always on, Coach Jones.) But I'll take it a step further. You could not definitively say whether Ohio State, Boston College, LSU, Oregon, Kansas or Hawaii was better then any of the other teams even if they did play.

College football is an unpredictable sport, played by 20 year old kids with a ball that doesn't bounce straight. Crazy things happen. Whether it’s fumbles, interceptions, injuries, hail marries, torrential downpours or snow storms, college football embodies Murphy’s Law. Therefore, making a definitive judgment based on one game isn’t just difficult, it’s impossible. So far this season, the #5 ranked Michigan Wolverines lost to an Appalachian State team that doesn’t even play Division I football; USC, winners of two of the last four national championships, lost to lowly Stanford, who played the second half back-up quarterback; There was a stretch where the number two team in the country lost to an unranked opponent four weeks in a row!

This vast volatility leaves pollsters and analysts scrambling for a way to fairly and accurately rank teams that have a handful of games, with incredibly dissimilar schedules. Needless to say, finding a fair and accurate way to rank college football teams is about as easy as getting fair and balanced coverage from Fox News. In lieu of any concrete means of comparison, analysts often resort to comparing games against shared opponents. For instance, the University of Hawaii needed a last minute field goal and overtime to beat Louisiana Tech 45-44. The next week, Cal torched La. Tech to the tune of 42-12. On paper, that seems to suggest that Cal is a far superior to Hawaii. Easy enough, right?
Unfortunately for Hawaii, that does not take into account the four Hawaii turnovers, the 103 degree game time temperature and 100% humidity, which caused the Hawaii Center to sweat so profusely he couldn’t hike the ball, or the 4000+ mile road trip Hawaii traveled to play La. Tech – To put that road trip in to context, LSU, the number two team in the country, will travel an estimated 1308 miles combined on all their road trips this season and had only one road trip longer then 300 miles, which they lost. Comparing Hawaii’s win over Louisiana Tech to Cal’s win is like comparing David Caruso’s work on NYPD Blue to his performance on CSI: Miami. Both feature over-dramatic punch line deliveries, scads of unintentional comedy and more melodrama then a freshman girl’s introduction to vodka, but the roles have such different context – in NYPD Blue he was the rugged, excessively cheesy up and comer, whereas in CSI Miami he doesn’t realize that we’re laughing at him, not with him – that they’re virtually impossible to compare.

But let’s imagine, for a second, that we live in some magical fairy tale kingdom, where we can accurately compare independent events from vastly dissimilar context. Let’s call it, say, ESPN. Even in the gum drop forest of this pristine imaginary utopia, we cannot escape the laws of probability.

Warning: The following section contains math-related material. Parental discretion is advised. (Becaues really, does anything make you want to read/watch something more then "Parental disrecresion is advised?")

As we discussed before, almost anything can happen in college football. Even the best teams have off days; Upsets happen all the time. So even the best teams in the country don’t have a 100% chance of winning when they step on the field. In fact, the best teams in the country have about an 80% chance to win each game, on average, which roughly equates to a 10-2 season.

From a purely statistical perspective, a twelve event sample size for an event that has an 80% chance of success has a margin of error of 11%, or 1.3 games. To say that in plain(er) English, in Football, a game in which one game represents the difference between a historic season and utter disappointment, one game a season is almost purely attributable to chance.

A moment of silence to let that sink in. (Sorry for ending a sentence in a preposition, but I’d sound like an absolute ass if I said ‘A moment of silence, in to let that sink.’)

That means, in any given football season, a team that should finish 10-2 has an equal chance of finishing 9-3, and out of the running for any major bowl game as it does to finish 11-1 and in the running for the national championship game. That does not mean that the team that went 10-2 (or 9-3 for that matter), is necessarily worse then a team that goes 11-1; It could just as easily mean that the team at 11-1 got a fortunate bounce from the oblong leather ball during one moment of their incredibly short season.

As American’s we tend to trust what our eyes can see or, as we move further in to the digital age, what our formula’s can deduce. We want to explain. We want the illusion of control. We want to confidently declare, my team is, without a doubt, the best team in the country. But, we can’t. This drives us crazy and leads to numerous injustices (like a 12-0 Auburn team missing out on the national championship game to a 11-1 Oklahoma team, who lost their last game of the season. Seriously, can we make a rule that any team who loses their last game of the season DOES NOT deserve to play for the national championship? Do I even need to bother telling you that the two teams for who this happened got slaughtered in the national championship game????) but it, like every cloud, has a silver lining.

The element of chance provides the opportunity for non-traditional powerhouses to make history in college football. In the past few seasons, we’ve seen Utah and Boise State play in BCS Bowls. This season, the University of Hawaii, a team never before ranked in the top 15, located 2500 miles away from its nearest opponent have a chance to knock down the door. Meanwhile, Boston College, Oregon and Kansas flirt with history as they attempt to qualify for the national championship game. Less games means less opportunities for teams to revert to the mean, which means more chance for the unexpected and more opportunities for an unheralded team to sneak in the back door (I hereby, solemnly swear, that this is the last time I ever use the phrase “sneak in the back door”.)

Similarly, the near infinite randomness of college football makes every game a “can’t miss” experience. It provides a heightened importance to every game, something no other sport can claim. Even a die-hard college basketball fan will miss a game or two each season. A die-hard college football fan will cancel flights and ask friends to postpone weddings in order to watch each game. (Which any true friend would gladly do.) Basketball games are events. Baseball games are events. Football games, on the other hand, are moments in history. As Ivan Maisel, senior college football writer for ESPN deftly put it, “Name me one regular season college basketball game that you remember.” (I have to go all the way back to the mid-90’s and UH’s Justice Sueing elbowing future NBA player Keith Van Horn of Utah in the head. To this day, the best elbow I’ve ever seen in a basketball game. What does it say about college basket ball that the last regular season game I remember involved an elbow to the head?)

This heightened sense of importance also fuels the fire of the hype machine, that drives college football. Debate over rankings, bowl games and national championship appearances starts before the season begins and often doesn’t stop until well after the season ends. Every other sport in the country, from college to the pros, answer their questions on the field. In college football, the burning questions are rarely answered on the field, so the ESPN’s, Sports Illustrated’s, Sporting News’s and the like are left to debate them off the field. It’s what makes the debate so compelling and, in all honesty, why you’re still listening to me blabber on about probability’s effect on college football for over 1500 words.

At the end of the day, as much as we want to compare and analyze college football teams and as much as we want to drink the kool-aid when ESPN proudly declares the superiority of our team, this chance, this randomness, this undeniable destiny of college football and laughably small sample size prevents us from making the type of declarative statements, which make us feel most comfortable. So stop trying to make sense of it all. Just sit back, relax and enjoy all that makes college football special.

Enjoy the tradition. Enjoy the pageantry. Enjoy the school spirit. And, of course, enjoy the randomness.

So next time someone asks you if Hawaii is one of the top twelve teams in the country, tell them the sample size is too small to say and no one will ever know, but as long as the polls say it, it’s alright with you.

If they then ask you whether I’m a better QB then Colt, tell them, at least I’ve got the hair…