Thursday, August 23, 2012

Criminal Ballpark Factors

Remember, WAR judges everything. Fielder is a poor baserunner (one run below average), a lousy fielder (six runs below average) and Comerica Park rates as a slight hitter's park this season.

The quote is from a piece that Dave Schoenfield wrote where he tries to explain the value of WAR. He introduces it as a tool that attempts to quantify performance while normalizing for different ballparks, eras, positions, etc. Unfortunately it's not that simple, and since there are only a couple sources that do WAR calculations the general public accepts the values as truth. Now for the most part the websites do an excellent job and work hard to keep updating their algorithms. Personally I prefer Baseball-Reference's version over Fangraph's due to the peripherals used in Fangraph's pitching calculations, but I'll often go back and forth between the websites depending on the tools I need. I'm absolutely sick of the park factors calculations. "Comerica Park rates as a slight hitter's park this season." What, as opposed to last season? Did they move the fences? Did they change the size of the infield? The only thing that could have changed from last season is the weather and I'd be shocked if that was accounted for in the data Dave used to write that line. So is it more likely that the characteristics of the entire MLB hitting and pitching profiles have changed since last season, or that people just happen to be hitting well in Comerica Park this season?

I'll rip on ESPN first because it's been a hobby of mine lately before I move onto others who calculate park factors slightly better. The formula ESPN uses to calculate park factors is
  • homeRS = Runs scored at home
  • homeRA = Runs allowed at home
  • homeG = Number of home games
  • roadRS = Runs scored on road
  • roadRA = Runs allowed on road
  • roadG = Number of road games
A value over 1 favours the hitter while a value under 1 favours the pitcher. This formula is based not on the actual characteristics of the park, but instead on the performances that occurred in the park. That makes the park factors extremely circumstantial and subject to high variability. As I mentioned before, park factors should not drastically change from year to year as all that changes in the park usually is weather conditions. If the makeup of players in general hasn't changed there is nothing inherent to the parks that would make the players perform any differently from the season before. The data from runs scored isn't nearly sufficient enough to draw meaningful conclusions. The following graph shows the ESPN Park Factors for runs for 6 parks from 2004 to 2012.

If I were to take this graph seriously I'd be forced to conclude that in the span from 2004-2006, Great American Ball Park went from being a great pitchers' park to being a better run scoring environment than Coors Field. Safeco was close to a league average park from 2005-2009 but this year is playing like one of the greatest pitchers' parks of all time. Miller Park goes back and forth between being a hitter's and pitcher's park. Same with Rangers Ballpark. That makes sense. Admittedly, there is a flaw in using the 2012 data since this season isn't over, but small sample size is hardly this data's only problem. Changes in other parks will also affect this data since it is all relative, but not by this much, and not with year-to-year changes. The variance can be seen across all parks, and especially when trying to determine park factors for different types of hits. The following table shows the average and standard deviations of park factors for ball parks that have been open from 2004-2012.

Park factors from 2004-2012 based on the average of single year park factors

The fact that they would even include data on triples for park factors is insane. There are an average of about 30 triples a season in each park. This makes the error on the triples park factor about 0.26 just from random deviations. How can any meaningful conclusions be drawn when the error is so significant?

As I mentioned also, these park factor calculations are entirely circumstantial. Last season Citizens Bank Park had a runs park factor of 0.997 making it a slight pitchers' park. Coincidentally Cliff Lee started 18 games at home and 14 games on the road. What happens if we balance that out? The difference between Cliff Lee and the rest of the team over 2 games is about 5 runs. With that change the the Phillies park factor goes to 1.01. If we take it a little more extreme and move a couple of Lee's better outings, two shutouts, from home to the road, and move two of the Phillies weaker outings, two 7-run games, back to Citizens Bank Park, we get a new park factor of 1.04. Due to a slight imbalance in Cliff Lee's scheduled starts Citizens Bank went from a hitters' park to a pitcher's park. What did Cliff Lee do to make Citizens Bank Park into a pitchers' park just by starting there? Did his act of starting change the size of the park. If so I for one would like to see what other magic tricks he's capable of. Also, it's insane that these numbers are being reported to three decimal places.

The weather in many parks can significantly change the run scoring environment month to month. More northern parks tend to have worse run scoring environments when the season starts in the colder months of March and April. According to the way these park factors are calculated, if the Blue Jays opened the season in Seattle in the lower run scoring environment, this would actually make Rogers Centre into a better hitters park. No other parks would be affected. Somehow a team's road schedule affects how well their home park plays.

There are ways to manage the variability and websites like Baseball-Reference to a better job in calculating their park factors. They use park factors averaged over three year periods and account for things which can affect game score, such as the home team not batting in the bottom of the ninth or by using the fact that some teams play some more than others so teams don't all have equal effects on each other's park factors. This is a step in the right direction but to me it's not nearly enough. It's basing the park factors on empirical results instead of the actual sources of the park factors themselves, the physical ballpark. It seems that the layout of the fences should be a more important determining factor than who actually hit the ball over those fences. Maybe the results do average out and approach the "real" park factor numbers, but why should we be forced into that scenario.

We're still far off from the proper solution though. Look around and try and find the exact dimensions of ballparks. I've tried and that information is not available. You can find the distance down the lines and to the power alleys, which is itself a laughable way to define a ballpark, but if you want the exact dimensions of the entire park, you're screwed. Without even knowing how big the outfields are, how are we to put any faith in park factor numbers. I sincerely hope that team's front offices are doing a better job than the public. That they've actually made measurements about the size of their park and use that information when making personnel decisions.

A fundamental question that still hasn't been asked is "what is the point of park factors?" What information do we want to get out of them? Taking the current park factors and using them to normalize WAR values doesn't make sense. Park factors as they are cannot be applied uniformly across all players. Each player would have their own park factor in every stadium depending on the player's hitting profile. Think of an asymmetric stadium where the left field line was 300 feet and the right field fence was 500 feet. One park factor can't be applied fairly to both a left handed and right handed batter. The numbers need to be individualized. A GM could much better custom build his lineup to his home park if he knew how it affected each player, not a player in general.

Maybe front offices really are a step ahead and are calculating these park factors properly. It's not enough though. Writers, fans, agents, etc. use sources like ESPN and Baseball-Reference for all purposes in the industry. BBWAA voters don't vote for Larry Walker because of the Coors field effect. We know that Coors field is generally a better hitter's environment, but did it affect Walker to the same extent that it affects others? Also, the longer these numbers and formulas are used, the more ingrained they become in baseball culture and harder to correct. Their use becomes more widespread and the flaws become like a virus affecting other stats. Calling something an advanced metric does not make it so, the reasoning behind the formula needs to be sound too.

No comments:

Post a Comment