Friday, June 15, 2012

Leveraged Bullpen Use

ESPN decided to really go after the outdated save rule and it's effects on bullpen management, or more appropriately mismanagement, today (see CapleSchoenfield, or Cameron) so I thought I'd join the bandwagon and look at how efficiently relievers are being used around the league.

I'm going to compare how the runs each teams' bullpen gives up once the leverage of the situation is taken into account. To do this I'll be using Fangraph's Leverage Index, specifically gmLI, a pitcher's average Leverage Index when he enters the game. I'll be using a metric called Leveraged ERA (L-ERA) calculated for each team's bullpen. I calculate L-ERA by first weighting each pitcher's earned runs by their respective Leverage Index (ER*gmLI). This is then summed for each pitcher in a teams bullpen and divided by the total number of innings pitched. This value is the unnormalized L-ERA. This value needs to be normalized by the team's Leverage Index in order to account for the fact that different teams face different leverage situations. The Twins bullpen has a particularly low Leverage Index because they are usually so far out of the game by the time the bullpen needs to pitch. The team LI is calculated by calculating a weighted average of the players' Leverage Indexes by the amount of innings pitched. The unnormalized L-ERA is divided by the team LI to get the final L-ERA value.

L-ERA weights the runs the bullpen has given up according to the leverage of the situation. Runs given up in a close game are much more important than those given up in a blowout. The difference between a team's ERA and L-ERA can determine how effective the bullpen has been and how appropriately it has been deployed. Along this vein it is important to note that the L-ERA number alone cannot distinguish between clutchness and management, although they are definitely related. We'll look through some of the extreme cases to try and determine whether the difference was caused by extreme performances or by managers deploying their less effective relievers in more important situations. Table 1 shows the ERA, L-ERA, and the difference between the two (Δ) for each team's bullpen. A higher Δ value indicates the bullpen has been extremely effective, having the best performances in the highest leverage situations. Similarly a low Δ value indicates that a bullpen has performed worse (or used worse) in more important situations.

Table 1: Bullpen ERA and L-ERA. Δ = ERA - L-ERA

We see that the bullpens of the Giants and Rangers have performed the best in higher leverage situations. Since Brian Wilson went down Sergio Romo and Santiago Casilla have performed incredibly well and have been used in high leverage situations. The Giants bullpen could probably be even better as Javier Lopez and his 3.29 ERA has thrown 13.2 innings in the highest leverage situations on the team. The Rangers seem to be benefiting much more from clutch performance than solid bullpen management. Robbie Ross has the lowest ERA and has thrown the most innings on the team, but he ranks only 4th among relievers in terms of leverage situations. Just like the Giants, the Rangers have also been giving their highest leverage innings to one of their weaker performers, that being Mike Adams and his 3.32 ERA.

At the other end of the spectrum we have the Red Sox and Astros. The Red Sox have been relying heavily on Vincente Padilla and Alfred Aceves to anchor their bullpen and the results haven't been great so far. These two players have the highest leverage situations on the team and both have ERAs over 4.50. Meanwhile Scott Atchison and Matt Albers have thrown 61 innings with an ERA under 2, but have been used almost exclusively in below average leverage situations. As for the Astros, their best reliever Brett Myers has been used in the highest leverage situations, but no where near often enough. Myers is only 4th in the bullpen in terms of innings pitched and he can easily pitch more given his past as a starter. Maybe the Astros are keeping him rested for a trade? So instead of letting Myers pitch the important innings, they've been going to Fernando Rodriguez and his 4.56 ERA.

Much has been made about the Phillies and their use, or lack thereof, of Papelbon. It might be a little surprising then to see the Phillies in the upper third of teams in terms of differential. Similar to Houston, the problem hasn't been Papelbon's performance, it's been his under use. Papelbon and Antonio Bastardo have been two of the best relievers on the team and have been used in high leverage situations. Unfortunately so has Chad Qualls who leads the team in innings but has a 4.56 ERA.

Sometimes a manager doesn't know exactly how a pitcher will do until he takes him for a bit of a test run. This often follows a call-up from the minors and is then frequently followed by a subsequent option back to AAA. To try and get a better reading on mainstays in the bullpen, I repeated this analysis using only pitchers that have pitched at least 10 IP for their respective team. The results are shown in Table 2. Not surprisingly the overall ERA drops from 3.65 when all pitchers are considered to 3.36 when only the 10 IP pitchers are included. Better quality pitchers have a better chance of sticking around. This also eliminates some of the more fluky incidents such as Darnell McDonald giving up 3 runs in a very important 17th inning.

Table 2: Bullpen ERA and L-ERA, Minimum 10 IP/player. Δ = ERA - L-ERA

Unfortunately for the Red Sox eliminating McDonald doesn't eliminate Aceves' early season blowups. We do see some shuffle in the chart though. The Angels jump way up the list by realizing that Bobby Cassevah and Kevin Jepsen belong in the minors. The Astros also look better as they the 10 IP minimum removes 4 pitchers from their bullpen, although they still need to see Myers more.

These results should be taken with a bit of a grain of salt. I used each player's average gmLI to determine the L-ERA. It would be more informative to weight each run by the exact leverage index of the situation it was given up with. Similarly, although the normalization tries to make it so that each team is compared to itself, since we are comparing the differences between teams it isn't entirely fair to use different normalization values for different teams. It isn't entirely a pitcher's control what the leverage situation is when he enters, he can only affect his own performance. A batter entering needing to get one out to close an inning and getting a strikeout has performed his job equally well whether the bases were empty or loaded. However, to lower the L-ERA with the highest probability, a manager can make the active decision to put his best reliever in when the stakes are raised. It is also important to note that the average bullpen has only thrown 186.2 innings so far, the results will become much more meaningful once the sample size can increase at least beyond an expected starter's workload.

No comments:

Post a Comment