Starting Pitcher Report

[TOLFG]

darn. Yes, makes sense. I just didn't want to believe it that way, because then the past reports have no value to me. I appreciate you clarifying.

[RAY@HQ]

Think of it this way: when you put in a specific date in the report URL, all that you're really loading (I believe) that is specific to that day are the scheduled pitchers and game matchups. All of the rest of the material on that page, the data and stats, are read from our database as if the date in question is today.

This is one of the reasons I haven't been able to repeat the initial research from a few years ago about the SP Report's accuracy... because we don't archive the scores on the day of the game they apply to, so I don't have an accurate historical record to use as the research input.

[TOLFG]

thx Ray. Do you know if it's just that last 5 PQS that is updated retroactively? or is it the formula pieces that make up the rating as well?

I guess I'm trying to understand why it would NOT be static. If we are using history as research to attempt to predict the future, don't we need history to remain exactly as it was at that point in time? or maybe I'm totally missing something here. You are good at explaining so thank you for all you do on these boards.

[RAY@HQ]

No, the past reports aren't static.

[TOLFG]

another question.....I understand from the article that the Last 5 PQS is not a part of the formula and just appears there as another tool to reference. As I play around in past reports, I notice the Last 5 PQS is actually pulling current data into the older reports, is this by design?

for example: 2012-05-01 shows Chris Sales with Last 5 PQS 5-0-5-5-5. More than 2 years later, today's report shows the same last 5. I checked and it's not coincidence, it's this way for other SP's too.

I would think the old reports should remain static, as they were that day in time. As I do research with them, it brings the more important question......is current data being pulled in other fields to the past reports? therefore rendering all past reports useless?

[RAY@HQ]

1. Correct. Not sure of the exact date, but somewhere around May 1 we cut over to looking at current year data.

2. Also correct... for a pitcher on initial callup, or who does not have a home/road start as appropriate for the matchup, they get a zero for their part of the formula.

[TOLFG]

I wanted to confirm 2 things on the SP report....

1. As of May 1st, it pulls only data from the current year, regardless of sample size, correct?
2. How does it compute rating for an SP who didn't pitch in April due to injury or a rookie coming up? Is the rating for that 1st game only based on the other 3 factors and the SP gets a neutral 0 for his part of the formula?

[stembot]

It would be ridiculously inaccurate to predict Dom. On the other hand, the interplay of a pitcher's home/road Dom with an offensive team's home/road/handedness whiff splits is difficult to visualize without a formula. A "high/medium/low" strikeout favorability rating for a pitching matchup may be helpful.

Can we define the margin of error we're looking for here?

I'd like to think that all baseball player projections are inaccurate to a certain extent. I'm just looking for an edge in knowing when a given pitcher is likely to outperform his projection based on the matchup for that day. Even if it's small, it's something.

[Gantner]

It would be ridiculously inaccurate to predict Dom. On the other hand, the interplay of a pitcher's home/road Dom with an offensive team's home/road/handedness whiff splits is difficult to visualize without a formula. A "high/medium/low" strikeout favorability rating for a pitching matchup may be helpful.

[RDeschain]

How much would it take to include predicted DOM alongside the daily rating figure?

Wouldn't that just be an exercise in futility? I would think it's impossible to accurately predict DOM on a game by game basis and even if you could it would be ridiculously inaccurate.

BUt maybe I'm wrong?

[stembot]

Really appreciate the discussion on this one, too.

How much would it take to include predicted DOM alongside the daily rating figure? While Travis Wood is great at racking up solid PQS scores, his fantasy value is limited due to low strikeouts.

[Gantner]

Good explanation. Thanks Ray.

[RAY@HQ]

Just another thought on the report.

PQS has its place and purpose. It's shorthand, and an improvement over the standard QS definition. When people bring up its flaws it is rightly defended on HQ with terms like "quick and dirty," "simple," "at a glance," etc. It's not meant to be comprehensive; it's a way for us to glance at a box score and instantly evaluate a start without running a complex formula, while looking past some of its inherent flaws such as its dependence on H%, the fact that sometimes an outstanding CG start that "just misses" on a couple categories while dominating others registers a PQS-3 while a more ordinary 6-inning start that barely clears a few categories registers a PQS-4 or 5, etc.

I think what it boils down to is that PQS is made for humans. Computers are way overqualified for this; if they had feelings they would feel insulted to be put to the task of calculating a score made for humans to figure in a couple of seconds while they can do much more sophisticated computations in much less time. So why do we rely on PQS for the pitchers report?

I'm taking Travis Wood as an example. Here's a guy with a 57 BPV and 4.18 xERA on the year, in other words pretty much a dead-average pitcher. His average PQS is a whopping 4.2 however. He has lots and lots of starts where he just barely clears the criteria for a 4 or 5; on the other hand, yes, he is a very solid pitcher who seldom disappoints, and PQS admittedly does capture that fairly well. But I don't think it's good enough for the Starting Pitcher Report. I don't know his full BPIs in home games since HQ doesn't offer those splits, but I do know his home Cmd is 2.2 which is actually worse than his overall Cmd of 2.4. The Cardinals are one of the best offensive teams in all of baseball, not as good on the road but their 3.10 rating on the road vs. LHP is nothing to sneeze at. I'm not sure if I'd recommend Wood for his start tomorrow. I wouldn't rate it a negative, but subjectively I'm thinking somewhere in the 0.80-1.00 range, just looking at the opponent and the fact that Wood is seems to be the very definition of an average pitcher. But he gets a ringing endorsement of 2.28 on his report, all on the basis of PQS.

Of course my "0.8-1.00" guess is purely subjective and has very little science behind it. Maybe the report is right and I'm wrong. It just doesn't seem to pass the smell test here, and in this case it's not due to small sample size. I think the problem is PQS.

As for small sample sizes, it would be great to somehow use HQ's projections (which incorporates past performance) in place of just past performance YTD, since HQ's projections already solve the problem of sample size. Of course, you can't project PQS so if we're using PQS in the report, this idea is dead in the water. But if it was something other than PQS ... or if somehow we could devise an xPQS ...

Probably better for Think Tank. If people like this line of discussion I'll move it over there, but for now, the topic of discussion is established in this forum so I'll start here.

Good post, but I think you're underselling PQS. Yes, we designed it to be easily accessible, in a "quick and dirty" way. And when looking at individual starts, that's going to lead to the appearance that some guys are "beating the system" by just squeaking into a particular PQS score that it looks like they may not deserve (good or bad). But overall, as you start aggregating large numbers of starts, that all washes out in the larger sample sizes.

Wood's skills are average, yes. But his results are better than average, as evidenced by his 2.69 ERA. Now, we know that PQS doesn't look at earned runs at all. But while Wood has a 2.69 ERA, his 78%/0% PQS Dom/Dis split correlates to a 2.84 qERA (that is, the ERA that would be predicted from his PQS scores). That tells me that the "quick and dirty" PQS is doing a very nice job of capturing what Wood has done so far this year. The xERA is more dubious of his results to date, and that's a reason for caution. But just because two metrics are in conflict doesn't mean that's automatically an indictment of PQS.

Are there ways that we could have done this differently, maybe comparing the pitcher's xERA to the BPV of the opposing lineup, rather than using PQS? Sure. But I'm not sure they would be any better at this, in fact I don't think they would be. What we have right now isn't perfect, but it's pretty darn good. And when trying to put a prediction on the outcome of one game, I think that's the classic case where you want to focus on being good, and not letting the unwinnable chase for perfection get in the way of the good.

I don't want to keep just pointing to the research article linked on the report, but the data there does sort of speak for itself. Look at this table:

Rating     Count   Wins   Win %  Avg PQS   DOM%    DIS%
=======    =====   ====   =====  =======   ====   ====
  >3         95     47     49%     3.7      68%    12%
 2 to 3     939    375     40%     3.5      60%    13%
 1 to 2    2362    896     38%     3.2      51%    17%
 0 to 1    1674    554     33%     2.9      42%    22%
-1 to 0     462    140     30%     2.6      39%    29%
-2 to -1    268     81     30%     2.5      29%    27%
  < -2      118     23     19%     2.1      28%    41%

You can see here that the ratings correlate directly to better PQS scores. And we know from other research (as mentioned with Wood above) that PQS scores correlate very well with ERA. Therefore, these ratings correlate to ERA. I'm more than willing to dump the data every season to keep showing this, but I really think everyone keeps focusing on the negative examples and ignoring the positive ones. This thing doesn't work perfectly, but it works pretty darn well at quantifying a particularly difficult problem.

[Gantner]

Also, while HQ is conservative in bumping playing time projections, I'm fine with how it projects skills in the vast majority of cases. At this point in the season it should be a fairly even mix of current performance and pre-season projection, hopefully leaned toward current performance assuming the player has played almost the entire first half of the season.

[Gantner]

I don't think playing time would come into play here, but yes, something BPV-ish. Perhaps PQS could be more like a per-game BPV, using things like GB% and HH% to replace hits and HR in the PQS formula. I do like how PQS also gives us a sense of consistency by showing us DOM% and DIS%, although admittedly DOM% and DIS% only indirectly reflects in the Starting Pitcher Report and I see no reason to change that (a "volatility index" separate from the recommendation score might be helpful however).

If we had that sort of metric, we could perhaps project how a pitcher will perform in all home games for the rest of the season, for example, and express that in some sort of per-game basis.