Post-mortem projections analysis

When a season ends, does HQ review the draft-day projections to see how they stacked up to reality? I know the projections would have to be prorated due to injuries, unforeseen trades, and such. I for one would love to see something like "x% of our players performed within y% of their projected BPVs", or some such analysis. It would also be interesting to know which area of projections carry the most risk (Wins? Saves?)

[hqforums]

Yes, we do a post-season analysis. However, the focus of the analysis is NOT on comparing projected to actual raw data, but on performance as compared to market expectation.

What I mean, for those who have not read me before...

"The best projections are the ones that are far enough away from the field of expectation to alter fanalytic decision-making. It doesn't matter if I project Player X to hit 20 HRs and he hits only 10. It matters that I projected 20 HRs, and everyone else projected 30."

Since we analyze our projections in this way, looking at things like "x% of our players performed within y% of their projected BPVs" is not relevant. We've learned long ago that there are far too many variables during a baseball season (and pro-rating for injuries is NOT appropriate in any case) to achieve any level of acceptable accuracy for those type of analyses.

Some perspective... Is it more important to you for me to tell you that Albert Pujols is going to hit 40 HRs or that Mike Lowell is going to hit a career high 29 HRs? By season's end, the Pujols projection will be far more accurate, but the Lowell projection will be far more VALUABLE.

After doing this for 18 years, I can tell you that this is the best we can EVER expect... no matter what any other self-proclaimed expert hypes in his promotional material. It's the nature of the beast.

RON@HQ

[to93elias]

Still say that a per ab performa comparison (percentage based) would be very helpfull

[Randall@HQ]

to93elias, could you explain a little bit more what you mean? While some stats can be divided down to a per-AB basis (HR, RBI, SB) others cannot (Avg). Also, using AB as the denominator can be problematic in and of itself, because a player who draws a lot of walks (*coughBarryBondscough*) will have warped per-AB values.

[Carlo]

Personally, I think what Ron posted makes a ton of sense.

[to93elias]

In reference to comparison to projections--:

A performance (outcome) comparison to projections prorated down or up to match HQ's projections (ie analagy...projected Durham for 585 AB's,. 285,26 sb's,etc-----ended up with 495 AB's,with corresponding #'s===what compared with projections if ab's same)

After giving Ron's reply some thought, I realized that what he said re: prorating is exactly right. It doesn't make sense for at least two reasons:

1. If a player lost time due to injury, it's likely that his performance in some of his appearances suffered due to playing hurt, coming back too soon, relapses. Prorating the numbers would make him look worse than if he had played the entire year healthy.

2. If a player was projected for 500 ABs and only gets 200, it makes no sense to project his numbers out to 500 ABs. What if Jason Giambi had gotten hurt for the season after two months? His prorated numbers would have been horrible, and completely wrong. Also, some players perform better in a part-time role than a full-time one.