The Margin of Error of Projections

[jackvdo]

Just an addendum to what I said above as I looked at the blog referred to by Michael above. In my humble opinion, that analysis is flawed because it only looked at batters with 500+ PA and it didn't say what pitchers, but I assume those with a lot of IP. I think we all could reasonably forecast the performance of those players with that large of a data set. As I said, it is those players with the small set of stats where the pros really shine. My ERA analysis looked at all players and BP clearly outpaced BBTF.

Of course to be totally honest, this reinforces Mr. Shandler's perspective of the uselessness of such an analysis but again I would maintain the usefulness is in for what purpose the analysis is done.

[gtgator]

I agree that, for me, HQ is there to help spot trends (up or down) in players as well as identifying those players whose stats are supported by the skills and those supported mainly with luck. And, as with bankerboy, I find it most helpful in the end game.

I also agree that picking one stat and comparing projections is inane. In fantasy, I only care about how close a projection is in the stats we use in our league. OPS may be great - but isn't one we use, so why do I care who did best at predicting that?

Even then, I understand the vagaries associated with wins and saves, which means those stats are essentially educated guesses. I mean, in 2005, if I told you Clemens would pitch all year and have a sub-2.00 ERA, I doubt anyone would have projected only 13 wins. And same goes for a hitter who has several balls hit off walls during the year as opposed to being hit a couple feet higher and having them become HR - literally a fraction of an inch difference in the point of contact between the bat and ball. No prediction system can predict that. RBIs, SB opportunities (and therefore SBs) are also contingent on external factors outside the player's control. As such, predicting actual numbers will always be tough.

But a prediction system that can predict skills that will be exhibited? That is where I think HQ is best (along with playing time). How those skills will turn into actual numbers is contingent on plenty of factors that can't be predicted, such as luck, personal issues, etc.

As for PECOTA - I think to some extent, they use skills as well, though they either don't know it or won't admit it. By comparing a current player to players with similar builds, ages and past performance of those players, they seem to guess that a player will do X -where "X" is the average of the comparable players (along with other factors such as home park, etc.). Since those averages are basically based on past performance, and since past performance is based, in part, on skills, they are effectively attributing the comparable players' skills to the current player. But, again, while I think it is similar in some aspects, I still feel HQ is better as it delves deeper into what the current player has done and shown as compared to examining what other players have done.

[usualsuspects]

I am going to have to do it without BaseballHQ's 2006 projections, which is disappointing.

I have HQ's final 2006 preseason projections file right here on my PC and would be happy to forward it to you if you'd like. I also have 2005 and 2004.

[Yardsmen GM]

You could also buy last years' Forecaster...

[Michael@HQ]

The Forecaster predictions are made in November. One needs to use a cud-off date closer to your other predictions if one wants to compare them.

[bankerboy]

One needs to use a cud-off date closer to your other predictions if one wants to compare them.

Wow, those are "chewy" projections. *groan*

[Michael@HQ]

Ouch!

[PD@HQ]

Right-to be fair you'd want the last projections before Opening Day from all contestants.

[jackvdo]

Thanks and Analysis Methodology

Just wanted to say Thanks to UsusalSuspects for last year's BBHQ forecasts and a thought about what I said before about the different ways of doing an analysis. I originally started analyzing ERA projections using Correlation, a statistical method which measures the correlation between two sets of numbers, which I believe most other analysis I have seen used. Doing it this way, Baseball Information Systems projections did not fare well. I decided to do it another way, using Average Deviation, which measures the difference between the projected and actual peformance of each individual player, which in my mind is actually what we want to know, and BIS numbers came out very well, because they seemed to do very well at predicting those players that had a huge swing in their numbers. I just thought this was interesting and food for thought.