HQ customer service

[WMR]

I never said the content wasn't worthwhile, I'm just pointing out some flaws in it. Constructive criticism should be welcome for any business. Certainly, if you were making women's shoes wouldn't you like to be told that they were causing blisters?

The biggest flaw, since you asked, is the premise that once a player displays a skill he owns it. This, I disagree with. Much like a golfer can come out of nowhere and win the Masters and never win a major again, a baseball player can come out of nowhere and have a great year and never again reach those numbers. And in that great year his BPI's will naturally rise, and if you assume that he has now mastered something new may not neccessarily be the case. Take Luis Gonzalez, for example. In '01 his PX levels were 202, and he had never before even came close to that figure and hasn't since. To project him at those same levels for the following seasons would have been foolish. And they're doing this very thing with David Ortiz. Could he do it again in '04? It's possible, but the chances of him slipping back to his norms are much greater than him duplicating his success. My informal research shows me that hitters who achieve a $5 increase in value over their norms one season have a 67% chance of losing that $5 or more in value the following season.

Conversely, you have Abreu, who's PX stumbled to 110 last season from norms that were closer to 150. HQ jumps on this and assumes he will not return to prior power norms. I disagree with this as well. I believe that when a player slips in any category for one season it's more likely to be a normal fluctuation in a player's career than the beginning of a trend. Maybe these signs are more important when a player ages, but for most players that are close to the prime of their careers I believe this is the case. I learned this at a young age looking at Reggie Jackson's baseball card. His numbers would fluctuate madly from season to season and, if there were such a thing back then, so would his BPI's. I found in my research that a player who loses $5 in value under his norm one season has a 72% chance of gaining the $5 or more back the following season.

In short, I think HQ punishes too harshly for a bad season and awards too lavishly for a good one.

Then there's other cases where they completely ignore trends that are shown in their own numbers. Take Garciaparra. This guys has been slipping in practically every number you can imagine since '99 -- BPI's or traditional stats -- but yet they have him having his best season since '99. It's like they punish more for a bad season than they do for a slow slide. Me, I have Garciaparra slipping into the .280's not only because of this slide (which is obvious not just with the BPI's, but also practically every other traditional indicator) but the fact that by the end of last season he had become a comletely different hitter than he used to be. No more ropes to all reaches of the park but a pull hitter who is too influenced by the Green Monster. And don't even get me going on Sosa.

I could go on further but this is getting long.

I do not think we can compare my projections from prior seasons with HQ's because I never had it on the Excel until these '04 ones. Up until now they've been so scratched and scribbled upon that I cannot decipher most of it. I am more than willing to send my '04 projections to anyone who I feel will be objective in comparing mine to HQ's at the end of the season. I only ask that it be kept sealed from the public until the final results are in because I don't want any, "That idiot had such and such with these numbers." Many of my projections are unorthodox and cannot be judged until the season is over.

And if anyone does accept this I suggest the criteria set in the HQ, which is whose is closer, HQ's or everyone else? In this case, it would be between me and HQ. As Ron will attest, all projections are wrong, it's just a matter of whose is more wrong.

[molokai]

HQ customer service

I've found this thread entertaining and informative. The originator did get some nice repsonses from HQ folk and he responded in kind. Just some of my own two cents.
1. I don't think RON can use HQ subcriber success to hang his hat on. I imagine I'm like most HQ subcribers, been playing for a number of years, play multiple leagues, and have always been successful in the sport long before HQ showed up. To really find out how HQ works you would need to find subcribers who never finished in the money and then after using HQ information started finishing in the money. I also expect that most HQ subcribers who are successful do not only use HQ for information. I've been using Prospectus/HQ together for a number of years and I found that you need the balance of one against another.
2. That said HQ has taught me many things since I started using the service and has greatly enhanced my appreciation of statistical analysis.
3. Derek is the main reason I still subscribe. Many people write about minor league players, I read them all. I don't think anyone does a better job of predicting what they can do at the ML level then Derek. And I don't read them all because of fanalytic baseball. I'm just a fan who decided to play fanalytic baseball because I had all this knowledge and this gave me the abiltiy to leverage that information into something that I already loved doing.
4. The Mack Engine is an awesome tool, and would be even better if the forecasts were better.
5. If your going to sell something called the Forecaster it does seem you should not so easily dismiss the results. When I hear that HQ isn't interested in checking out their numbers one of two things come to mind. They did check them out and didn't like the results and found it was better not to mention it, or to they actually don't do year end comparsions between their forecaster product and reality. Which I find amazing since to me that is the product they are selling.
6. On the other hand the subscribers here seem to love everything about HQ and always rally behind them so Ron is putting out a product that most of his subscribers love and that is the bottom line, make your customers happy.

[Rob]

WMR,

We can all cite the most egregious examples. I call them "the alien invasion," the best example being Brady Anderson's 50HR season that came out of nowhere. What I think you're talking about in terms of Ron's projections is regression to the mean, i.e., how much he regresses his projections to that player's historical performance (after adjusting for age, ballpark, etc). Every ballplayer's season is merely a sample of his "true" ability (whatever that is). We can't really get a handle on his true ability, at least not until his career is over. We can only estimate his true ability using established statistical methods.

How much to regress is open to question, but if you're interested in the subject, I suggest you check out the BaseballPrimer site and look for "Primate Studies," in particular articles written by Tangotiger and Mitchel Litchman (MGL). MGL has done a great deal of analysis on appropriate regression levels that you should find of interest.

I do not think we can compare my projections from prior seasons with HQ's because I never had it on the Excel until these '04 ones. Up until now they've been so scratched and scribbled upon that I cannot decipher most of it.

You did all your projections longhand? How many players do you project each year? They're not saved in ANY electronic format that can be easily shared? I find that amazing, but I'll take your word for it.

I am more than willing to send my '04 projections to anyone who I feel will be objective in comparing mine to HQ's at the end of the season. I only ask that it be kept sealed from the public until the final results are in because I don't want any, "That idiot had such and such with these numbers." Many of my projections are unorthodox and cannot be judged until the season is over.

Why do your projections need to be kept secret? I don't think anyone here is going make fun of your projections. Frankly, any of your more outrageous projections would likely be the most valuable to the fanalytic player. I think you should encourage feedback, not hide out of fear of criticism. After all, you have been the one who has been casting aspersions about other people's work. And to hide your own work, after touting it is, well, kinda weenie. Honestly, I think opening up your projections and methodology to peer review could be helpful both to you and this forum.

Unless you are trying to sell your projections, I see no reason not to show us your superior methodology.

Respectfully,
Rob

[WMR]

Rob,

Well, I'm a computer idiot. The only reason why I did it on Excel this year is because my wife made me. She said I was wasting too much time so she set it up for me. She was right, of course.

Well, I suppose you are right. Even though I have visions of setting up shop someday, that day probably won't be soon so I see no harm in what you suggest.

I also agree with your assessment about the "regression to the mean". It is impossible to really know just what that mean is until, as you suggest, after a player's career is over. I do believe, though, that you must grasp the concept if you are to get closer to a correct projection. You will be much more accurate if you assume all players who outperform or underperform their norms will bounce back to their norms the following season than assuming they won't. From there you can look deeper for the exceptions, which I have also in my projections. That is just one theory I have anyway.

[Rob]

Regressing to the mean

I also agree with your assessment about the "regression to the mean". It is impossible to really know just what that mean is until, as you suggest, after a player's career is over. I do believe, though, that you must grasp the concept if you are to get closer to a correct projection. You will be much more accurate if you assume all players who outperform or underperform their norms will bounce back to their norms the following season than assuming they won't. From there you can look deeper for the exceptions, which I have also in my projections. That is just one theory I have anyway.

What I "mean" is to regress that player TOWARD his historical average and/or the league average. There are also quick/dirty methods like 5x2003+3x2002+1x2001+3*lgAVG or something like that. Search on Baseball Primer for "Marcel the Monkey" and/or "Projections."

Note to moderator: you might want to move this thread over to ThinkTank

[Play2Win]

WMR brings up something that Moneyball also discusses. Much like the traditional scouts look at body type, swing, mechanics, etc. at the expense of stats like OBP and walks, fantasy players might focus too much on stats (it is a numbers game) and not focus enough on the skills and abilities that generate those stats. For instance, a few years ago many people predicted that Sean Casey would breakthrough the 30 homer barrier due to his rising homer totals and his age. After reading some traditional scouting reports, I realized his swing is inside out meaning he's not going generate a ton of power unless he gets a perfect pitch. Same with Matt Clement. He needed to gain consistency with his mechanics before he could turn the corner. He got a consistent arm slot and his numbers turned around. Could these have been predicted with BPIs or traditional stats? Maybe, maybe not. I picked up the information from scouting reports, not by looking at numbers.

Much like real baseball, looks can be deceiving. Finding out that the guy striking out 10 guys per 9 IP in AA can't break glass from 20 feet with his fastball is just as important as knowing his minor league numbers. If Sosa's bat has slowed down or Beltre keeps getting hurt because he's too bulky, those are important pieces of information that can't be dismissed.

[Beane Pot]

The biggest flaw, since you asked, is the premise that once a player displays a skill he owns it.

When a player demonstrates the skill, what are you to do with it then? Balance it with his weakness, stir it with his skill set (eye, ct % PX, G/F ratio, playing time, ballpark adjustments, etc.) and make your best forecast then.

A pre-season baseball projection is just an isolated snapshot in time. Once the first batter steps to the plate on Opening Day, that projection becomes obsolete. It's value to fantasy leaguers is merely to provide a starting point, a baseline for what is to occur. Foundation Philosophies

Forecasting is not an exact science. Foundation Philosophies

Baseball performance forecasting is inherently a high-risk exercise with a very modest accuracy rate. Foundation Philosophies

Simply put, Ron and his staff are consultants. You, as the GM of your franchise must make the decision, on your own, to pay heed or ignore their advice on rosters, players skills, organizational minor league rankings, possible trade options, budgeting, freezing players, FAAB and so on.

HQ might not give me the exact street address of where I can pick up my trophy, but they'll give me the zip code on where to start driving to.

[usualsuspects]

Originally posted by Play2Win
WMR brings up something that Moneyball also discusses. Much like the traditional scouts look at body type, swing, mechanics, etc. at the expense of stats like OBP and walks, fantasy players might focus too much on stats (it is a numbers game) and not focus enough on the skills and abilities that generate those stats. For instance, a few years ago many people predicted that Sean Casey would breakthrough the 30 homer barrier due to his rising homer totals and his age. After reading some traditional scouting reports, I realized his swing is inside out meaning he's not going generate a ton of power unless he gets a perfect pitch. Same with Matt Clement. He needed to gain consistency with his mechanics before he could turn the corner. He got a consistent arm slot and his numbers turned around. Could these have been predicted with BPIs or traditional stats? Maybe, maybe not. I picked up the information from scouting reports, not by looking at numbers.

Play2win, I completely agree with you here. But, where in the world do you find this kind of scouting information? The Scouting Notebook (?) used to have it, then STATS bought it out and now it is mostly analytically driven. A couple years ago Lindy's Fantasy annual featured really good scouting-type info, but they ditched that the following season in favor of a number crunching approach.

It has come 180 degrees since I started with this hobby 20 years ago. Then, Bill James was little known, stats and statistical analyses were hard to come by, but there were several publications which would tout, oh, Dwight Gooden's killer curveball. Now, everyone with a spreadsheet riffs on the same numbers in their books and sites, but information like "Casey's inside-out swing is unlikely to generate high HR numbers" is next to impossible to find.

[to93elias]

I've been reading Ron since 80's and I have "disagreed" with many of his projections . The latest being Sabathia's last yr , when I posted my objections to a over 5era and 1.5+ whip , when he never was over 4.39 and 1.31.Well I extended him last yr to keep this yr also-------but that doesn't mean that the site,all it's resources/info are not tremendously valuable. Let's face it, it's all put together, organized and there for YOUR interpretation--TOOLS man, TOOLS

[Play2Win]

Lindy's and the Major League Scouting Notebook are still pretty good sources. Having about 10 years of the minor league scouting notebook helps too.

[roto rooter]

I rarely look under HQ Help -- I'm glad I stumbled accross this interesting thread. As noted above by Tom, the last time I was here, I asked why there were no projections for several players (Leiber included) and was properly scolded

I've enjoyed the Forecaster for about 10 years and the past two years on-line at BBHQ. The value I personally find in it is:

1. Deric's minor league work.
2. Staying current on players and getting opinions in the Forum (with HQ staff and regular intelligent posters like Tom, Old Tom, Ron V, Nick, Carlo, Beane Pot, Oechsli, etc.).
3. In-season roster management -- trades, FAABing, who's hot for real and who will likely flame out, etc.
4. As a baseball fan even more than a roto fan, it's enjoyable to share non-roto baseball opinions in the Clubhouse forum (Pete Rose, my hated Yankees, baseball economics, Don Zimmer's head, etc.).

I have found my league's draft to be so unpredictable every year that I don't go as crazy as most trying to forecast numbers. I mostly try to identify guys to avoid and a group of players to target. So the "forecasting" part is fun to read, but not the key reason I like the service.

WMR might have set the wrong tone with his message, but it was a valid point to raise and the ensuing discussion was very worthwhile.

[WMR]

Thanks, Roto. You know me, I'm just trying to spread sunshine wherever I go.

[Michael@HQ]

1) Baseballprospectus.com had an article over the weekend (it's in the premium site, not the free content) that concludes that BaseballHQ was about average of the 7 sets of 2003 projections they analyzed. Given that that the Baseball Prospectus forecasts were considered #1, you might want to take that with a grain of salt.

2) I'm not sure that average correlations is the best method for evaluating forecasts frankly. BaseballHQ intentionally tries to have very gutsy forecasts, to take a stand on a player. If they project a player to earn $25, other sources I consult say $15 - $18, and I get the guy for $15, then I'm happy if the guy earns $21 even if someone else's projection technically might be better. If I could ever get enough data at the beginning of the season to do it, I'd like to evaluate the results that way.

3) Even if BaseballHQ isn't the best, I'd still subscribe. I've been a customer since they assumed Rick Wilton's Hot Sheet. The stuff they write is entertaining and thought-provoking. I don't use any one company's projections but pick and choose what I believe anyway. Looking back, I sometimes followed their advice generally with good results (trading for Jay Gibbons before 2003 for example) and sometimes didn't. They should evaluate the forecasts in my opinion (and I'll volunteer to do it if they want) but most of us would subscribe regardless of the results.

[roto rooter]

Originally posted by Michael
Even if BaseballHQ isn't the best, I'd still subscribe. The stuff they write is entertaining and thought-provoking. They should evaluate the forecasts in my opinion but most of us would subscribe regardless of the results.

Good job. That's about as fair a summary as I could put on it.

[usualsuspects]

Originally posted by Michael
1) Baseballprospectus.com had an article over the weekend (it's in the premium site, not the free content) that concludes that BaseballHQ was about average of the 7 sets of 2003 projections they analyzed. Given that that the Baseball Prospectus forecasts were considered #1, you might want to take that with a grain of salt.

The Prospectus article compared hitters based on OPS and pitchers based on ERA. OPS not a very useful metric for most fantasy players, and the Prospectus doen't forecast RBI's. The answers might be pretty different if all the roto categories were considered.

The Forecaster has a blend of predictions that are right on, mediocre, and spectacularly bad. But the other guys are right. You'd think some post-hoc analysis would be warranted just for quality control and improvement. But really what is helpful is if the HQ numbers are different enough from the pack to help you draft the breakouts and avoid the busts!