HQ customer service

[WMR]

Whether you believe the results of that contest or not, you'd still expect a little better result from HQ than just beating the freebie and cheap-o sites and being nosed out by a novice. Is it too much to ask a better showing from the site that claims to have "the most accurate performance projections in the industry" (quote taken from the back of the Forecaster)? I probably shouldn't be too hard on them since the Forecaster is the cause for many major league teams to improve on average 8 games last season and finish no lower than third place.

[roto rooter]

WMR: Any comments on Ron's article today on the front page?

[WMR]

Nope.

[DougD@HQ]

jumping in late

WMR says that he likes to "question things that others tend to accept without question"--man, to me, that is the very best thing we can hope for on these forum boards, because that generates the useful conversation, rather than a 100% love-fest of in-breeded and rigid thinking.

*If* you think that we (HQ writers) all agree on projections on the site or in the Forecaster--we don't. But we are free to give our two cents on the disagreements in our articles and on the forum (and often do). I try to give the courtesy of telling the HQ folks that I disagree with ahead of time. Am I always right? Heck no. For instance, I thought MacDougal would not get the closer job in KC because his skills were so poor, and I said sell sell sell until his skills caught up with him. Someone who ignored me and bought on risk, then sold after April made a killing. It is in the buying and selling and relative expectations that you really make hay with this site--because you get a sounding board, you get perpetual commentary, you get to ask questions to the crowd, you get articles about the changing landscape. What value are static projections? Some value, sure, but you have to see it as an ever changing dynamic process and be ready to change your roster when risk variables change--and the ability to do that is where HQ helps (and differentiates itself) most.

Nate Silver has a very nice article and product in PECOTA over on BP. I subscribe to BP and like it very much. But--I'm not so sure that PECOTA is above anyone else. Look at the differences he creates through his accounting method--we are talking a margin in the ten-thousandths place for two categories he chose (ERA and OPS) and then used a ranking method to create separation where it doesn't exist. The results for 5 of the 7 prognosticators he uses is really something of a virtual tie, and HQ is in that group.

Lastly, I'm with you on the defensive posts that a lot of subscribers and some writers put on forums. To my mind, it is tone-related, not content-related problem. Disagreement isn't an issue, but spleen venting offends some, and then their response generates a response than very quickly leaves the content-related issue far behind.

[RJM713]

Just My Two Cents

I've been an HQ subscriber since '98, and have posted wins in the League I commissioner in three of those six seasons to gom along with two second place and a third place finish - a .500 average, which is plenty OK by me. Prior to that I had won four times in the first eleven years, so my "luck" improved once I had HQ to bring with me to Draft Day and bank on during the season. Coincidence? I don't think so. I was pretty good before I came across HQ, but like I told Ron, now I feel as if I always have an edge over my fellow owners before sitting down at the table, and that's a great feeling.

Without sounding like a "lemming" or a paid advertisement on this matter (no one told me to write this,) I have to say that for the money, Ron and his staff provide a Grade A product, and to the best of my knowledge have never failed to reply to any communication I have ever sent, whether it be a roster analysis (which, granted is a paid-for option,) a question posted in one of the Subscriber Forums, or a simple e-Mail directly to Mr. Shandler himself.

Might I suggest that you make an effort to attend one of HQ's "First Pitch Forums" and get a chance to meet Ron and some of the experts he has as accompaniment? You'll see that he does not enter the room on a golden throne, nor is he bowed-down to by those in his presence - he's a regular guy just like you and me who loves rotiss and has become a success at what he does - analyzing and projecting, but not predicting what might occur based on data he has at his disposal. I attended a session in Long Island and met Ron for the first time - he's a nice man who I think you will come to like as well, plus it was a couple of enjoyable hours discussing what we all love to boot and worth every cent.

Give it a shot. Best regards,


Rich M.

[WMR]

Some final thoughts.

First to RJM713, forgive me for quoting myself, but here are a few lines that I wrote earlier in this thread.

"I met Ron and he's a good man. Smart. I can't say enough good things about him."

"HQ's an invaluable treasure of information, that is why I'm here."


This has been a good discussion, but there is one thing that still bothers me. This thread started because I had not gotten any answers to a thread I had started on December 2nd. Here is that post.

"HQ report card.
I remember reading that there was going to be an article that compared the projections on HQ to other services. Was that article posted, and is it still available?

Also, I know it is tough to qualify projections as far as which are good and which are not, but maybe an HQ staffer could answer this. Would you say your 2003 projections were better, worse, or the same as previous years? I would also like to hear answers to this question from long-time users.

lastly, I know the book starts shipping on the 8th, and I am looking forward to my copy, but will the projections be posted online at that time as well? I looked for that info on the site today but could not find it.

By the way, I have the gold package, and have every intention of continuing year to year. The content on this site is well worth the money to me, so these questions were not asked to help me make up my mind about re-joining.

Thanks."

Since last year was my first on HQ, I had no point of reference to judge if 2003 was a good year for HQ, a bad year, or an average year. That was the point of the question. I was hoping for some HQ response and some response from other subscribers. Ron did not respond to that original post, but when he chimed in on this thread he explained why he had passed on responding to my first post. Here is what he wrote.

"As for not responding to your post, let me explain my philosophy on that, acquired over 18 years in this industry... Tone is everything. If someone legitimately wants to have a dialogue, you can determine that by the tone of his post. However, the problem with attempting to create a dialogue with an abrasive post is that, in most of the cases, it ends up becoming a pissing match, which needless to say, I have no interest in participating in."


Now, I am a reasonable man, most times, and I think there are a lot of reasonable people in here as well. So, let me ask you, was my first post written in a bad tone? Was it abrasive?

Here are a couple other lines from Ron in that post.

"Several of my other writers asked if I was going to respond to this thread and I told them that I was going to wait to see whether I needed to (allocation of scarce resources)"

"These days, I can choose to spend my time writing responses like this, or spend it working on the new player database engine, already three weeks behind schedule. I guess I've made my choice for this afternoon."

Do you want to talk about tone? Well, if the new player database engine is late, you guys can balme me because Ron had to waste some of this valuable time on me.

I remember reading a story Ron once wrote. I don't remember all the details, but the jist of it was that he had a new book out, I think his first, and he was at a show and who walks up but Bill James, whom Ron revered. Ron was excited when James picked up his book and began to read because James was an idol of Ron's, and maybe they could have some discussion about it. But what does James do? He pages through the book, points out a minor statistical error, closes the book and walks away.

What is that old saying, "and then the student became the teacher"?

[hqforums]

Re: HQ customer service

Not to make any more of an issue about this than it already is, but the "abrasive" post that I was referring to was the below, NOT what you reference:

RON@HQ

Originally posted by WMR
Here's the typical response anytime anyone dares ask a question that could be construed as critical to anything HQ:

1) A collective "gasp" from the subscribers.

2) A verbal whipping by Ron or one of his cronies.

3) An immediate apology from the questioner.

What kind of customer service is this? No other company I know of can stay in business for long when their response to customer's problems is, "You're lucky you're even allowed to use this product."

A month or so ago I asked the question, "How did HQ's '03 projections do compared to prior seasons?" A simple question I think, but no one had the guts to answer it. It wasn't to be critical either, just an honest question from a person who just started using HQ in the past year and was wondering if this was a normal season for HQ.

Because of the lack of responses, I started digging deeper in the '03 projections. I took the ones that were pulled right before the '03 season started and used as many objective test I could think of to find out how well the projections were. My conclusions? The projections did, at best, no better than random chance.

Now, I am sure HQ or one of the other lemmings will respond with, "Well, it's not the outcome that is important, it's the process." What? The only other place I've heard this type of nonsense is when liberals are explaining their policies. The outcome is how you decide whether the process is correct. You know the saying, "Garbage in. Garbage out?" What would you say if you bought a cookie from a bakery and spitted it out because it was so bad but the baker says, "We followed the recipe?" Unless Ron's the baker, you say, "Well, you got a bad recipe there, buddy."

Is this frustration talking? You bet. Didn't someone last year question the accuracy of xba and get ripped to shreds by Ron or am I mistaken? But low and behold, this year's Forecaster comes out and admits there is a flaw, but where is the apology? I just find this stuff to be very rude and stiffling of ideas that could make this site better. And all this abuse heaped upon the "heretics" could instead be changed into explanations that could appease the doubters. This would be more helpful if you ask me.

I don't care if I piss off Ron. Do I care if I piss off Compaq if I call them with a problem? Only on HQ are you supposed to worry about what the management thinks and not the other way around.

Alright, Ron, blast away.

[LongDuckDong]

I think its unfortunate that this extremely valuable thread got sent way off track by the confrontational tone of the original post and a discussion of customer service, which has nothing to do with this discussion.

The original post in essence challenged the HQ team to support their claim that using BPI's and the other analytical tools available allows one to more accurately forecast future performance. The way he wished for the HQ team to do this was to analyze the performance of the 2003 projection set and compare it to other services who did the same thing and see if indeed the HQ method allows one to forecast more accurately. In and of itself, this does not seem to be an unreasonable request.

On several occassions both in the book and in the website, Ron has made the point that it is very difficult to define a "good" projection. It is my assumption that this is the main reason why the HQ team chooses not to spend a great deal of time analyzing their old projections.

However, I take significant objection with Tom@HQ's point that "the forecasts are not the product". (I apologize if thats a paraphrase, but thats how i remember it being said)

If Ron and co. can not demonstrate that the BPI's and analytics allow one to project more accurately, then the BPI's lose most of their value.

What Tom likely meant to say was something to the effect of "Player X hit .278 this year, but the underlying data indicates that he should have hit .315, therefore it is reasonably likely that his BA will rise. It might be to .290 or .350, but the idea is that this guy is better than he showed last year." The idea being that for that player, a projection might not be accurate if Ron projects .290 and he hits .350, but the analysis was accurate.

But I think it is extremely inaccurate for the HQ staff or users to somehow imply that the projections don't matter because its all about the tools being provided. By definition, the tools drive the projections, the projections drive my draft day/inseason decisions, those decisions drive my finish in my league. I do not get 12 points for the best K/BB ratio in my league or the best DOM, I get 12 points for the best ERA/Ratio/etc...

I am extremely grateful to Ron and co. for enhancing my ability to evaluate statistical performance - there is NO ONE else out there who does this the way he does. I show him this thankfulness by plunking down my money each year for his product.

In my ideal world, each player would have the following capsule done every year: 2003 Projection, 2003 Actual, then a "what happened" summary, in a bit more depth than whats included in the forecaster but along those lines. Then a 2004 Projection with explanation.

I realize likely time doesn't allow for that type of player capsule but the idea would be that we could get a feel for where the statistical analysis pointed out things that occurred and where things didn't occur.

I'll stop typing now but I just wanted to chip in and give my two cents because I think its an EXTREMELY valuable conversation and its getting weighed down by the tone of some of the posts.

[WMR]

Alright, alright, my tone of me second post was very harsh, but it seemed like the only way to get HQ's attention. I tried to be nice with my first post and it was totally ignored, and judging by the length of this thread I got my desired reaction with my second. Plus, originally I was more irritated with the lack of responses, so customer service was my primary gripe.

With that out of the way, LongDuckDong articulated my point about HQ's method better than I ever could (with a lot less rancor to boot). Well said, Dong.

[kefischer]

I find this discussion fascinating. WMR raises some valid points, I think, not the least of which is his original question: How well do HQ's forcasts hold up to scrutiny? I, for one, was drawn to this site as a new subscriber precisely for the forcasting it provides. I find it discomforting to now hear that "forecasting" is not HQ's bread and butter. Why then did i just shell out $90???

Anyway, for 10 years I've covered education for some of the nation's largest newspapers, and I must say, I find this debate remarkably simialr to those I hear in education reform circles: How do you measure success? Seems to me that looking at end results (whether it be student test scores or forecast accuracy) is extremely important. And I think WMR is exactly right on that point: If you don't measure success, how can you improve? Indeed, how will you even know that you have failed? The position taken by HQ seems awfully like that I hear from educators who chafe under new government testing requirements that are intended to gage how well schools perform. Teachers say: We know that our schools are good because our process are good -- reliable teaching techniques, good curriculum. Yet many of those schools never seem to notice that they had 50 percent drop out rates and were graduating illiterates. Had they bothered to measure their success, they would have seen that they were failing thousands of kids. Could not the same be said of this discussion? How can any successful organization rely on process and ignore results??

And as for Ron's assessment that HQ is accurate because of its subscribers' track record of fantasy success, i offer up this point; I'm a new subscriber (less than two weeks, in fact) and i've won my fantasy league 3 out of the last 4 years. I am bringing MY success to HQ; HQ is not bringing its success to ME (at least not yet, anyway.)

Great discussion. And i love the site.

[WMR]

Boy, I'm starting to feel like the kid who pointed out that the emperor was parading around without any clothes. Was this something that everyone noticed but was afraid to mention?

[PD@HQ]

Originally posted by kefischer
I find it discomforting to now hear that "forecasting" is not HQ's bread and butter. Why then did i just shell out $90?

At the risk of pre-empting Ron, who can defend himself, here is my take:
You are paying for the projections, but also for access to all the other discussions, essays, articles, forums and tools on the BHQ site. These are supposed to empower you to get better at developing and executing winning strategies and tactics, including how to cope with the inevitable truth that forecasting is a vastly inexact art.

...I find this debate remarkably simialr to those I hear in education reform circles: How do you measure success? Seems to me that looking at end results (whether it be student test scores or forecast accuracy) is extremely important.

With all respect, I disagree because student test scores are easily measured while ballplayer projections are not.
Students can be tested on fundamental ability against fixed standards. You ask a kid 100 math questions, all of which have known answers, and count his right and wrong answers. You tell a kid to read a passage and then answer some factual questions about it.
(And yes, I know about the "subjectivity" of grading essay questions, but most established testing systems have training and other mechanisms to reduce it.)
But, as Ron has suggested repeatedly here on the site and elsewhere, how do you "grade" projections? What constitutes a "right" answer?
The BHQ projection for Jim Thome last year was
287-47-115-1/$32 and his actual was
266-47-131-0/$30.
At a glance, a pretty good projection. An 8% error on the high side for BA, 14% error on the low side in RBI, 0% error on HR, 6% on R$. Pass or fail?
Scott Rolen: projected was:
.285-30-103-10/$28, actual was
.286-28-104-13/$28. A little high on HR, a little low on SB. Pass or fail? This is closer than Thome's but if we now call this one "good," is Thome's now "bad"?
It is possible to assess the validity of the entire group. A simple correlation of the batter projected R$ values with their actual values gives a 0.74, which is pretty good (a 0.0 means no correlation and 1.0 means absolute correlation). The stat categories scored 0.44 for BA, 0.78 for HR and RBI, 0.74 for SB.
The median R$ error was +$1, meaning the actual production was +$1 to the projection. Again, pretty good.
For pitchers, the score was 0.57, and the median error was $0. Again pretty good.
These R$ results are based on league-specific 4x4 scoring, and projected players with 0 actual AB or 0 actual IP were given $0 results (eliminating them didn't materially affect the outcomes).
These kinds of measured "results" seem pretty impressive, but I submit they are really not meaningful. The statistical measure of the entirety doesn't add any confidence on any individual case, like when you draft Jermaine Dye, Pat Burrell or Vladimir Guerrero and lose big, or draft Scott Podsednik, Mike Young, or Angel Berroa and win big.
Ultimately, we'll try to help you become your own best guide to valuation (which also depends on your leaguemates, by the way) so you can keep winning your league.
Just my two cents' worth. I could be wrong.

[roto rooter]

I think a few sub-threads have taken life in this discussion.

1. Customer service/BBHQ response.
Not sure why WMR's original post went unanswered. Perhaps it was just the dog days of winter. As Ron indicated, a high renewal rate is a pretty good sign of overall satisfaction so I'd take that as an isolated incident (which unfortunately prompted WMR's beligerant tone in his second post).

2. Thread "tone" and defensive posting (I think you get a break on your car insurance for that).
If you don't want to hear different opinions, why post in the first place? As far as tone, this forum is much better than most but will probably never be perfect. I think 95% of the posters are reasonably polite and I've enjoyed some of the jabs going back and forth.

3. Value of the overall BBHQ service.
It's pretty clear that a subscriber gets a lot more here than just what Jim Thome is expected to do in 2004. The minor leagues, drafting strategies, FAABing, etc. all play a big role in winning your league. I don't think anyone has taken issue with the value of all of the other services.

4. How well does BBHQ forecast?
Davitt raised great points on how hard it is to judge what a good forecast is -- what margin or error is good or bad? He did indicate that a simple comparison of R$ for hitters yielded a 0.74 correlation and 0.57 for pitchers. Is there a better measure of success? I'm sure Todd Zola could up with a plan to make our heads spin

[WMR]

I was hoping this thing was going to die weeks ago but since this thing has legs of its own I'll chime in one more time.

This whole issue could have been over early on if HQ's customer service was worth a damn (my original point). This is what should have been said:

"We here at HQ are always trying to improve our products. In one area is found lacking we shall work until it is improved to our subscribers' satisfaction. We may not always agree with our subscribers' complaints, but we do take them seriously."

But what do we get? A litany of excuses. I'll tell you too, they're pathetic. My favorite was the "no way to judge the accuracy of projections." Come on, if they can find a way to judge figure skating, I think there's somehow an objective way to judge forecasting. Besides, didn't you use that little competition between other sites from last season (with players such as Contraras that you proudly did a review on just a month ago) as proof of your prowess? The grading scale was simple -- whoever was closest wins. After all, isn't that the purpose of projections?

Ron, you got to understand that we are not dummies. Most of us are veterans who've been playing this game for many, many years and we don't get fooled by this type of stuff. On the back of the Forecaster you claim to have the best projections, so I think it's within our right as paying customers to ask you to prove it.

[Todd Zola]

I'm sure Todd Zola could up with a plan to make our heads spin

Sorry, not yet anyway.

Projection theory is an area that I dabble in but is not really a concentration. It will be, that's for sure, but not just yet. I pick up what I can from people like Ron and the crew here at BBHQ, Peter Kreutzer, the gang at BP and some other discussion groups (Baseball Primer in particular). Once it becomes my chief focus, I hope to be able to season it with my own ideas, but until then, I defer to those more learned in the area.