Tweet
First, I would like to preface this inquiry by stating that I am a loyal HQ user and I think that HQ is far and away the best of its kind in the industry.
I typically also subscribe to another baseball service as I like to get another perspective, especially in the area of draft prep when it comes to identifying sleepers and prospects. I will simply refer to this alternate service is Brand X, but suffice to say that they are well regarded (I think that their 2004 projections were ranked #1 by some standard) and they do seem to be "quantitatively" oriented.
I understand that there are some "subtle" differences in valuation methodologies that relate to allocating $$ value to scarce categories such as SB's or Saves. However, I always thought that the impact of these factors should not be overly material. However, I was shocked at the SIGNIFICANT differences between HQ and Brand X's valuation and resultant rankings for players with which the underlying HQ and Brand X projections were virtually identical. Furthermore, the differances seem to transcend the "SB impact" (whatever that impact is??) and also creates BIG differences in players who are not SB threats. I will list a few examples here to illustrate the differences.
League Parameters:
12 Team Mixed League, 5x5, 13 hitters (standard positions include 5 OFers, CI, MI, 2 Utility). $260 benchmark budget, I believe that I inputted a hitting/pitching split of ~ 70/30. This league is a draft, so the relative rankings are what is most important -- although I have auction leagues to follow as well.
OUTFIELD (Runs-HR-RBI-AVG-SB; # denotes ranking within position)
Guerrero (#1 for both, but provided for a baseline)
HQ - #1 ($50) 108-38-114-328-13
X - #1 ($37) 119-37-121-330-14
Carl Crawford
HQ - #2 ($50) 102-14-62-292-59
X - #27 ($25) 104-16-57-289-61
Damon
HQ - #11 ($23) 110-16-75-291-15
X - #14 ($28) 111-18-85-290-17
Above you have 3 OF-ers with extremely similar projections from HQ and Brand X. I am much less troubled by the fairly large differences in raw $$ values, although I'd like have an intuitive feel for the difference between $50 and $37 for Vlad since its more than a couple of bucks and will be an issue for auction leagues. However, I'm much more confused about someone like Crawford, whereby for a draft, HQ would say that he's clearly an early to mid-1st round value, while Brand X suggests that I don't touch him earlier than the late 3rd or 4th round, despite the fact that the underlying projections are virtually identical. I present Damon for reference to try to put the Crawford disparity in better context.
2B (even more confusing, since SB's are less of an issue)
Soriano
HQ - #1 ($26) 90-30-86-276-21
X - #1 ($31) 92-31-100-293-31
Jeff Kent
HQ #9 ($12) 86-24-97-271-6
X #2 ($24) 89-25-100-274-7
Mark Loretta
HQ - #4 ($15) 76-11-57-317-3
X - #8 ($17) 97-14-69-319-5
Now take Kent who has identical projections by both HQ and X, yet for a draft strategy that employs position scarcity, Brand X would suggest that Kent should go several rounds earlier than HQ (or twice the price in auction), with SB's not likely being an explanation at all. Loretta is provided for context, but also shows that their a very different "$$-value decline rate" (for lack of a better term) between the HQ and X valuation methodologies -- HQ's in some cases start 25-30% higher (in the case of Vlad), yet decline at a much steeper slope (as best evidenced by Kent).
Now I know that HQ can't vouch for someone else's "model". However, with no "transparency" provided with regards to the underlying algorithms, I am extremely confused as to understand the significant disparities. I'm pretty sure that I have entered all relevant "setup" assumptions correctly. I would think that the resident "quant jocks" who created the custom valuator understand enough of the alternate theories to look at the above facts and either offer some possible explanation or conclude simply that one of the models is not working properly. I hate "black boxes" to begin and certainly would not raise issues with a few dollars of discrepency, but when it throws completely out of whack the "relative" values and rankings, I need to throw out a screaming call for HELP!!!!!
Thanks for taking the time to review this long post.
I typically also subscribe to another baseball service as I like to get another perspective, especially in the area of draft prep when it comes to identifying sleepers and prospects. I will simply refer to this alternate service is Brand X, but suffice to say that they are well regarded (I think that their 2004 projections were ranked #1 by some standard) and they do seem to be "quantitatively" oriented.
I understand that there are some "subtle" differences in valuation methodologies that relate to allocating $$ value to scarce categories such as SB's or Saves. However, I always thought that the impact of these factors should not be overly material. However, I was shocked at the SIGNIFICANT differences between HQ and Brand X's valuation and resultant rankings for players with which the underlying HQ and Brand X projections were virtually identical. Furthermore, the differances seem to transcend the "SB impact" (whatever that impact is??) and also creates BIG differences in players who are not SB threats. I will list a few examples here to illustrate the differences.
League Parameters:
12 Team Mixed League, 5x5, 13 hitters (standard positions include 5 OFers, CI, MI, 2 Utility). $260 benchmark budget, I believe that I inputted a hitting/pitching split of ~ 70/30. This league is a draft, so the relative rankings are what is most important -- although I have auction leagues to follow as well.
OUTFIELD (Runs-HR-RBI-AVG-SB; # denotes ranking within position)
Guerrero (#1 for both, but provided for a baseline)
HQ - #1 ($50) 108-38-114-328-13
X - #1 ($37) 119-37-121-330-14
Carl Crawford
HQ - #2 ($50) 102-14-62-292-59
X - #27 ($25) 104-16-57-289-61
Damon
HQ - #11 ($23) 110-16-75-291-15
X - #14 ($28) 111-18-85-290-17
Above you have 3 OF-ers with extremely similar projections from HQ and Brand X. I am much less troubled by the fairly large differences in raw $$ values, although I'd like have an intuitive feel for the difference between $50 and $37 for Vlad since its more than a couple of bucks and will be an issue for auction leagues. However, I'm much more confused about someone like Crawford, whereby for a draft, HQ would say that he's clearly an early to mid-1st round value, while Brand X suggests that I don't touch him earlier than the late 3rd or 4th round, despite the fact that the underlying projections are virtually identical. I present Damon for reference to try to put the Crawford disparity in better context.
2B (even more confusing, since SB's are less of an issue)
Soriano
HQ - #1 ($26) 90-30-86-276-21
X - #1 ($31) 92-31-100-293-31
Jeff Kent
HQ #9 ($12) 86-24-97-271-6
X #2 ($24) 89-25-100-274-7
Mark Loretta
HQ - #4 ($15) 76-11-57-317-3
X - #8 ($17) 97-14-69-319-5
Now take Kent who has identical projections by both HQ and X, yet for a draft strategy that employs position scarcity, Brand X would suggest that Kent should go several rounds earlier than HQ (or twice the price in auction), with SB's not likely being an explanation at all. Loretta is provided for context, but also shows that their a very different "$$-value decline rate" (for lack of a better term) between the HQ and X valuation methodologies -- HQ's in some cases start 25-30% higher (in the case of Vlad), yet decline at a much steeper slope (as best evidenced by Kent).
Now I know that HQ can't vouch for someone else's "model". However, with no "transparency" provided with regards to the underlying algorithms, I am extremely confused as to understand the significant disparities. I'm pretty sure that I have entered all relevant "setup" assumptions correctly. I would think that the resident "quant jocks" who created the custom valuator understand enough of the alternate theories to look at the above facts and either offer some possible explanation or conclude simply that one of the models is not working properly. I hate "black boxes" to begin and certainly would not raise issues with a few dollars of discrepency, but when it throws completely out of whack the "relative" values and rankings, I need to throw out a screaming call for HELP!!!!!
Thanks for taking the time to review this long post.
Comment