Pages

2006-07 Fantasy Hoops: Projection Comparison

4/13/2007
Now that the season is winding down, it's time to pay the piper. If you play fantasy hoops, prior to your draft you probably utilized or looked at 'expert' rankings or player projections that are out there and readily available. You may have even paid for them. But how do you know if those projections are even worth the paper that their printed on? Since I'm all for the level of accountability expresed last year by my hero Matthew Berry, aka the Talented Mr. Roto, and since half of the fantasy-worthy players in the NBA have decided to take the last week of the season off, let's see how good these prognostications really are.

I did the same thing back in October 2006 to compare how well various projections did for the 2005-06 season. In that analysis, I looked at how well each projection did against a simple benchmark of just using the prior year's per game data. This is the starting point for any projection. Any time a projection deviates from the prior year numbers, you have to assume that the prognositicator has a good reason for moving that number. Because of this, we can measure how well a given projection does relative to the benchmark in each category...or put another way, how much information value does each projection provide on average.

The 2005-06 results showed that two projections...Rotowire.com (which you had to pay for) and CBS Sportsline actually performed worse than the prior year numbers. That's hard to do. Basketballmonster.com and NBA.com (provided by Talented Mr Roto) performed just slightly better than the previous year's data, and my initial projections were about 5% better than any of the competitors.

So let's see how well everybody did this year. For the projections, I looked at a dataset of 230 players that were common to each projection set. For each category, the error between the actual and projection was squared (hence, the term sum of squared errors!) and summed for the 230 player pool. Each projection was compared relative to the benchmark 2005-06 per game data...the benchmark was converted into an index equal to 100. A projection with a rating of 80 shows the projection provided a 20% improvement in that category relative to the benchmark. [Note: for 8 rookies the consensus view of 7 projections (my original plus the others) was used.]

Here are the candidates (if you know of others, send them in and I'll add them to the mix):

Jim - Original: My original projections dated October 24, 2006 and available for free
Jim - Preseason: Based on an analysis I did here, I adjusted my projections based on player-by-player preseason numbers
Jim - Global Preseason: In the same analysis, I made global adjustments based on overall preseason trends. Designed to account for changes caused by the much ballyhooed new game ball.
CBS Sportsline: Free projections available with their fantasy package.
Rotofreak.com: Free projections from a dude with a website
Fantasy Sports Central: Free projections
THE Talented Mr Roto: You have to pay $9.99 for a draft kit to get these projections.
Basketballmonster.com: You get the projections if you pay $12 for full-access to their site features (well worth it in my book).
Rotowire.Com: More pay-for-projections. I think these are $14.99. This is what you get if you sign-up for Yahoo's draft kit.

And here are the results:

_PROJECTION__MIN_ _FG%__FT%__3PM_ _PTS__REB__AST_ _STL__BLK__AVG_
JIM-PRE75.4 65.588.182.3 71.681.865.7 79.192.578.3
JIM-GLOB75.468.689.1 84.274.682.867.278.7 85.978.9
JIM-ORIG75.466.691.3 83.572.481.566.2 79.894.079.4
FREAK 72.095.1 101.478.189.968.5 78.3102.685.7
CENTRAL78.878.4103.0 79.482.283.574.6 127.098.890.9
MONSTER79.673.498.5 101.788.391.580.795.5102.5 91.5
ROTOWIRE77.676.0121.4 104.379.891.288.190.4108.4 95.0
TMR94.4 82.2142.393.5 89.598.777.699.9 109.299.1
2005 DATA100.0 100.0100.0100.0 100.0100.0100.0 100.0100.0100.0
CBS 71.6113.0 111.0106.798.5 99.1127.393.3 102.6

My projections did very well again, performing over 20% better than the baseline and a 7% improvement over the next closest competitor, rotofreak. The preseason numbers do provide some information value, similar to what we found last year. The global projections might have held up better if David Stern hadn't switch back to the old ball in the middle of the season.

Bringing up the rear again we have Talented Mr Roto (seriously, who's paying for these?) and CBS Sportsline who gets the unique distinction for performing worse than the prior year baseline two years in a row. Whoever is doing their projections should be fired.

Another way of looking at it is this...for each player the projection which comes the closest gets a score of '10' and the projection which does the worst gets a '1'. Summing up all 230 players, we get the following results:

JIM - PRESEASON: 1,526
JIM - ORIGINAL: 1,460
JIM - GLOBAL: 1,460
FANTASY SPORTS CENTRAL: 1,306
ROTOFREAK: 1,305
BASKETBALLMONSTER: 1,234
TALENTED MR ROTO: 1,166
ROTOWIRE: 1,107
CBS SPORTSLINE: 1,045
2005-06 BASELINE: 1,033

Hey, CBS inched ahead of the baseline! The lesson here is be careful what you pay for.

The Wegoblogger / Talented Mr Roto NBA Projection Challenge

Back in October, I also looked at 50 player projections where I differed the most from Talented Mr Roto and put together a little projection challenge. From the results above, you can probably guess how this is going to end up, but for closure's sake, let's post the results.

 WEGO HIGHER WEGO TMR  WINNER
 Bosh,Chris 1.15   1.49  WEGO
 Foye,Randy 1.20   2.67  WEGO
 Martin,Kevin 4.80   8.69  WEGO
 Richardson,Jason 3.94   2.03  TMR
 Ridnour,Luke 1.23   0.33  TMR
 Smith,Josh 1.74   3.71  WEGO
 Crawford,Jamal 2.28   4.18  WEGO
 Butler,Caron 1.92   3.65  WEGO
 Kirilenko,Andrei 8.38   4.20  TMR
 Nocioni,Andres 0.58   1.55  WEGO
 Anthony,Carmelo 0.84   1.80  WEGO
 Pachulia,Zaza 0.59   1.36  WEGO
 Cassell,Sam 1.92   1.34  TMR
 Gasol,Pau 2.79   3.64  WEGO
 Wilcox,Chris 0.71   0.47  TMR
 Bryant,Kobe 2.48   1.52  TMR
 Arenas,Gilbert 2.14   1.96  TMR
 Bell,Raja 1.02   1.78  WEGO
 Peterson,Morris 4.83   3.28  TMR
 Williams,Deron 3.84   4.74  WEGO
 TMR HIGHER WEGO TMR  WINNER
 Butler,Jackie 7.79   19.87  WEGO
 Brewer,Ronnie 6.22   14.04  WEGO
 Dixon,Juan 0.35   1.37  WEGO
 Swift,Stromile 1.90   5.13  WEGO
 Dampier,Erick 1.50   2.88  WEGO
 Thomas,Tyrus 1.90   7.03  WEGO
 McDyess,Antonio 0.43   1.28  WEGO
 Williams,Marvin 1.35   1.92  WEGO
 Gomes,Ryan 0.81   1.71  WEGO
 Stoudemire,Amare 3.24   2.34  TMR
 Stojakovic,Peja 2.67   4.25  WEGO
 Mobley,Cuttino 0.31   0.56  WEGO
 O'Neal,Shaquille 2.66   5.00  WEGO
 Jackson,Stephen 1.17   1.79  WEGO
 Maggette,Corey 1.21   1.89  WEGO
 Marshall,Donyell 2.05   3.34  WEGO
 Jefferson,Al 5.27   2.88  TMR
 Abdur-Rahim,Shareef 1.58   2.40  WEGO
 Williams,Shelden 3.98   6.87  WEGO
 West,David 2.30   2.23  TMR
 CAT DIFFERENCES WEGO TMR  WINNER
 Morrison,Adam 4.18   6.19  WEGO
 Bogut,Andrew 1.71   3.12  WEGO
 Roy,Brandon 1.05   1.72  WEGO
 Gay,Rudy 0.36  0.67   WEGO
 Diogu,Ike 3.06   2.91  TMR
 James,Mike 4.56   6.52  WEGO
 Walker,Antoine 1.85   2.81  WEGO
 Wells,Bonzi 6.40   5.83  TMR
 Davis,Baron 3.47   5.33  WEGO
 Wallace,Rasheed 0.78   0.45  TMR


Again, the methodology is sum of squared errors across the 8-categories. Lowest number wins per player. The results aren't even close. Overall, I win 36-14. In the players that I had higher, I won 17-8. In the players where TMR was higher, I won 22-3. And of the 10 players where we had the players rank similarly but the overall categories were different, I won 7-3. For these 50 players, my projections were 30% better on average. Not much of a challenge after all.

Speaking of TMR, this was about the time of year last season when he reviewed his 'signature' Guys I Love/Guys I Hate list and evaluated himself with flying colors (despite overwhelming evidence to the contrary). I don't know if he's brave enough to do the same this year, so let's do it for him. His basic premise is that his 'Guys I Love' are guys that will exceed expectations and vice versa. We can evaluated how he did using the consensus of the projections used above as the baseline. If a guy he loves outperformed the consensus expectations, then he gets a win. Simple enough.

TMR: GUYS I LOVE

GUYS I LOVEConsensus TMR ProjActualW/L
Barbosa,Leandro 8.41 7.98 13.53WIN
Warrick,Hakim 4.60 5.27 7.90WIN
Okafor,Emeka 10.38 10.10 13.61WIN
Williams,Maurice 9.48 9.11 12.71WIN
Gordon,Ben 10.14 10.56 12.71WIN
Smith,Josh 12.16 10.92 14.64WIN
Smith,J.R. 6.69 7.73 9.03WIN
Howard,Josh 11.13 10.76 13.46WIN
Pietrus,Mickael 6.25 6.56 8.21WIN
Brown,Kwame 6.09 5.99 7.71WIN
Hilario,Nene 8.76 8.86 9.97WIN
Livingston,Shaun 7.51 8.09 8.69WIN
Hinrich,Kirk 12.12 11.86 13.29WIN
Terry,Jason 12.22 12.60 13.15WIN
Curry,Eddy 7.83 7.41 8.66WIN
Krstic,Nenad 9.29 9.27 9.97WIN
Parker,Tony 11.76 11.30 12.41WIN
Bogut,Andrew 9.95 10.06 10.32WIN
Williams,Marvin 7.62 9.18 7.91WIN
Wallace,Gerald 14.08 13.65 14.34WIN
Bosh,Chris 14.60 13.30 14.74WIN
Granger,Danny 10.03 10.69 10.04WIN
Miles,Darius 8.03 8.39 0.00LOSS
Kirilenko,Andrei 16.42 14.88 10.51LOSS
James,Mike 11.93 12.32 7.04LOSS
Claxton,Speedy 9.23 9.83 5.50LOSS
Daniels,Marquis 7.86 7.54 4.14LOSS
Nelson,Jameer 12.19 12.05 8.76LOSS
Mohammed,Nazr 8.89 9.38 5.49LOSS
Frye,Channing 9.49 9.47 6.25LOSS
Stojakovic,Peja 12.79 13.96 9.86LOSS
Szczerbiak,Wally 10.86 10.70 8.22LOSS
Milicic,Darko 9.13 9.70 6.74LOSS
Kaman,Chris 10.93 11.09 8.75LOSS
Ilgauskas,Zydrunas 12.10 12.26 9.98LOSS
Jones,Fred 7.05 6.05 5.11LOSS
Villanueva,Charlie 10.17 11.28 8.38LOSS
Marbury,Stephon 11.95 12.31 10.22LOSS
Okur,Mehmet 11.74 11.64 11.16LOSS
Felton,Raymond 11.39 11.97 10.82LOSS
Odom,Lamar 13.55 13.03 13.03LOSS
West,David 11.51 12.61 11.00LOSS
Battier,Shane 10.59 11.21 10.12LOSS
Wilcox,Chris 9.77 9.23 9.36LOSS
West,Delonte 9.94 10.51 9.70LOSS
Alston,Rafer 11.05 10.87 10.81LOSS
Gomes,Ryan 8.11 9.84 7.97LOSS
Johnson,Joe 14.16 13.04 14.06LOSS


TMR did a great job selecting Barbosa, Warrick, Okafor, Josh Smith and Josh Howard, but this was far outweighed by missing the mark on Kirilenko, Claxton (he kept pimping Speedy all year hoping for a tunraround), Marquis Daniels, Channing Frye, etc. Overall, I have him with a win-loss record of 22-26 on his win list, and on average a Berry recommended player UNDERPERFORMED consensus expectations by 4.5%.

One other thing worth noting is that while TMR recommended a whole slew of players, a number of his 'Guys I Love' had much lower than consensus numbers in the projections that he was selling on his site! Check out Josh Smith, Chris Bosh, Eddy Curry, Barbosa, Tony Parker and Gerald Wallace. Why the disconnect? Keep this in mind if he decides to write a column next week and calls Joe Johnson a win.

TMR: GUYS I HATE

GUYS I HATEConsensus TMR ProjActualW/L
O'Neal,Shaquille 9.89 11.28 6.33WIN
Przybilla,Joel 7.77 8.62 4.31WIN
Telfair,Sebastian 7.50 5.96 4.42WIN
Rose,Jalen 5.56 6.73 2.72WIN
Redick,JJ 6.64 5.37 3.81WIN
Brown,P.J. 7.36 7.77 4.61WIN
Francis,Steve 9.82 8.92 8.34WIN
Gooden,Drew 9.16 9.09 8.50WIN
Maggette,Corey 10.90 11.91 10.44WIN
Dunleavy,Mike 9.76 9.37 9.64WIN
Stackhouse,Jerry 7.60 7.68 7.64LOSS
Bell,Raja 9.79 9.17 9.85LOSS
Thomas,Tim 7.70 7.38 8.41LOSS
Dalembert,Samuel 10.87 10.69 11.64LOSS
Davis,Ricky 11.45 11.27 12.26LOSS
Tinsley,Jamaal 9.28 9.04 10.16LOSS
Pachulia,Zaza 8.41 6.88 9.53LOSS
Crawford,Jamal 9.04 7.47 10.47LOSS
Anthony,Carmelo 13.59 12.63 15.25LOSS
Duncan,Tim 13.58 13.61 15.37LOSS
Davis,Baron 13.79 13.25 15.67LOSS
Miller,Mike 11.50 12.48 13.79LOSS
Boozer,Carlos 12.48 12.24 15.05LOSS
Butler,Caron 12.78 11.33 15.36LOSS
Chandler,Tyson 8.67 8.68 11.57LOSS
Randolph,Zach 9.57 9.60 13.16LOSS
Richardson,Quentin 5.79 5.46 9.45LOSS


TMR was spot on with Shaq and Telfair. Way to go! However, his projections had much higher numbers for Shaq, Jalen Rose and Corey Maggette. Berry completely missed the boat on Zach Randolph, Tyson Chandler, Caron Butler (a big miss), Carlos Boozer and Mike Miller. I've got him 10-17 in his 'Guys I Hate' list, and a guy on this list OUTPERFORMED consensus expectations by 2.9% on average.

All in all, not good. We'll see how TMR grades himself if he decides to write that column again this year. In the words of Berry himself: "When you are an “Expert” – that is to say, when you are paid for your writing and predictions – especially in fantasy sports, you are only as good as your prognostications." What does it say when his recommended players underperform and his avoid players outperform?

0 comments:

 
Wegoblogger #31 © 2011 | Designed by Bingo Cash, in collaboration with Modern Warfare 3, VPS Hosting and Compare Web Hosting