Grading the recruiting services: Who nailed the NFL Draft?

[InstiGATOR1]

I found this interesting article and thought many here would like it. There may be some confirmation bias, but that is open for discussion:

At 247Sports, the NFL Draft has always been our compass. Our 32 five-stars every year is reflective of the 32 first round draft picks every year. Our four-star rankings represent guys we think are NFL-bound. The scouts and personnel departments in the NFL are looking for talent, regardless of offensive system, team politics, depth charts and all the other factors that can impact a college career. They're just looking for the best players and that's what we want our rankings to reflect.

With the completion of the NFL draft, it's time to grade our work as it compares to the other major recruiting networks and rankings systems.

First, a note about our formula:

We used five sets of rankings: 247sports, 247Sports Composite, Rivals, Scout and ESPN. For each of the Top 100 picks in the NFL draft we assigned a point value to the team 1-5, depending on which rankings was the closest to the player's actual draft position. The lower the score the better. Strictly in the first round, we assigned bonus points – in this case bonus deductions – for each ranking that had a player ranked within its top 32 prospects for that cycle. For all 100 picks, we added a bonus deduction for any ranking that was a positive outlier (defined as 100 spots or more higher than any of the other 3 sites, excluding the composite) as well as a penalty point for a negative outlier (defined as 100 spots or more lower than any of the other 3 sites, excluding the composite). A three-star ranking also merited a penalty point if any of the other four sites had the prospect as a four-star or better.

We limited the rankings to the top 100 picks because the deeper you get into the draft the harder it is to measure the accuracy of each pick due to the variety of ways each site ranks 3-star prospects. We also didn't assign any points to prospects that every site missed and ranked below the four-star threshold.

This is how the rankings shook out.

Click on the link to see how there analysis came out:

http://247sports.com/Article/247Spo...s-best-the-competition-in-predicting-52607156

 

[EuroGator]

Blaspheme!

You know that only one site shall be regarded as worthy here! Rivals is the one and only site which can be sited because...


...well, we said so!

Thanks for sharing.

 

[URGatorBait]

so 247sports rated 247sports 1st...imagine that

 

[InstiGATOR1]

so 247sports rated 247sports 1st...imagine that

Like I said, it could be confirmation bias.

The way I read the incomplete discussion of the metric, if one service was best on all players they would have 100 points. If another was second on all players they would have 200 points. ETC. Up to one service was worst on each player, they would have 500 points since there are five rankings. So by my count 100+200+300+400+500=1500. But the rankings they list are 129+151+158+161+164=763 so we are missing 747 points? Is this because of the bonus deduction they are giving out? Are they really giving out a net 747 points in bonus deductions?

 

[maheo30]

I have a difficult time believing Scout did better than Rivals since their staff isn't even half of what Rivals' is.

 

[T REX]

247 is an average of all the sites...lol.

Stars don't matter

 

[InstiGATOR1]

247 is an average of all the sites...lol.

Stars don't matter

Not quite, their composite is some average of 247, ESPN, Rivals and Scout. Their own even beat their composite in the metric they devised.

 

[Gator Fever]

Rivals is just pickier with the 5 and 4 stars usually. I think 2 of the 1st round consensus 5 stars were Rivals 4 stars.

 

[BMF]

Blaspheme!

You know that only one site shall be regarded as worthy here! Rivals is the one and only site which can be sited because...


...well, we said so!

Thanks for sharing.

I think there's really just one poster here who subscribes to the thought that Rivals is the gold standard.....

I personally like the 247 composite....it makes sense (if 3 say 3-stars, and 1 says 4-stars....guess what? He's a 3-star!! Not rocket science....)

 

[Swamp Donkey]

LOL. 247 thinks 247 is best. Shocking. Also laughable.

Scout and ESPN are wastes. I could **** on toilet paper and count the specks and be more accurate. They both use interns and Bowl-type kids, mostly kids who don't even know damn thing about football, to do it.

Rivals is still the best but 247s OWN rankings are getting much better. Remember that the 247 owners used to be the Rivals owner. If you showed me a kid that was 4 star on Rivals and 4 star on 247 and 3 star on ESPN and Scout, then he is a four star. If you show me a kid who is 4 star on ESPN and Scout and 3 star on Rivals and 247, then he is a three star.

The strength and weakness of Rivals is that it focuses more on the southeast. They don't spend as much time scouring Vermont or Idaho for the next great four star that has never even played against another future D1 player. They are out there, no doubt, but it's a much bigger risk to take one of those players. ESPN in particular likes to add players from every state. Their job is to first sell TV advertising. It's why you see so many buffoons all over their staff.

247s composite is crap. It is better than ESPN or scout, but only because it averages Rivals in the mix.

You guys are just enamored by 247 advertising on everything all the time. Newest shiny whatever.

 

[Swamp Donkey]

Rivals is just pickier with the 5 and 4 stars usually. I think 2 of the 1st round consensus 5 stars were Rivals 4 stars.

Yeah, and sometimes those guys are five star bodies with three star heads (like Driskel). They are risks. Maybe maybe not. You would expect SOME of them to develop with the right coaching so it isn't really a shock.

I also like the fact that Rivals is pretty clear about levels within the star system. You can see their rivals rankings (like 5.5 or 5.9) to see who is a high four star, just on the cusp of being a 5 star, or who is a 4-. They have explanations as to why, ie with Driskel it was lack of awareness and inability to read defenses despite his incredible speed and arm strength.

247 spends most of it's time at the camps. That's ok because it's the only time you see kids against similar talent as will be the case in college, but it also leads to Camp Queens like Shytkowski being rated highly despite a total lack of production in HS.

 

[GatorJ]

OK. A couple things. Of course their final rankings mimic a lot of rivals anyway. Once somebody starts getting offers or rifles ends up ranking them higher they will immediately change the ranking. Rivals ranks the players and only changes rankings a handful of times. 247 depends on other sites to create their rankings. 247 will start ranking sophomore players out of the gate. Rivals waits to see them mature and then makes the rankings.

Also it's important to note that the rival rankings have nothing to do with NFL potential. Go to the website and look at them. They have to do with collegiate potential. It's very clear.

 

[Swamp Donkey]

Also it's important to note that the rival rankings have nothing to do with NFL potential. Go to the website and look at them. They have to do with collegiate potential. It's very clear.

This is a good point. Also by this standard, DJ Humphries was a success despite lack of interest in playing or producing in college because he was drafted highly in the NFL.

I note they don't compare their 5 star rankings to either all conference or all american achievements or by championship teams. This is what Rivals excels at and the only thing that matters to me, college performance.

 

[InstiGATOR1]

Yeah, and sometimes those guys are five star bodies with three star heads (like Driskel)..

Or get 3 star coaching in college.

 

[InstiGATOR1]

Also it's important to note that the rival rankings have nothing to do with NFL potential. Go to the website and look at them. They have to do with collegiate potential. It's very clear.

This is the metric I would use, but it is work. With the first 100 in the NFL draft, you only have to compare 100 players over 4 sites + the composite, while if you care about college then you probably need to look at 50 to so schools with a class of about 20ish players or 1000 players. That is a task, but it would tell you if recruiting rankings translated to production in college.