Just how noisy (how much error) is there in the NFL draft

[dwmyers]

http://codeandfootball.wordpress.co...hing-moments-just-how-noisy-is-the-nfl-draft/

Every draft has a moment where you see a player drafted, and you wonder what drove a team to take this player. In the 2011 draft I can recall off the top of my head at least three four head scratching moments: the draft by San Francisco of Aldon Smith (Ourlads 8.99, but rising), by Tennessee of Jake Locker (Ourlads 9.15, considered by many to be late first, second round) , by Seattle of James Carpenter (Ourlads 7.05), and the draft by New England of Ras-I-Dowling (Ourlads 7.82, but perhaps scheme related). All four left me wondering. Perhaps the same do to you, perhaps they don’t. But what I’m getting at is the number of these moments defines an error level by its recognizable tails, and using that, we can back track to an estimate of the actual error involved in selecting players.

 

[nathanlt]

The lack of preciseness when it comes to draft grades calls into question any mathematical analysis, no matter how exhaustive.

There's just too much variation from one scouting report and grades on the same player.

 

[DFWJC]

http://codeandfootball.wordpress.co...hing-moments-just-how-noisy-is-the-nfl-draft/



[URL="http://codeandfootball.files.wordpress.com/2011/05/draft_dist.png"][/URL]

This is nonsense.
Maybe in 5 years you can say what was "error' or not.

In fact last year, one of the uh-oh moments was when Jacksonbille took Alualu at #10.
Turns out that they were the ones that got it right and not the so-called draft graders (or the preceeding mock drafters).

 

[speedkilz88]

This is nonsense.
Maybe in 5 years you can say what was "error' or not.

In fact last year, one of the uh-oh moments was when Jacksonbille took Alualu at #10.
Turns out that they were the ones that got it right and not the so-called draft graders (or the preceeding mock drafters).

The talk on Alualu was that they took him too high, not that he wasn't a good prospect. The Cowboys had him as the 22nd player in the first and had 23 prospects worthy of a first round pick.(according to the leaked board) Reports are that the Cowboys really liked him also.

 

[DFWJC]

The talk on Alualu was that they took him too high, not that he wasn't a good prospect. The Cowboys had him as the 22nd player in the first and had 23 prospects worthy of a first round pick.(according to the leaked board) Reports are that the Cowboys really liked him also.

That's true, but since then we have learned that Pittsburgh (they did fine anyway) was trading up to get him as well. And as many as 3 other teams were hot for him too. I guess my point is aimed more at the mocks drafts being off and the media folks using mock draft projections to do post draft grades.

I think you're right about the Cowboys assessment of him. It turns out that he very well may have deserved a top ten slot though. The Boy's did just fine though....if our guy can keep his head on his shoulders, he was a massive steal.

 

[dwmyers]

This is nonsense.

No, What I did and said is definitely not nonsense. I'm saying that you can get an estimate of the scouting error in the NFL by looking at reaches throughout the draft, and further, that any model of scouting that can't generate that error isn't a good model of NFL scouting.

Whether you wish to argue with my conclusions or my methods is another issue entirely, but I'm eminently sensible here.

David.

 

[DFWJC]

No, What I did and said is definitely not nonsense. I'm saying that you can get an estimate of the scouting error in the NFL by looking at reaches throughout the draft, and further, that any model of scouting that can't generate that error isn't a good model of NFL scouting.

Whether you wish to argue with my conclusions or my methods is another issue entirely, but I'm eminently sensible here.

David.

I see what you're doing statistically and have no problem with that.

It's the input data that does not reflect reality. The entire premis is that someone knows where exactly a player should be chosen based on some preconceived notion. The only way to know where someone should go in correllation to how good a team drafted would be be look at how a player later performs, not how some made-up pre draft grade or rating says they should be picked.

And who is to say one pre-draft ranking is better than another one?
Recall that many of these so-called rankings had JaMarcus Russell rated very highly or Tom Brady very lowly....Obviously they were way off.

Just because some pre-draft format grades a player a certain way, does not at all mean a team who picks him earlier drafted poorly. It may turn out to be entirely the opposite actually.

The bottom line is that this does not at all refect how good a team drafts....not at all.
It only reflects how good a team drafted based on a preconceived notion of how good players are or will become---which has been proven to be very inaccurate.
As they say---garbage in, garbage out.

 

[dogberry]

Speaking or writing of leaked boards, was a picture leaked this year?

 

[dwmyers]

It's the input data that does not reflect reality. The entire premis is that someone knows where exactly a player should be chosen based on some preconceived notion. The only way to know where someone should go in correllation to how good a team drafted would be be look at how a player later performs, not how some made-up pre draft grade or rating says they should be picked.

I disagree. Teams draft players based on scouting rankings, which have to come from somewhere. Those valuations are required in order to draft. You cannot draft really, unless you have some kind of valuation of players.

Those "preconceived notions" do exist, else no draft takes place. This idea that you can't value picks until, oh, 20 years has passed has value in the context of history, but it won't tell you how to pick a better choice *now*.

Understand, my methodology doesn't give a hoot about what the best picks are, only that they exist and that assuming an efficient market, that scouting has some rough idea what those values are. You're obsessing over what those best picks are.

I want to estimate what the error level in this process is, so I can figure out how much benefit I get by eliminating that error.

I'm after the Moneyball play. You're after historical context.

I develop my ideas further here.

http://codeandfootball.wordpress.com/2011/05/15/draft-accuracy-matters/

 

[DFWJC]

I disagree. Teams draft players based on scouting rankings, which have to come from somewhere. Those valuations are required in order to draft. You cannot draft really, unless you have some kind of valuation of players.

Those "preconceived notions" do exist, else no draft takes place. This idea that you can't value picks until, oh, 20 years has passed has value in the context of history, but it won't tell you how to pick a better choice *now*.

You assume that teams use a common (or very near identical) database and, of course, they do that...though some may have rating more similar to others.

A teams valuations are private and they may vary greatly from some random scouting database out there. Why hire your own scouts and personnel dept. if there's a common database.

We just seem to disagree on the input data here. Saying that someone drafted poorly becaseu they didn't draft the way some scouting report (obviously not one used by the team drafting) is just plain wrong....IMHO.
If that's the case then Tom Brady was a poor draft pick.

I do think you can wait as few as 1-3 years to determine how well teams are drafting...nopt sure where you got 20 from.

Let's just agree to disagree. It's not personal or anything.

I DO think your stats work will tell how certain teams differ form the "norm". If that is the attempt, then it is effective....as long as we all know that the "norm" can be very average or even poor at times.

 

[dwmyers]

You assume that teams use a common (or very near identical) database

I have a 10 year old sourceforge project that says differently. I've never made that particular assumption. Nor did my analysis, in this case, in any way assume the points of view of the 32 teams were the same. Each was uniquely scouted.

I will note that if you're trying to claim I'm examining my simulations from a single point of view in my analysis, I'd say that is reasonable. But if I'm Team A, trying to gain an advantage on Team B, I *don't need* to be sympathetic to any point of view Team B might have. If their players suck, they suck.

D-