How Passer Rating Works

[AbeBeta]

I'm not talking about the established fact that the two stats differentially predict wins. I acknowledge that fact, OK?

I'm pointing out that there is no significant difference in the results when you put the two stats to practical use.

Percy, first, don't say "significant difference" - that has a specific statistical meaning and you aren't using the phrase correctly.

Second, the statistics you presented have no bearing on predicting wins or as DMyers highlights, points.

Third, and a bit more troubling is that the values you present confound defensive play with offensive performance. How a defense plays has zero bearing on how we evaluate a QB's performance. For example, Chicago was a terrible passing team -- bottom quarter of the league, yet given your passing "statistics" they are top ten. Obviously they are top 10 because they have a tough D to pass against.

Fourth, you still haven't put the statistics to "practical use" -- practical use is predicting wins, not predicting some sort of newly made up ranking.

 

[Yakuza Rich]

By "QB Rating Differential" do you mean something other than passer rating differential? It was .80 for 2011.

Team's Quarterback Rating - Team's Quarterback Rating Allowed = Quarterback Rating Differential

I don't use ESPN's metric.




YR

 

[Yakuza Rich]

Another metric I liked to look at is yards gained per points scored. Conversely, yards allowed per points allowed.

Ideally, teams want to have a low yards gained per point scored. And a high yards allowed per point allowed.

For the offensive side of the ball, it not only gives an indication of a team's ability on offense, but their ability to get turnovers on defense (shortens the field or if the defense scores even better). It also has a reflection on the offensive side of special teams (returns and field goals) and the ability for the offense to punch it in for a TD instead of settling for FG's.

For the defensive side, it shows the defense's ability and the offense's ability to not turn the ball over, particularly in their own side of the field and the defensive side of special teams (punting and kick coverage).

It takes into consideration more of the entire game than QBRD. The issue is dissecting the team's actual ranking in these metrics.





YR

 

[percyhoward]

Percy, first, don't say "significant difference" - that has a specific statistical meaning and you aren't using the phrase correctly.

I'd say that interpretation is in the eye of the beholder. Anyway, I'm glad we're clear on that point now.

Second, the statistics you presented have no bearing on predicting wins or as DMyers highlights, points.

They weren't intended to predict wins or points. They were intended to show how similar passer rating is to ANY/A, irrespective of their correlation to winning.

Third, and a bit more troubling is that the values you present confound defensive play with offensive performance. How a defense plays has zero bearing on how we evaluate a QB's performance. For example, Chicago was a terrible passing team -- bottom quarter of the league, yet given your passing "statistics" they are top ten. Obviously they are top 10 because they have a tough D to pass against.

Doesn't matter, because it's all about evaluating QB performance. You're making a distinction where there is no real difference, since the way we know the Bears were tough to pass against is by evaluating the performance of the QB that faced them. The only difference is point of view. It's a net stat, in other words.

Fourth, you still haven't put the statistics to "practical use" -- practical use is predicting wins, not predicting some sort of newly made up ranking.

Go to any NFL stats page, and you'll see statistics in practice. Passing leaders, rushing leaders, etc.

I'm flattered that you think passer rating differential is something I just came up with myself in this thread, though.

 

[percyhoward]

Team's Quarterback Rating - Team's Quarterback Rating Allowed = Quarterback Rating Differential

For 2012, that would be using the passer ratings found here, for example, correct?

http://www.coldhardfootballfacts.com/stats/2012/Final/PRD/

http://www.pro-football-reference.com/years/2012/

 

[dwmyers]

No, I got it the first time. Teams that suck but that have good QB will win more games than teams that suck but don't have good QB, true. But teams that suck will still lose more games than teams that don't, no matter how good the QB is.

Your reply is insulting, avoids my point, and is unnecessary. And I am insulted by your clear unwillingness to take a single thing I say seriously.

I hope to return the favor someday.

D-

 

[AbeBeta]

I'd say that interpretation is in the eye of the beholder. Anyway, I'm glad we're clear on that point now.

No, it isn't. You shouldn't use specific terms that have specific meanings in a sloppy manner.

They weren't intended to predict wins or points. They were intended to show how similar passer rating is to ANY/A, irrespective of their correlation to winning.

But you weren't showing that. You were showing differential. Any difference statistic is likely to have substantially reduced variability compared to the values that go into the statistic. Simply list the PR vs. ANY/A in terms of ranking if you want that sort of comparison.

Of course, as I've noted since both values contain a great deal of similar information then you are bound to have similar rankings. However, as I've also demonstrated two statistics that show the SAME rankings can have a substantially different impact on correlation.

Doesn't matter, because it's all about evaluating QB performance. You're making a distinction where there is no real difference, since the way we know the Bears were tough to pass against is by evaluating the performance of the QB that faced them. The only difference is point of view. It's a net stat, in other words.

If you truly think passer rating differential is a clean measure of a QB's performance then you really are a lost cause. You simply cannot measure your QB's value in terms of how the opposition's QB performed.

Go to any NFL stats page, and you'll see statistics in practice. Passing leaders, rushing leaders, etc.

In practice is not a metric of practical value. Practical value relates to what is on the scoreboard (and what puts butts in the seats).

 

[Yakuza Rich]

For 2012, that would be using the passer ratings found here, for example, correct?

http://www.coldhardfootballfacts.com/stats/2012/Final/PRD/

Yes, it's the same. CHFF aped it off me as well since I've been using starting in 2002. I had an old blog that used to discuss it and follow it each week. They just named it 'Passer Rating Differential' when I named it QBRD. They should credit me for this.




YR

 

[percyhoward]

Your reply is insulting, avoids my point, and is unnecessary. And I am insulted by your clear unwillingness to take a single thing I say seriously.

Wow, I'm sincerely sorry that you took it that way, since it wasn't meant that way at all.

We're clearly not on the same page when it comes to discussing this topic. I simply asked if there was a metric that you considered superior, all things considered. And not with respect to ease of calculation, but with respect to the stat's ability to measure passing performance and correlate to winning.

I asked you that out of respect for your opinion and your experience in dealing with the topic, so it's unfortunate that I've offended you.

I'm sure your answer would be of interest to me, anyway.

 

[percyhoward]

Yes, it's the same. CHFF aped it off me as well since I've been using starting in 2002. I had an old blog that used to discuss it and follow it each week. They just named it 'Passer Rating Differential' when I named it QBRD. They should credit me for this.

Is that a fact? It's a great stat, and the key of course was coming up with the idea of looking at the differential. Although, I don't think there's much difference in the two stats, I'm surprised that ANY/A differential is so hard to find. Seems like it would have been the next logical step, after coming up with ANY/A.

I started using passer rating for receivers before I knew of anybody else doing it.

 

[percyhoward]

No, it isn't. You shouldn't use specific terms that have specific meanings in a sloppy manner.

I'll try to better anticipate how you might interpret what I say.

But you weren't showing that. You were showing differential. Any difference statistic is likely to have substantially reduced variability compared to the values that go into the statistic. Simply list the PR vs. ANY/A in terms of ranking if you want that sort of comparison.

Sure. Just remember it was your idea.

PR (vs. ANY/A)
offense only
1 GB (3)
2 Den (1)
3 Was (2)
4 SF (6)
5 Sea (7)
6 Atl (8)
7 NE (4)
8 NO (5)
9 Dal (11)
10 Hou (13)

14 NYG (10)
15 Car (9)
22 MIn (27)
27 Ind (20)

Passer rating puts the top 7, and 8 of the top 10 in the playoffs. Minny and Indy don't follow the pattern.
ANY/A puts the top 4, and 7 of the top 10 in the playoffs. Minny and Indy don't follow the pattern.

Now tell me how that shows why ANY/A is a much different, much better stat.

If you truly think passer rating differential is a clean measure of a QB's performance then you really are a lost cause. You simply cannot measure your QB's value in terms of how the opposition's QB performed.

I had no idea how little you understood of what I was trying to do. I was measuring differential to see how many playoff teams were in the top 12. Obviously, you're going to have more good teams at the top of the rankings if you use differential rather than using just half the picture (offense or defense only).

 

[AbeBeta]

I'll try to better anticipate how you might interpret what I say.

Percy -- there is a clear definition of the phrase "statistically significant" -- look it up.

I'll try to better anticipate how you might interpret what I say.

I had no idea how little you understood of what I was trying to do. I was measuring differential to see how many playoff teams were in the top 12. Obviously, you're going to have more good teams at the top of the rankings if you use differential rather than using just half the picture (offense or defense only).[/quote]

Oh poor misunderstood Percy! You start the thread as some sort of clinic in explaining PR, then start talking about correlations with wins, then move on to some silly analysis that supposedly proves the two are indistinguishable (by some criteria that you have established). The reason why I misunderstand what you are doing is that I assumed some base level of statistical competence on your part. Finding that lacking, you can't blame me for being confused.

 

[ABQCOWBOY]

Good info Percy. Thanks for posting this thread.

 

[percyhoward]

Percy -- there is a clear definition of the phrase "statistically significant" -- look it up.

I didn't use that familiar phrase, Abe.

You start the thread as some sort of clinic in explaining PR...

It was to show how passer rating works when attempts are added. Specifically to dispel the idea that more attempts equal a higher rating. You won't find anything in that 50-word OP other than an explanation of the effects of added attempts on passer rating.

You said, "If you want a comparison, simply list the PR vs. ANY/A in terms of ranking" OK, I did.

Again I ask, how does this show why ANY/A is a much different, much better stat?

 

[AbeBeta]

I didn't use that familiar phrase, Abe.

You said "significant difference" you said "that difference doesn't significantly affect" and "difference shown by the analysis of the stats is really a significant one" -- all the same family of phrases. Don't say "significant" in a statistical conversation without using it as it is precisely defined statistically.

It was to show how passer rating works when attempts are added. Specifically to dispel the idea that more attempts equal a higher rating. You won't find anything in that 50-word OP other than an explanation of the effects of added attempts on passer rating.

You said, "If you want a comparison, simply list the PR vs. ANY/A in terms of ranking" OK, I did.

Again I ask, how does this show why ANY/A is a much different, much better stat?

Percy - I don't need to show you now ANY/A is different. You posted a .55 correlation between wins and AYA/A vs. a .51 correlation between wins and PR. STATISTICAL FACT ALERT: .55 is larger than .51 and substantially so in statistical terms. PR explain about 26% of what goes into wins whereas AYA/A explains 30%. The proof is right there, too bad you don't understand it.

And Percy -- when I said "simply list" I assumed you understood that doing this for a single year was a stupid idea given my multiple comments on that.

 

[percyhoward]

You said "significant difference"

You're really trying to nail me on this point, aren't you? The phrase in question was "statistically significant," and I never used it. At no time did I say the correlation coefficients' differential of .04 was not statistically significant. In fact, for purposes of this debate, I even acknowledged that .04 is statistically significant.

What I did do was raise the question of how significant (substantial, important, meaningful), in practical terms, that .04's worth of difference could be when there is no significant difference in the comparison of the two stats using the 2012 rankings of passer rating differential and ANY/A differential, and no significant difference in the comparison of the two stats using the Cowboys' offensive and defensive passer ratings and ANY/A in the Romo era.

Percy - I don't need to show you now ANY/A is different. You posted a .55 correlation between wins and AYA/A vs. a .51 correlation between wins and PR. STATISTICAL FACT ALERT: .55 is larger than .51 and substantially so in statistical terms. PR explain about 26% of what goes into wins whereas AYA/A explains 30%. The proof is right there, too bad you don't understand it.

No confusion about what has been proved by the math -- that ANY/A has a higher correlation to winning than passer rating. There is no doubt that there is a mathematical difference. For you, it ends there. So whenever somebody posts a passer rating stat, I'm sure you'll be there to remind them that ANY/A is substantially better, and you'll post the ANY/A version (which may or may not be an improvement, as we have seen).

And Percy -- when I said "simply list" I assumed you understood that doing this for a single year was a stupid idea given my multiple comments on that.

So name your sample size. That was 256-game sample I showed you, and ANY/A did worse than passer rating at predicting playoff teams. Would it take another 256 games before ANY/A begins to catch up? I don't know, maybe you should look at the 2011 season. A thousand games before we begin to see ANY/A's obvious superiority? Have at it.

 

[percyhoward]

Here's a 13-year sample of team offensive passer rating vs. team offensive ANY/A, comparing the number of playoff teams that made the top 10 in each of those categories. It shows no significant difference in the ability of the two stats to predict playoff teams. Just more evidence that the two stats are basically the same.

The ANY/A used here is (Passing Yards + 20*Passing Touchdowns - 45*Interceptions - Sack Yards Lost) / (Pass Attempts + Sacks)

2000-2012
Number of playoff teams in top 10

2012
PR 8
ANYA 7

2011
PR 9
ANYA 8

2010
PR 7
ANYA 6

2009
PR 7
ANYA 7

2008
PR 6
ANYA 6

2007
PR 8
ANYA 8

2006
PR 9
ANYA 7

2005
PR 8
ANYA 9

2004
PR 8
ANYA 9

2003
PR 6
ANYA 6

2002
PR 8
ANYA 8

2001
PR 7
ANYA 7

2000
PR 6
ANYA 7

13-year totals
PR 97 of 130
ANYA 95 of 130

Here are some more links, all of them with analyses of the two stats. Interestingly, the more recent the time period covered by the study, the less significant the statistical difference between the two stats.

http://www.itsalloverfatman.com/bro...total-quarterback-rating-correlate-to-winning


http://www.bloggingtheboys.com/2013...on-priority-fixing-passer-rating-differential


http://thesportingtruth.com/?p=1723

I understand why some people prefer ANY/A to passer rating. ANY/A is easier to figure, logical in its design (being based as it is on the actual calculated values of its components), and completely transparent. Passer rating is quite complicated, and seemingly random in its design. For me, those two points lose their meaning in light of the fact that the stat correlates so highly with winning, and that I know how to use an online passer rating calculator.

 

[Yakuza Rich]

Is that a fact? It's a great stat, and the key of course was coming up with the idea of looking at the differential. Although, I don't think there's much difference in the two stats, I'm surprised that ANY/A differential is so hard to find. Seems like it would have been the next logical step, after coming up with ANY/A.

I started using passer rating for receivers before I knew of anybody else doing it.

When I used to write my blog, I used to talk and do predictive forecasting on QBRD almost every week. Eventually people like KC Joyner, Aaron Schatz and others started to read my blog for some of my statistical work (I did things like giving a grade for O-Linemen which was a pretty good ballpark figure and many stat-heads were interested in it).

Anyway, Schatz and Joyner were nothing but nice and helpful to me. I'm pretty positive the CHFF guys were reading the blog back then. There's stuff with me referencing QBRD here...I think back to 2006 when I joined here. A friend of mine came up with the term QB Rating Differential and I started using it from there. It's no surprised to me that CHFF just happens to have something called 'Passer Rating Differential.' The 'differential' part is most damning since it's not a common term, even in statistical terms. And they just so happened to use that word.




YR