1. Welcome to CowboysZone!  Join us!  Come on!  You know you want to!

How Passer Rating Works

Discussion in 'Fan Zone' started by percyhoward, Jul 1, 2013.

  1. AbeBeta

    AbeBeta Well-Known Member

    25,597 Messages
    1,013 Likes Received
    Percy, first, don't say "significant difference" - that has a specific statistical meaning and you aren't using the phrase correctly.

    Second, the statistics you presented have no bearing on predicting wins or as DMyers highlights, points.

    Third, and a bit more troubling is that the values you present confound defensive play with offensive performance. How a defense plays has zero bearing on how we evaluate a QB's performance. For example, Chicago was a terrible passing team -- bottom quarter of the league, yet given your passing "statistics" they are top ten. Obviously they are top 10 because they have a tough D to pass against.

    Fourth, you still haven't put the statistics to "practical use" -- practical use is predicting wins, not predicting some sort of newly made up ranking.
  2. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    Team's Quarterback Rating - Team's Quarterback Rating Allowed = Quarterback Rating Differential

    I don't use ESPN's metric.




    YR
  3. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    Another metric I liked to look at is yards gained per points scored. Conversely, yards allowed per points allowed.

    Ideally, teams want to have a low yards gained per point scored. And a high yards allowed per point allowed.

    For the offensive side of the ball, it not only gives an indication of a team's ability on offense, but their ability to get turnovers on defense (shortens the field or if the defense scores even better). It also has a reflection on the offensive side of special teams (returns and field goals) and the ability for the offense to punch it in for a TD instead of settling for FG's.

    For the defensive side, it shows the defense's ability and the offense's ability to not turn the ball over, particularly in their own side of the field and the defensive side of special teams (punting and kick coverage).

    It takes into consideration more of the entire game than QBRD. The issue is dissecting the team's actual ranking in these metrics.





    YR
  4. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    I'd say that interpretation is in the eye of the beholder. Anyway, I'm glad we're clear on that point now.

    They weren't intended to predict wins or points. They were intended to show how similar passer rating is to ANY/A, irrespective of their correlation to winning.

    Doesn't matter, because it's all about evaluating QB performance. You're making a distinction where there is no real difference, since the way we know the Bears were tough to pass against is by evaluating the performance of the QB that faced them. The only difference is point of view. It's a net stat, in other words.

    Go to any NFL stats page, and you'll see statistics in practice. Passing leaders, rushing leaders, etc.

    I'm flattered that you think passer rating differential is something I just came up with myself in this thread, though.
  5. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
  6. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    Your reply is insulting, avoids my point, and is unnecessary. And I am insulted by your clear unwillingness to take a single thing I say seriously.

    I hope to return the favor someday.

    D-
  7. AbeBeta

    AbeBeta Well-Known Member

    25,597 Messages
    1,013 Likes Received
    No, it isn't. You shouldn't use specific terms that have specific meanings in a sloppy manner.


    But you weren't showing that. You were showing differential. Any difference statistic is likely to have substantially reduced variability compared to the values that go into the statistic. Simply list the PR vs. ANY/A in terms of ranking if you want that sort of comparison.

    Of course, as I've noted since both values contain a great deal of similar information then you are bound to have similar rankings. However, as I've also demonstrated two statistics that show the SAME rankings can have a substantially different impact on correlation.


    If you truly think passer rating differential is a clean measure of a QB's performance then you really are a lost cause. You simply cannot measure your QB's value in terms of how the opposition's QB performed.


    In practice is not a metric of practical value. Practical value relates to what is on the scoreboard (and what puts butts in the seats).
  8. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    Yes, it's the same. CHFF aped it off me as well since I've been using starting in 2002. I had an old blog that used to discuss it and follow it each week. They just named it 'Passer Rating Differential' when I named it QBRD. They should credit me for this.




    YR
  9. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    Wow, I'm sincerely sorry that you took it that way, since it wasn't meant that way at all.

    We're clearly not on the same page when it comes to discussing this topic. I simply asked if there was a metric that you considered superior, all things considered. And not with respect to ease of calculation, but with respect to the stat's ability to measure passing performance and correlate to winning.

    I asked you that out of respect for your opinion and your experience in dealing with the topic, so it's unfortunate that I've offended you.

    I'm sure your answer would be of interest to me, anyway.
  10. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    Is that a fact? It's a great stat, and the key of course was coming up with the idea of looking at the differential. Although, I don't think there's much difference in the two stats, I'm surprised that ANY/A differential is so hard to find. Seems like it would have been the next logical step, after coming up with ANY/A.

    I started using passer rating for receivers before I knew of anybody else doing it.
  11. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    I'll try to better anticipate how you might interpret what I say.

    Sure. Just remember it was your idea.

    PR (vs. ANY/A)
    offense only
    1 GB (3)
    2 Den (1)
    3 Was (2)
    4 SF (6)
    5 Sea (7)
    6 Atl (8)
    7 NE (4)

    8 NO (5)
    9 Dal (11)
    10 Hou (13)

    14 NYG (10)
    15 Car (9)
    22 MIn (27)
    27 Ind (20)


    Passer rating puts the top 7, and 8 of the top 10 in the playoffs. Minny and Indy don't follow the pattern.
    ANY/A puts the top 4, and 7 of the top 10 in the playoffs. Minny and Indy don't follow the pattern.

    Now tell me how that shows why ANY/A is a much different, much better stat.


    I had no idea how little you understood of what I was trying to do. I was measuring differential to see how many playoff teams were in the top 12. Obviously, you're going to have more good teams at the top of the rankings if you use differential rather than using just half the picture (offense or defense only).
  12. AbeBeta

    AbeBeta Well-Known Member

    25,597 Messages
    1,013 Likes Received
    Percy -- there is a clear definition of the phrase "statistically significant" -- look it up.


    I had no idea how little you understood of what I was trying to do. I was measuring differential to see how many playoff teams were in the top 12. Obviously, you're going to have more good teams at the top of the rankings if you use differential rather than using just half the picture (offense or defense only).[/quote]

    Oh poor misunderstood Percy! You start the thread as some sort of clinic in explaining PR, then start talking about correlations with wins, then move on to some silly analysis that supposedly proves the two are indistinguishable (by some criteria that you have established). The reason why I misunderstand what you are doing is that I assumed some base level of statistical competence on your part. Finding that lacking, you can't blame me for being confused.
  13. ABQCOWBOY

    ABQCOWBOY Moderator Staff Member

    35,186 Messages
    2,027 Likes Received
    Good info Percy. Thanks for posting this thread.
  14. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    I didn't use that familiar phrase, Abe.

    It was to show how passer rating works when attempts are added. Specifically to dispel the idea that more attempts equal a higher rating. You won't find anything in that 50-word OP other than an explanation of the effects of added attempts on passer rating.

    You said, "If you want a comparison, simply list the PR vs. ANY/A in terms of ranking" OK, I did.

    Again I ask, how does this show why ANY/A is a much different, much better stat?
  15. AbeBeta

    AbeBeta Well-Known Member

    25,597 Messages
    1,013 Likes Received
    You said "significant difference" you said "that difference doesn't significantly affect" and "difference shown by the analysis of the stats is really a significant one" -- all the same family of phrases. Don't say "significant" in a statistical conversation without using it as it is precisely defined statistically.

    Percy - I don't need to show you now ANY/A is different. You posted a .55 correlation between wins and AYA/A vs. a .51 correlation between wins and PR. STATISTICAL FACT ALERT: .55 is larger than .51 and substantially so in statistical terms. PR explain about 26% of what goes into wins whereas AYA/A explains 30%. The proof is right there, too bad you don't understand it.

    And Percy -- when I said "simply list" I assumed you understood that doing this for a single year was a stupid idea given my multiple comments on that.
  16. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    You're really trying to nail me on this point, aren't you? The phrase in question was "statistically significant," and I never used it. At no time did I say the correlation coefficients' differential of .04 was not statistically significant. In fact, for purposes of this debate, I even acknowledged that .04 is statistically significant.

    What I did do was raise the question of how significant (substantial, important, meaningful), in practical terms, that .04's worth of difference could be when there is no significant difference in the comparison of the two stats using the 2012 rankings of passer rating differential and ANY/A differential, and no significant difference in the comparison of the two stats using the Cowboys' offensive and defensive passer ratings and ANY/A in the Romo era.

    No confusion about what has been proved by the math -- that ANY/A has a higher correlation to winning than passer rating. There is no doubt that there is a mathematical difference. For you, it ends there. So whenever somebody posts a passer rating stat, I'm sure you'll be there to remind them that ANY/A is substantially better, and you'll post the ANY/A version (which may or may not be an improvement, as we have seen).

    So name your sample size. That was 256-game sample I showed you, and ANY/A did worse than passer rating at predicting playoff teams. Would it take another 256 games before ANY/A begins to catch up? I don't know, maybe you should look at the 2011 season. A thousand games before we begin to see ANY/A's obvious superiority? Have at it.
  17. percyhoward

    percyhoward Research Tool

    8,642 Messages
    2,752 Likes Received
    Here's a 13-year sample of team offensive passer rating vs. team offensive ANY/A, comparing the number of playoff teams that made the top 10 in each of those categories. It shows no significant difference in the ability of the two stats to predict playoff teams. Just more evidence that the two stats are basically the same.

    The ANY/A used here is (Passing Yards + 20*Passing Touchdowns - 45*Interceptions - Sack Yards Lost) / (Pass Attempts + Sacks)

    2000-2012
    Number of playoff teams in top 10


    2012
    PR 8
    ANYA 7

    2011
    PR 9
    ANYA 8

    2010
    PR 7
    ANYA 6

    2009
    PR 7
    ANYA 7

    2008
    PR 6
    ANYA 6

    2007
    PR 8
    ANYA 8

    2006
    PR 9
    ANYA 7

    2005
    PR 8
    ANYA 9

    2004
    PR 8
    ANYA 9

    2003
    PR 6
    ANYA 6

    2002
    PR 8
    ANYA 8

    2001
    PR 7
    ANYA 7

    2000
    PR 6
    ANYA 7

    13-year totals
    PR 97 of 130
    ANYA 95 of 130


    Here are some more links, all of them with analyses of the two stats. Interestingly, the more recent the time period covered by the study, the less significant the statistical difference between the two stats.

    http://www.itsalloverfatman.com/bro...total-quarterback-rating-correlate-to-winning


    http://www.bloggingtheboys.com/2013...on-priority-fixing-passer-rating-differential


    http://thesportingtruth.com/?p=1723

    I understand why some people prefer ANY/A to passer rating. ANY/A is easier to figure, logical in its design (being based as it is on the actual calculated values of its components), and completely transparent. Passer rating is quite complicated, and seemingly random in its design. For me, those two points lose their meaning in light of the fact that the stat correlates so highly with winning, and that I know how to use an online passer rating calculator.
  18. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    When I used to write my blog, I used to talk and do predictive forecasting on QBRD almost every week. Eventually people like KC Joyner, Aaron Schatz and others started to read my blog for some of my statistical work (I did things like giving a grade for O-Linemen which was a pretty good ballpark figure and many stat-heads were interested in it).

    Anyway, Schatz and Joyner were nothing but nice and helpful to me. I'm pretty positive the CHFF guys were reading the blog back then. There's stuff with me referencing QBRD here...I think back to 2006 when I joined here. A friend of mine came up with the term QB Rating Differential and I started using it from there. It's no surprised to me that CHFF just happens to have something called 'Passer Rating Differential.' The 'differential' part is most damning since it's not a common term, even in statistical terms. And they just so happened to use that word.




    YR

Share This Page