1. Welcome to CowboysZone!  Join us!  Come on!  You know you want to!

How Passer Rating Works

Discussion in 'Fan Zone' started by percyhoward, Jul 1, 2013.

  1. erod

    erod Well-Known Member

    4,854 Messages
    1,973 Likes Received
    The ESPN quarterback ratings are far better. It takes into account passes thrown away on purpose, and doesn't credit garbage time stats as much. It also takes into account a swing pass in the backfield that goes 90 yards for a score, versus a bomb hitting a receiver in stride.
  2. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    I'm looking for evidence of where this substantial mathematical difference that you're talking about manifests itself in the practical world of the rankings.

    Again, can you show this evidence?
  3. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    Evidence. Your claim that a correlation of .51 and .55 is meaningless and that they are the "essentially the same statistic'' is totally flawed and without basis. This tells us that over the time period you cite, PR was CONSISTENTLY less accurate in predicting WINS. You provide no argument based on WINS.

    I don't need to show you evidence. It is right there in the values you cite. That is, if you have more than an Intro Stats understanding of what those values mean.
  4. theogt

    theogt Surrealist Zone Supporter

    43,929 Messages
    989 Likes Received
    Well, I don't think there's any meaningful difference between .51 and .55 correlation coefficients, but I'd point out two things:

    1. The other formula is not much different from passer rating. The biggest difference is the inclusion of sack yards, which -- if itself correlates with winning to any real degree -- would presumably add to the correlation with winning percentage if combined with passer rating.

    2. Simply having a higher correlation with winning percentage is not the end goal. There are tons more stats that have a higher correlation with winning percentage. The question is -- are those a better judge of individual QB performance?

    I don't think there's any real argument for adding sack yardage to formula that ranks the performance quarterbacks.
  5. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Abe, I'm not claiming that the difference in the coefficients is meaningless. I'm saying it doesn't significantly affect the rankings to the point where there's any obvious advantage in using one stat or the other. That's the whole purpose of the stats anyway, to see where certain players or team rank, either from week to week, year to year, or against the rest of the league.

    When comparing the top teams' rankings in passer rating differential to their rankings in ANY/A differential, you get two strikingly similar lists. When doing the same thing for one team over time with Dallas' offensive and defensive passer ratings and corresponding ANY/A, you get two strikingly similar lists.

    So why argue over something that has virtually no effect on the rankings?
  6. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,071 Messages
    1,081 Likes Received
    It's saying that for 1992 that the Chris Miller was as effective of a passer of the ball as Aikman was.

    This is kinda the entire point of the post, how QB rating works and its level of importance in a season. My counter is that the formula is flawed towards dink-n-dunk passing schemes that throw the ball quite often near the goal line. Therefore, somebody like Aikman has a more difficult time achieving a higher QB rating since he was playing in a deep passing scheme that ran the ball near the goal line compared to Miller in the 'run-n-shoot' which threw a lot of short passes and threw the ball quite a bit near the goal line.

    Do I think Chris Miller was a better QB in 1992 than Aikman?

    Absolutely not.





    YR
  7. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Yes. WPA, EPA, and various versions of a ANY/A (and scoring, for that matter) all have a higher correlation with winning, but that doesn't make them passer ratings.
  8. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Again, Aikman destroyed Miller in passer rating when you compare each player's five best seasons, so Miller probably isn't the best example to use.

    You can only point to one season (1992) in which Miller was anywhere near Aikman's league, and there sample size is a major issue because Miller only played 8 games.

    Which is kinda what I said before.
  9. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,071 Messages
    1,081 Likes Received
    The issue here is we are missing out on several factors.

    One being is that the baselines are skewed and not on the same wavelength because there is nothing in place that puts in strength of schedule.

    If a QB averages an 85 QB rating going against defenses that average 70 QB rating allowed, he is a more effective QB than the one that has a 95 QB rating and has faced defenses averaging 100 QB rating allowed.

    This is really the biggie with any type of Bill James/Moneyball analysis. How the player performs with relation to the average as the baseline and the environment they are in (be it the opponent, weather, stadium influences, etc).

    I don't hate QB rating. And like I said, I find QBRD to be very helpful. But, there are inherent flaws to QB Rating and once we understand that we can understand how to use it for more accurate forecasting.



    YR
  10. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,071 Messages
    1,081 Likes Received
    Again, the entire point was how important QB Rating is in a season. You can argue Miller's sample size, but if we look at 1995 when the Cowboys won another Super Bowl, there was Scott Mitchell with virtually the same QB rating as Aikman.

    My main point is that the metric is flawed for a few reasons, one of them being it favors certain schemes and passing philosophies over others. It's not a useless metric, but it certainly has some noticeable flaws.



    YR
  11. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Not usually as much of a factor when comparing career numbers, but for week-to-week, and year-to-year comparisons, it makes a difference.

    Top 10 Passers 2012 (Rank of Schedule)
    1. Rodgers (7th most difficult)
    2. Manning (25th)
    3. Griffin (19th)
    4. Wilson (12th)
    5. Ryan (32nd)
    6. Brady (15th)
    7. Roethlisberger (27th)
    8. Brees (9th)
    9. Schaub (13th)
    10. Romo (6th)
  12. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    First - the difference between .51 and .55 is substantial. Anything that improves prediction of a complex outcome is useful.

    Second -- I initially thought sack yards was a strange addition on its own. If you add that, QB fumbles, and maybe some QB rushing metric you get a more well rounded picture of a QB's performance IMO. For example, a QB who is really good at getting rid of the ball when pressured is penalized under the PR formula but does get a bounce for avoiding sacks.

    I don't understand how not having a higher correlation to winning is not the goal. How your performance relates to the undisputed most important outcome is very important. Unless we are talking about predicting how many points an offense scores, I don't see a better value to evaluate against.
  13. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Ah, OK, you're saying don't put a lot of faith in one season's worth of games. Yeah, it's significant that the skewing you're talking about doesn't seem to withstand larger sample sizes. When you look at the list of QB who have finished in the top 10 in passer rating six times or more, you don't see any Chris Miller's or Scott Mitchell's.
  14. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    Percival, you are taking one analysis and then applying it in a manner that does not reflect the original analysis. The original analysis address how those values relate to wins. Not how they relate to where a team ranks. Two variables can RANK in exactly the same manner but capture remarkably different amounts of explanatory power. That ANY/A and PR reflect similar rankings fails to provide evidence that the two statistics are indistinguishable.
  15. nickjamesw43

    nickjamesw43 Active Member

    389 Messages
    126 Likes Received
    One point I would make is the "highest correlation with winning" is not enough on it's own. For instance I could award one million passer points whenever the quarterback throws a TD in the last 30 seconds to pull ahead. But this would not be an accurate formula to judge a quarterback skill as he is essentially being rewarded or punished for factors beyond his control. The key to creating any sort of statistic to evaluate a player is to isolate only the factors he does control. Therefore any useful statistic must also be context neutral.

    For instance if you know anything about baseball sabermetrics DIPS are defense independent pitching statistics. The idea is this: take the number of flyballs, groundballs, strikeouts and walks and come up with a number which represents the pitcher's true skill level. The idea is to isolate the pitcher from his defense and luck related to balls in play finding holes (the opponents batting average on balls in play was found to be largely uncontrollable by pitchers). The advantage is the contribution of the pitcher was isolated much more so then with ERA which can be strongly influenced by luck or bad defense (not to mention W-L record). Thus they are considered by the sabermetric community to be the best indicator of a pitchers skill.

    On the other hand in football we are going backwards. The latest offerings like ESPN's QBR instead try to take into account a quarterbacks "clutch" rating. Which as has been discussed before is not context neutral. Instead they should look to further isolate the quarterbacks performance perhaps accounting for things like skill of receivers, time in the pocket, strength of competition ect. I don't deny the passer rating is imperfect but most attempts to fix it actually are worse then passer rating itself.
  16. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    Poppycock.

    This isn't fantasy football.

    In the NFL winning is the only outcome that matters.
  17. percyhoward

    percyhoward Research Tool

    8,362 Messages
    2,269 Likes Received
    Yes, that coincidence can indeed occur.

    Now show how my examples were instances of such a coincidence.
  18. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    A. That is not a coincidence.

    B. Your "examples" have nothing to do with the winning % analysis. You are saying that ANY/A and PR both predict offense rank similarly. That tells us nothing about the winning % which is the analysis you told us showed no difference.

    C. (Bonus!) Here are three sets of #s A(1, 2, 3), B(1, 2, 40), C(0, 0, 1) -- let's say C is wins -- you get a 0 for a loss a 1 for a win. A and B could be two different passer ratings. Both A and B produce the same ranking (first is lowest, second middle, last highest). However the correlations of each with wins are different, meaning one is more predictive. Simple statistical fact there.
  19. nickjamesw43

    nickjamesw43 Active Member

    389 Messages
    126 Likes Received
    Winning also matters in baseball or any other sport. The idea is to isolate the factors purely under the quarterbacks control which lead to winning. That's what leads to a statistic which can evaluate a player accurately.
  20. AbeBeta

    AbeBeta Well-Known Member

    25,457 Messages
    921 Likes Received
    There are no factors "purely under the quarterbacks [sic] control."

Share This Page