1. Welcome to CowboysZone!  Join us!  Come on!  You know you want to!

How Passer Rating Works

Discussion in 'Fan Zone' started by percyhoward, Jul 1, 2013.

  1. nickjamesw43

    nickjamesw43 Active Member

    392 Messages
    132 Likes Received
    As purely as possible anyway that's the idea. As of now the closest we have are the components of passer rating.

    If you could further isolate the quarterback you can improve the statistic which is the point I was making earlier. If you judge by wins and losses you add statistical noise and further muddy the waters.
  2. AbeBeta

    AbeBeta Well-Known Member

    25,602 Messages
    1,017 Likes Received
    How on earth can you claim that is correct. How is an int all on the QB? How is a TD? How is a sack not part of that? How is a fumble by the QB not part of it?

    If you judge by wins and losses you add in an actual outcome that matters. There is no point to ranking performance if you are uninterested in how it relates to wins.
  3. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    I'm not saying that the analysis shows no difference. I'm saying that, when the stats are put into practical use, that difference doesn't significantly affect the resulting set of statistics to the point where there's any obvious advantage in using one stat or the other.

    IOW, if the difference shown by the analysis of the stats is really a significant one, then you would expect that difference to be reflected in the application of those stats. Otherwise, what makes it a significant difference?
  4. AbeBeta

    AbeBeta Well-Known Member

    25,602 Messages
    1,017 Likes Received
    Again, your "application" of the stats is meaningless. The stats have been applied to prediction of WINS and it shows that one statistic is more predictive than another. That is the application. Your approach is just taking that apple juice and putting it in a Sunny D container.
  5. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    You're obviously very passionate about ANY/A being a substantially better metric. If you can show a significant difference between the two stats, then why don't you?

    Take "Seasons Among Top 10 Rated Passers" for example.
    Peyton Manning (13) 1999-2010, 2012
    Joe Montana (12) 1980-85, 87-90, 93-94
    Fran Tarkenton (11) 1964, 67-70, 72-77
    Dan Marino (11) 1983-87, 90-92, 94-96
    Tom Brady (11) 2001-07, 09-12
    Johnny Unitas (10) 1956-60, 63-67
    Brett Favre (10) 1992, 94-97, 2001, 03, 04, 07, 09
    Sammy Baugh (9) 1937, 40-45, 47, 49
    Y.A. Tittle (8) 1948, 52-54, 57, 61-63
    Len Dawson (8) 1962-68, 71
    Ken Anderson (8) 1973-75, 77, 79, 81-83
    Dan Fouts (8) 1978-85
    Steve Young (8) 1991-98

    Show how "Seasons Among Top 10 in ANY/A" is significantly different from this.

    Or compare "Passer Rating Differential" with "ANY/A Differential" for any season you choose.
  6. AbeBeta

    AbeBeta Well-Known Member

    25,602 Messages
    1,017 Likes Received
    No Percy - I'm passionate about people using statistics correctly and not making claims that they can't substantiate. And again -- you seem to fail to understand, this discussion is about wins. You claimed the two were not different -- the statistics said different. Yet you think that throwing out various rankings of QBs is some sort of evidence that the two statistics do not differentially predict WINS.

    Why is this point so difficult for you to grasp?
  7. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    I'm not talking about the established fact that the two stats differentially predict wins. I acknowledge that fact, OK?

    I'm pointing out that there is no significant difference in the results when you put the two stats to practical use.
  8. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    Theo, they didn't understand the AY/A "TD" term.

    In the simplest sense, it's a disguised percentage, multiplied by 100.

    In a strict definition, the total value to be gained by taking a team from the zero yard line, to score, must equal the value gained by the yardage plus the extra value needed to equal the value of the touchdown. It's a sum.

    When the TD term has a magnitude of 10, it means the odds of scoring the equivalent of a touchdown, in a game where the only possible results are a touchdown or a neutral result, are 90% and there is a 10% chance not to score. Football is more complicated than that, so the real statistical tree of results is more complicated, but the math for the simplified model and the real model has to add up to the same.

    You can also look at it another way. 90% of the value of the touchdown is the yardage, 10% is the act of scoring.

    Likewise, when the TD term is 20, there is an 80% chance to score the equivalent of a touchdown and a 20% chance of no score at all.

    As the TD term increases, the odds of scoring decrease. Larger TD terms are associated with a lesser chance of a score as you get closer and closer to the goal.

    As a consequence, the "true" formula, whatever that is, is highly dependent on the rules and the era in which you play.

    But using real data, you can actually determine that formula.

    There are similar relationships embedded in the NFL passer rating, but they're highly complicated because completions and yardage are interrelated. AY/A formulas are much simpler to understand, and thus misunderstand.
  9. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    Ok, in my last post, when I'm talking about odds to score, that's in the limit as you get infinitesimally close to the goal line.
  10. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    Since there's no significant practical difference between passer rating and the ANY/A that profootballreference uses, have you come across (or come up with) a passer rating or AY/A that correlates more highly with winning?
  11. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    I would disagree with your premise, as AYA stats are easier to understand, easier to derive, easier to relate to real data.

    ~~~

    The best possible stat relating QB play to winning, IMO, is win probability added. I see no need to otherwise reinvent that wheel. There is only one way to calculate win probabilities, and for a given play by play data set, everyone is going to get the same answer.

    It's easy to understand, no weird double counting, etc.

    in passing, I'll note the ESPN passer rating looks to me like WPA x 100 :/

    I'll note that stats related directly to winning may be more narrative than predictive stats, and further, that winning stats get enhanced if you're the QB on a team that sucks and you regularly are called upon to save the day after your team falls routinely behind.

    So you have to ask yourself, is the degree to which your QB contributes to wins really the thing you're looking for? After all, Pythagorean expectations are more predictive of winning in baseball than are the previous season's W-L%.

    So, for certain kinds of analyses, such as showing folks how good your QB is even when your team doesn't win games, or say, make playoffs, AYA and/or NFL passer ratings are better stats than WPA.

    Think about ERA and pitcher W-L records. There are analogies here.

    ~~~

    The NFL passer rating was concocted to crown a "best passer" in the NFL for publicity reasons, nothing else. And for doing that, it is far superior to the older alternatives. That's very old history, well told in the appropriate chapters of "The Hidden Game of Football." Having a best passer and having folks compete for best passer puts money in the NFL's pockets.
  12. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    There's problems with looking at careers as well.

    There's a reason why today's QB's can easily achieve better career QB Ratings than the great QB's of the past. The schemes and offensive philosophies were extremely different.

    I also think in the 90's, the schemes were vastly different as well. You had Zampese style offenses like the Cowboys which threw it deep versus West Coast and Run-n-Shoot offenses.

    Today we see less variation in passing patterns, but it explains why Joe Flacco's QB rating wasn't that high (until they fired Cam Cameron) while systems that use more WR screens and shorter pass patterns can more easily achieve a higher QB rating. It doesn't make them a better QB that Flacco, it's just they are in a system that benefits their QB rating more.

    I don't have a problem with QB rating and I think it's worth looking at. But there are other key performance indicators that are important to look at as well.






    YR
  13. Yakuza Rich

    Yakuza Rich Well-Known Member

    12,347 Messages
    1,391 Likes Received
    Take a look at the metric I created years ago that I call 'QB Rating Differential' (QBRD).

    Simply take the team's offensive QB Rating - team's QB Rating allowed

    For 2012 the correlation coefficient was 0.808605





    YR
  14. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    I should have been more clear. My point is that you get basically the same results no matter which of the two stats you use -- that there's no practical difference between the 2012 team rankings in passer rating and ANY/A, for example. I'm not talking about there being no practical difference in the ease of doing the calculations. Most of us not being mathematicians, the simplicity of the formula isn't a factor.

    I use WPA a lot. Witten led all TE in that stat in 2012, and no QB has more top 5's in WPA than Romo during his career.

    The "QB on a team that sucks" phenomenon is easy to check. Since 2000, only 17 of the 130 QB who have made the top 10 in WPA played on losing teams. 16 of the 130 QB who made the top 10 in passer rating played on losing teams. That's a huge sample, and it doesn't show that WPA has catapulted QB on bad teams into the top 10 any more than passer rating would have.

    Interesting that you make a connection between WPA and ESPN's rating, since Romo does so well by WPA, and so poorly by Total QBR.

    I think the ideal is to have a stat measures passing performance and correlates highly to winning percentage. I've seen correlation coefficients for passer rating differential at 80.0 and WPA/game at .75, so I'd conclude that passer rating must be very good at doing both.
  15. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    Talking about practical differences between AYA and the NFL passer rating.. usually it's pretty easy to reconstruct a scoring model for which the AYA function fits.

    In the case of the PFR AYA model, you have one with these properties.

    1. The value of the model at 0 yards is -2 points (you score a safety for the other team at 0).
    2. The value of a touchdown is 7 points.
    3. The slope of the points per yard curve is 0.075 points per yard.
    4. At 100 yards, the value of the yards is 5.5 points, 1.5 point less than the touchdown value. The difference, 1.5 points, translates into 1.5 points/(0.075 (pts/yd)) = 20 yards. That difference is where the TD term comes from. Not that hard to understand.

    So what does this mean? It means that if I multiply 0.075 by 30 attempts, yielding the number 2.25, then that's the conversion from AYA to expected points per 30 attempts.

    There is a direct and easy to find correlation between scoring and yards in a AYA stat, and the stats could just as easily be turned into adjusted points per attempt.

    In other words, it's a scoring stat, dealing in expected points, but using a metric that hides the direct relationship with EP.

    So what's the practical difference here? Try doing that with the NFL passer rating :).
    AbeBeta likes this.
  16. dwmyers

    dwmyers Well-Known Member

    2,049 Messages
    38 Likes Received
    You're missing the point. Then win probability of a team starts at 0.5. If the team's chances to win gradually get better over time, then the maximum net gain is 0.5 wins. So, a team like the 1970s Steelers didn't help Terry Bradshaw's WPA.

    If you're Eli Manning, say, and your team is riding entirely on your shoulders and your late game heroics, then typically late in the game your team is behind by a touchdown or two. You have to make scoring drives at the end of the game to win it. But because your teams chances of winning, by falling behind, may be something like 0.05 or 0.1, you have 0.9 wins to gain in those long drives.

    So, good teams that don't need their QB to win for them will have lower QB WPAs than sloppy teams that routinely require late game QB heroics for the team to win. This is true regardless of the actual skill of the QB, as measured by AYA or NFL passer rating or whatever scoring rate metric you desire to use.

    D-
  17. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    That's true, but I wasn't referring to cross-era comparison, where there are a lot of factors to consider besides scheme anyway. I was talking about comparing careers in the same era -- about the rarity of any two teams having vastly different strength of opponent after an accumulation of 10 seasons, for example.

    Scheme does play a role, but that tends to get overstated. Take the case of Ken Anderson winning four passing titles and not being in the Hall of Fame.
  18. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    No, I got it the first time. Teams that suck but that have good QB will win more games than teams that suck but don't have good QB, true. But teams that suck will still lose more games than teams that don't, no matter how good the QB is.
  19. percyhoward

    percyhoward Research Tool

    8,648 Messages
    2,754 Likes Received
    By "QB Rating Differential" do you mean something other than passer rating differential? It was .80 for 2011.
  20. Apollo Creed

    Apollo Creed Stackin and Processin, Well

    8,610 Messages
    488 Likes Received
    I must be lost. Way too many numbers and facts in this thread - not enough sensationalism and anti-Romo rhetoric.

    Great work as always Percy.

Share This Page