How Passer Rating Works

[85Cowboy85]

There are no factors "purely under the quarterbacks [sic] control."

As purely as possible anyway that's the idea. As of now the closest we have are the components of passer rating.

If you could further isolate the quarterback you can improve the statistic which is the point I was making earlier. If you judge by wins and losses you add statistical noise and further muddy the waters.

 

[AbeBeta]

As purely as possible anyway that's the idea. As of now the closest we have are the components of passer rating.

If you could further isolate the quarterback you can improve the statistic which is the point I was making earlier. If you judge by wins and losses you add statistical noise and further muddy the waters.

How on earth can you claim that is correct. How is an int all on the QB? How is a TD? How is a sack not part of that? How is a fumble by the QB not part of it?

If you judge by wins and losses you add in an actual outcome that matters. There is no point to ranking performance if you are uninterested in how it relates to wins.

 

[percyhoward]

You are saying that ANY/A and PR both predict offense rank similarly. That tells us nothing about the winning % which is the analysis you told us showed no difference.

I'm not saying that the analysis shows no difference. I'm saying that, when the stats are put into practical use, that difference doesn't significantly affect the resulting set of statistics to the point where there's any obvious advantage in using one stat or the other.

IOW, if the difference shown by the analysis of the stats is really a significant one, then you would expect that difference to be reflected in the application of those stats. Otherwise, what makes it a significant difference?

 

[AbeBeta]

I'm not saying that the analysis shows no difference. I'm saying that, when the stats are put into practical use, that difference doesn't significantly affect the resulting set of statistics to the point where there's any obvious advantage in using one stat or the other.

IOW, if the difference shown by the analysis of the stats is really a significant one, then you would expect that difference to be reflected in the application of those stats. Otherwise, what makes it a significant difference?

Again, your "application" of the stats is meaningless. The stats have been applied to prediction of WINS and it shows that one statistic is more predictive than another. That is the application. Your approach is just taking that apple juice and putting it in a Sunny D container.

 

[percyhoward]

Again, your "application" of the stats is meaningless.

You're obviously very passionate about ANY/A being a substantially better metric. If you can show a significant difference between the two stats, then why don't you?

Take "Seasons Among Top 10 Rated Passers" for example.
Peyton Manning (13) 1999-2010, 2012
Joe Montana (12) 1980-85, 87-90, 93-94
Fran Tarkenton (11) 1964, 67-70, 72-77
Dan Marino (11) 1983-87, 90-92, 94-96
Tom Brady (11) 2001-07, 09-12
Johnny Unitas (10) 1956-60, 63-67
Brett Favre (10) 1992, 94-97, 2001, 03, 04, 07, 09
Sammy Baugh (9) 1937, 40-45, 47, 49
Y.A. Tittle (8) 1948, 52-54, 57, 61-63
Len Dawson (8) 1962-68, 71
Ken Anderson (8) 1973-75, 77, 79, 81-83
Dan Fouts (8) 1978-85
Steve Young (8) 1991-98

Show how "Seasons Among Top 10 in ANY/A" is significantly different from this.

Or compare "Passer Rating Differential" with "ANY/A Differential" for any season you choose.

 

[AbeBeta]

You're obviously very passionate about ANY/A being a substantially better metric. If you can show a significant difference between the two stats, then why don't you?

Take "Seasons Among Top 10 Rated Passers" for example.
Peyton Manning (13) 1999-2010, 2012
Joe Montana (12) 1980-85, 87-90, 93-94
Fran Tarkenton (11) 1964, 67-70, 72-77
Dan Marino (11) 1983-87, 90-92, 94-96
Tom Brady (11) 2001-07, 09-12
Johnny Unitas (10) 1956-60, 63-67
Brett Favre (10) 1992, 94-97, 2001, 03, 04, 07, 09
Sammy Baugh (9) 1937, 40-45, 47, 49
Y.A. Tittle (8) 1948, 52-54, 57, 61-63
Len Dawson (8) 1962-68, 71
Ken Anderson (8) 1973-75, 77, 79, 81-83
Dan Fouts (8) 1978-85
Steve Young (8) 1991-98

Show how "Seasons Among Top 10 in ANY/A" is significantly different from this.

Or compare "Passer Rating Differential" with "ANY/A Differential" for any season you choose.

No Percy - I'm passionate about people using statistics correctly and not making claims that they can't substantiate. And again -- you seem to fail to understand, this discussion is about wins. You claimed the two were not different -- the statistics said different. Yet you think that throwing out various rankings of QBs is some sort of evidence that the two statistics do not differentially predict WINS.

Why is this point so difficult for you to grasp?

 

[percyhoward]

No Percy - I'm passionate about people using statistics correctly and not making claims that they can't substantiate. And again -- you seem to fail to understand, this discussion is about wins. You claimed the two were not different -- the statistics said different. Yet you think that throwing out various rankings of QBs is some sort of evidence that the two statistics do not differentially predict WINS.

Why is this point so difficult for you to grasp?

I'm not talking about the established fact that the two stats differentially predict wins. I acknowledge that fact, OK?

I'm pointing out that there is no significant difference in the results when you put the two stats to practical use.

 

[dwmyers]

This is precisely the problem with this formula. When it was initially created, someone set the multipliers arbitrarily -- most likely to get the formula to spit out whatever they thought was the "right" result. Then someone else came along and said "no, I like this result better, so I'll change the multiplier". Instead, each variable should have a multiplier so that the variables are directly comparable, not because someone decides what the result should be.

Theo, they didn't understand the AY/A "TD" term.

In the simplest sense, it's a disguised percentage, multiplied by 100.

In a strict definition, the total value to be gained by taking a team from the zero yard line, to score, must equal the value gained by the yardage plus the extra value needed to equal the value of the touchdown. It's a sum.

When the TD term has a magnitude of 10, it means the odds of scoring the equivalent of a touchdown, in a game where the only possible results are a touchdown or a neutral result, are 90% and there is a 10% chance not to score. Football is more complicated than that, so the real statistical tree of results is more complicated, but the math for the simplified model and the real model has to add up to the same.

You can also look at it another way. 90% of the value of the touchdown is the yardage, 10% is the act of scoring.

Likewise, when the TD term is 20, there is an 80% chance to score the equivalent of a touchdown and a 20% chance of no score at all.

As the TD term increases, the odds of scoring decrease. Larger TD terms are associated with a lesser chance of a score as you get closer and closer to the goal.

As a consequence, the "true" formula, whatever that is, is highly dependent on the rules and the era in which you play.

But using real data, you can actually determine that formula.

There are similar relationships embedded in the NFL passer rating, but they're highly complicated because completions and yardage are interrelated. AY/A formulas are much simpler to understand, and thus misunderstand.

 

[dwmyers]

Ok, in my last post, when I'm talking about odds to score, that's in the limit as you get infinitesimally close to the goal line.

 

[percyhoward]

There are similar relationships embedded in the NFL passer rating, but they're highly complicated because completions and yardage are interrelated. AY/A formulas are much simpler to understand, and thus misunderstand.

Since there's no significant practical difference between passer rating and the ANY/A that profootballreference uses, have you come across (or come up with) a passer rating or AY/A that correlates more highly with winning?

 

[dwmyers]

Since there's no significant practical difference between passer rating and the ANY/A that profootballreference uses, have you come across (or come up with) a passer rating or AY/A that correlates more highly with winning?

I would disagree with your premise, as AYA stats are easier to understand, easier to derive, easier to relate to real data.

~~~

The best possible stat relating QB play to winning, IMO, is win probability added. I see no need to otherwise reinvent that wheel. There is only one way to calculate win probabilities, and for a given play by play data set, everyone is going to get the same answer.

It's easy to understand, no weird double counting, etc.

in passing, I'll note the ESPN passer rating looks to me like WPA x 100 :/

I'll note that stats related directly to winning may be more narrative than predictive stats, and further, that winning stats get enhanced if you're the QB on a team that sucks and you regularly are called upon to save the day after your team falls routinely behind.

So you have to ask yourself, is the degree to which your QB contributes to wins really the thing you're looking for? After all, Pythagorean expectations are more predictive of winning in baseball than are the previous season's W-L%.

So, for certain kinds of analyses, such as showing folks how good your QB is even when your team doesn't win games, or say, make playoffs, AYA and/or NFL passer ratings are better stats than WPA.

Think about ERA and pitcher W-L records. There are analogies here.

~~~

The NFL passer rating was concocted to crown a "best passer" in the NFL for publicity reasons, nothing else. And for doing that, it is far superior to the older alternatives. That's very old history, well told in the appropriate chapters of "The Hidden Game of Football." Having a best passer and having folks compete for best passer puts money in the NFL's pockets.

 

[Yakuza Rich]

Ah, OK, you're saying don't put a lot of faith in one season's worth of games. Yeah, it's significant that the skewing you're talking about doesn't seem to withstand larger sample sizes. When you look at the list of QB who have finished in the top 10 in passer rating six times or more, you don't see any Chris Miller's or Scott Mitchell's.

There's problems with looking at careers as well.

There's a reason why today's QB's can easily achieve better career QB Ratings than the great QB's of the past. The schemes and offensive philosophies were extremely different.

I also think in the 90's, the schemes were vastly different as well. You had Zampese style offenses like the Cowboys which threw it deep versus West Coast and Run-n-Shoot offenses.

Today we see less variation in passing patterns, but it explains why Joe Flacco's QB rating wasn't that high (until they fired Cam Cameron) while systems that use more WR screens and shorter pass patterns can more easily achieve a higher QB rating. It doesn't make them a better QB that Flacco, it's just they are in a system that benefits their QB rating more.

I don't have a problem with QB rating and I think it's worth looking at. But there are other key performance indicators that are important to look at as well.






YR

 

[Yakuza Rich]

Since there's no significant practical difference between passer rating and the ANY/A that profootballreference uses, have you come across (or come up with) a passer rating or AY/A that correlates more highly with winning?

Take a look at the metric I created years ago that I call 'QB Rating Differential' (QBRD).

Simply take the team's offensive QB Rating - team's QB Rating allowed

For 2012 the correlation coefficient was 0.808605





YR

 

[percyhoward]

I would disagree with your premise, as AYA stats are easier to understand, easier to derive, easier to relate to real data.

I should have been more clear. My point is that you get basically the same results no matter which of the two stats you use -- that there's no practical difference between the 2012 team rankings in passer rating and ANY/A, for example. I'm not talking about there being no practical difference in the ease of doing the calculations. Most of us not being mathematicians, the simplicity of the formula isn't a factor.

The best possible stat relating QB play to winning, IMO, is win probability added...in passing, I'll note the ESPN passer rating looks to me like WPA x 100 :/ ..I'll note that stats related directly to winning may be more narrative than predictive stats, and further, that winning stats get enhanced if you're the QB on a team that sucks and you regularly are called upon to save the day after your team falls routinely behind.

I use WPA a lot. Witten led all TE in that stat in 2012, and no QB has more top 5's in WPA than Romo during his career.

The "QB on a team that sucks" phenomenon is easy to check. Since 2000, only 17 of the 130 QB who have made the top 10 in WPA played on losing teams. 16 of the 130 QB who made the top 10 in passer rating played on losing teams. That's a huge sample, and it doesn't show that WPA has catapulted QB on bad teams into the top 10 any more than passer rating would have.

Interesting that you make a connection between WPA and ESPN's rating, since Romo does so well by WPA, and so poorly by Total QBR.

The NFL passer rating was concocted to crown a "best passer" in the NFL for publicity reasons, nothing else. And for doing that, it is far superior to the older alternatives. That's very old history, well told in the appropriate chapters of "The Hidden Game of Football." Having a best passer and having folks compete for best passer puts money in the NFL's pockets.

I think the ideal is to have a stat measures passing performance and correlates highly to winning percentage. I've seen correlation coefficients for passer rating differential at 80.0 and WPA/game at .75, so I'd conclude that passer rating must be very good at doing both.

 

[dwmyers]

Talking about practical differences between AYA and the NFL passer rating.. usually it's pretty easy to reconstruct a scoring model for which the AYA function fits.

In the case of the PFR AYA model, you have one with these properties.

1. The value of the model at 0 yards is -2 points (you score a safety for the other team at 0).
2. The value of a touchdown is 7 points.
3. The slope of the points per yard curve is 0.075 points per yard.
4. At 100 yards, the value of the yards is 5.5 points, 1.5 point less than the touchdown value. The difference, 1.5 points, translates into 1.5 points/(0.075 (pts/yd)) = 20 yards. That difference is where the TD term comes from. Not that hard to understand.

So what does this mean? It means that if I multiply 0.075 by 30 attempts, yielding the number 2.25, then that's the conversion from AYA to expected points per 30 attempts.

There is a direct and easy to find correlation between scoring and yards in a AYA stat, and the stats could just as easily be turned into adjusted points per attempt.

In other words, it's a scoring stat, dealing in expected points, but using a metric that hides the direct relationship with EP.

So what's the practical difference here? Try doing that with the NFL passer rating .

 

[dwmyers]

The "QB on a team that sucks" phenomenon is easy to check. Since 2000, only 17 of the 130 QB who have made the top 10 in WPA played on losing teams. 16 of the 130 QB who made the top 10 in passer rating played on losing teams. That's a huge sample, and it doesn't show that WPA has catapulted QB on bad teams into the top 10 any more than passer rating would have.

You're missing the point. Then win probability of a team starts at 0.5. If the team's chances to win gradually get better over time, then the maximum net gain is 0.5 wins. So, a team like the 1970s Steelers didn't help Terry Bradshaw's WPA.

If you're Eli Manning, say, and your team is riding entirely on your shoulders and your late game heroics, then typically late in the game your team is behind by a touchdown or two. You have to make scoring drives at the end of the game to win it. But because your teams chances of winning, by falling behind, may be something like 0.05 or 0.1, you have 0.9 wins to gain in those long drives.

So, good teams that don't need their QB to win for them will have lower QB WPAs than sloppy teams that routinely require late game QB heroics for the team to win. This is true regardless of the actual skill of the QB, as measured by AYA or NFL passer rating or whatever scoring rate metric you desire to use.

D-

 

[percyhoward]

There's problems with looking at careers as well. There's a reason why today's QB's can easily achieve better career QB Ratings than the great QB's of the past. The schemes and offensive philosophies were extremely different.

That's true, but I wasn't referring to cross-era comparison, where there are a lot of factors to consider besides scheme anyway. I was talking about comparing careers in the same era -- about the rarity of any two teams having vastly different strength of opponent after an accumulation of 10 seasons, for example.

Scheme does play a role, but that tends to get overstated. Take the case of Ken Anderson winning four passing titles and not being in the Hall of Fame.

 

[percyhoward]

You're missing the point.

No, I got it the first time. Teams that suck but that have good QB will win more games than teams that suck but don't have good QB, true. But teams that suck will still lose more games than teams that don't, no matter how good the QB is.

 

[percyhoward]

Take a look at the metric I created years ago that I call 'QB Rating Differential' (QBRD).

Simply take the team's offensive QB Rating - team's QB Rating allowed

For 2012 the correlation coefficient was 0.808605

By "QB Rating Differential" do you mean something other than passer rating differential? It was .80 for 2011.

 

[Apollo Creed]

I must be lost. Way too many numbers and facts in this thread - not enough sensationalism and anti-Romo rhetoric.

Great work as always Percy.