QBR Ratings

[BlindFaith]

Similar how? I am a trained engineer that knows how to normalize data sets. I am telling you that adding a nonlinear element for aesthetics is a big no no.

That is not nitpicking. That is bull****. If you want to continue to find it meaningful despite knowing that it is nonlinear is 100% you and goes to show that you are more interested in confirmation of a conclusion then letting the chips fall where they may.

Similar how? I am a trained engineer that knows how to normalize data sets. I am telling you that adding a nonlinear element for aesthetics is a big no no.

That is not nitpicking. That is bull****. If you want to continue to find it meaningful despite knowing that it is nonlinear is 100% you and goes to show that you are more interested in confirmation of a conclusion then letting the chips fall where they may.

Give me a minute to Google nonlinear and normalized data sets. Ah, forget it. I'll just go with - the guys who came up with the algorithm probably didn't have to Google it.

And how can you, being an engineer and all, even begin to denounce this without even knowing exactly how it works? Do you have the exact algorithm they use? Do you know how normalized it really is?

 

[BlindFaith]

You will notice that in a sport like baseball, one almost driven fanatically by numbers and algorithms, the idea of "clutch" has been thoroughly discarded in key stats like WAR.

Only a media company would somehow rate TD's in the last minute of the 4th quarter as more meaningful than TD's in the first quarter, because only someone who wants to drive ratings and story bylines would try to create a metric around "clutch."

They use Probability of Winning to weight stats. Just means they don't give as much credit for TDs and Passing yards when its very probable that the defense is playing soft, trying to run out a game or has back ups in. If the score then becomes close, they get more credit. Without knowing exactly how much the weighting impacts it, we are only guessing.

Baseball has nothing to compare to this. The game is over when the final out is made. Doesn't matter how long it takes. And the defense doesn't adjust how it plays. The pitching doesn't change. Nothing changes about the game depending on the score.

 

[FuzzyLumpkins]

QBR takes something similar to quarterback rating passing metrics plus runs. What this is exactly is anyone's guess. Blind faith helps.

If that was it then that would be one thing but there is more. They then take the individual play valuations that total into the above metric and normalize it based on game situations weighted by win probabilities. They did not double down on this under scrutiny. Instead of getting rid of it though, they instead capped it.

That is bad enough but there is more! The output from the above valuations did not have a pretty range if they based it on wins it was probably rather low numbers as an efficiency stat based on wins. .114 to .412 is unacceptable. People will see .300 and think it is a good score so we need to change .114 to .412 into 1 to 100. Well because the data distribution is not linear you get to do fun stuff to change scale.

People are ignorant and will like it better this way. Much better that then have to try to explain to them that .300 is a bad score.

 

[percyhoward]

I haven't seen much discussion on this. Maybe I missed it. But there seems to be a lot of merit to these rankings.

Stats are judged by their correlation to wins. The individual stats with the highest win correlations are "traditional" passer rating and adjusted net yards per attempt. They are basically the same stat, except that the second one incorporates sacks and sack yardage.

ESPN admits that they were never in search of a stat that correlates to winning.

http://espn.go.com/nfl/story/_/id/6909058/nfl-total-qbr-faq#faq9
"QBR is meant to correlate to offense, moving the ball downfield, turning good field position into points, avoiding giving it back to the defense. Good offensive performance is not the same as winning. To the degree that offense correlates to winning, QBR should be helpful."

IOW, there may be other things that a QB does that lead to wins that aren't included in ESPN's stat. And you can be sure that there are things that a QB does that don't necessarily lead to wins that are included, just because ESPN wanted it that way.

 

[FuzzyLumpkins]

Give me a minute to Google nonlinear and normalized data sets. Ah, forget it. I'll just go with - the guys who came up with the algorithm probably didn't have to Google it.

And how can you, being an engineer and all, even begin to denounce this without even knowing exactly how it works? Do you have the exact algorithm they use? Do you know how normalized it really is?

you live up to your handle. I know what changing scale entails. I write filters that are akin to what they do in weighing values.

the cutesy marketing change of scale is nothing new. weighing stats based on wins is nothing new. QBR guys are not breaking new ground. they just like feeding increasingly complex signals back into themselves.

 

[BlindFaith]

You should also read your article



I would also point out that doing garbage like this so you can sell stats that look pretty is complete and utter garbage and makes trying to compare apples to apples a fools errand.



EQA did the same thing and it is why it was rejected. Have to love adding a nonlinear scaling so it looks pretty to people that don't know any better.

Football Outsiders did basically the same thing called DYAR with the following differences:

• Total QBR incorporates information from game charting, such as passes dropped or thrown away on purpose.
• Total QBR splits responsibility on plays between the quarterback, his receivers, and his blockers. Drops, for example, are more on the receiver, as are yards after the catch, and some sacks are more on the offensive line than others.
• Total QBR has a clutch factor which adds (or subtracts) value for quarterbacks who perform best (or worst) in high-leverage situations.
• Total QBR combines passing and rushing value into one number and differentiates between scrambles and planned runs.
• Total QBR is not adjusted for strength of opponent.

And Matt Ryan is still ahead of Romo, this time much higher.

PASSING: Minimum 100 passes, 45 players ranked
PlayerTeamDYARRkYARRkDVOARkVOAPassesYardsEYdsTDFKFLINTC%DPI
P.ManningDEN2,47512,674143.2%147.5%6795,3477,04055351068.5%12/176
P.RiversSD1,79921,884234.8%337.0%5754,3635,59932121169.6%13/197
D.BreesNO1,70131,550326.9%523.6%6874,9065,91639321268.9%4/66
M.RyanATL1,1244825813.3%96.8%6954,1955,22826041667.7%10/151
N.FolesPHI1,01151,111435.6%240.2%3472,6903,1172722264.4%2/55
T.BradyNE9796859610.9%118.2%6654,0654,91325421061.0%13/223
T.RomoDAL8397898511.5%1013.1%5723,5564,1203131964.2%7/81
C.KaepernickSF79186501316.6%711.7%4562,9693,3582113758.4%5/92
R.WilsonSEA77096991215.6%813.1%4533,0703,3642655864.0%6/142
A.RodgersGB740107621125.4%626.4%3102,4132,6441830666.8%2/37
B.RoethlisbergerPIT72411764106.6%127.6%6213,9814,26728151464.9%4/97
M.StaffordDET6901283874.9%158.3%6584,4314,40229341959.1%6/64
J.McCownCHI65913773932.1%439.6%2361,7862,1241311166.7%1/19
A.LuckIND65014623144.6%163.9%6063,6124,2132332960.5%13/238

So I guess this one is flawed too?

 

[BlindFaith]

you live up to your handle. I know what changing scale entails. I write filters that are akin to what they do in weighing values.

the cutesy marketing change of scale is nothing new. weighing stats based on wins is nothing new. QBR guys are not breaking new ground. they just like feeding increasingly complex signals back into themselves.

So now they are basing their formula on wins? Or did you mean probability of winning?

 

[BlindFaith]

Stats are judged by their correlation to wins. The individual stats with the highest win correlations are "traditional" passer rating and adjusted net yards per attempt. They are basically the same stat, except that the second one incorporates sacks and sack yardage.

ESPN admits that they were never in search of a stat that correlates to winning.

http://espn.go.com/nfl/story/_/id/6909058/nfl-total-qbr-faq#faq9
"QBR is meant to correlate to offense, moving the ball downfield, turning good field position into points, avoiding giving it back to the defense. Good offensive performance is not the same as winning. To the degree that offense correlates to winning, QBR should be helpful."

IOW, there may be other things that a QB does that lead to wins that aren't included in ESPN's stat. And you can be sure that there are things that a QB does that don't necessarily lead to wins that are included, just because ESPN wanted it that way.

"How QBR "relates to winning" is a question that can be interpreted various ways. One way that we addressed this on TV was that the team winning the QBR battle within a game wins the game 86% of the time. Teams that win the turnover battle don't win this often. Teams that win the NFL Passer Rating battle don't win this often. This result is primarily emphasizing that QBR is capturing team results and that quarterbacks performance is quite important to that."

 

[FuzzyLumpkins]

Football Outsiders did basically the same thing called DYAR with the following differences:

• Total QBR incorporates information from game charting, such as passes dropped or thrown away on purpose.
• Total QBR splits responsibility on plays between the quarterback, his receivers, and his blockers. Drops, for example, are more on the receiver, as are yards after the catch, and some sacks are more on the offensive line than others.
• Total QBR has a clutch factor which adds (or subtracts) value for quarterbacks who perform best (or worst) in high-leverage situations.
• Total QBR combines passing and rushing value into one number and differentiates between scrambles and planned runs.
• Total QBR is not adjusted for strength of opponent.

And Matt Ryan is still ahead of Romo, this time much higher.

PASSING: Minimum 100 passes, 45 players ranked
PlayerTeamDYARRkYARRkDVOARkVOAPassesYardsEYdsTDFKFLINTC%DPI
P.ManningDEN2,47512,674143.2%147.5%6795,3477,04055351068.5%12/176
P.RiversSD1,79921,884234.8%337.0%5754,3635,59932121169.6%13/197
D.BreesNO1,70131,550326.9%523.6%6874,9065,91639321268.9%4/66
M.RyanATL1,1244825813.3%96.8%6954,1955,22826041667.7%10/151
N.FolesPHI1,01151,111435.6%240.2%3472,6903,1172722264.4%2/55
T.BradyNE9796859610.9%118.2%6654,0654,91325421061.0%13/223
T.RomoDAL8397898511.5%1013.1%5723,5564,1203131964.2%7/81
C.KaepernickSF79186501316.6%711.7%4562,9693,3582113758.4%5/92
R.WilsonSEA77096991215.6%813.1%4533,0703,3642655864.0%6/142
A.RodgersGB740107621125.4%626.4%3102,4132,6441830666.8%2/37
B.RoethlisbergerPIT72411764106.6%127.6%6213,9814,26728151464.9%4/97
M.StaffordDET6901283874.9%158.3%6584,4314,40229341959.1%6/64
J.McCownCHI65913773932.1%439.6%2361,7862,1241311166.7%1/19
A.LuckIND65014623144.6%163.9%6063,6124,2132332960.5%13/238

So I guess this one is flawed too?

Oh, I get it. You are emotionally invested in making this about how Romo is ranked. You will take whatever you can find that says that.

If you read the bolded portion and still think that they are the same or even remotely similar then you should read it again. I will say that I am disappointed that FO went with clutch but it is what it is. They are the types that would actually try to justify it though. I will go see.

 

[percyhoward]

"How QBR "relates to winning" is a question that can be interpreted various ways. One way that we addressed this on TV was that the team winning the QBR battle within a game wins the game 86% of the time. Teams that win the turnover battle don't win this often. Teams that win the NFL Passer Rating battle don't win this often. This result is primarily emphasizing that QBR is capturing team results and that quarterbacks performance is quite important to that."

What were the 2013 rankings in the category of defensive QBR?

 

[BlindFaith]

QBR takes something similar to quarterback rating passing metrics plus runs. What this is exactly is anyone's guess. Blind faith helps.

If that was it then that would be one thing but there is more. They then take the individual play valuations that total into the above metric and normalize it based on game situations weighted by win probabilities. They did not double down on this under scrutiny. Instead of getting rid of it though, they instead capped it.

That is bad enough but there is more! The output from the above valuations did not have a pretty range if they based it on wins it was probably rather low numbers as an efficiency stat based on wins. .114 to .412 is unacceptable. People will see .300 and think it is a good score so we need to change .114 to .412 into 1 to 100. Well because the data distribution is not linear you get to do fun stuff to change scale.

People are ignorant and will like it better this way. Much better that then have to try to explain to them that .300 is a bad score.

I never said that QBR was an indication strictly of how good of a passer the QB is. But rather an overall indicator to how valuable they are to the team.

So what's your biggest beef with the 2013 rankings? Because Romo isn't higher?

 

[Muhast]

The system has Kaepernick above Brady. No. Not last year. Not ever.

 

[BlindFaith]

Oh, I get it. You are emotionally invested in making this about how Romo is ranked. You will take whatever you can find that says that.

If you read the bolded portion and still think that they are the same or even remotely similar then you should read it again. I will say that I am disappointed that FO went with clutch but it is what it is. They are the types that would actually try to justify it though. I will go see.

I hardly get emotional over football anymore. I just think this is a good point to discuss as it relates to why there is no consensus on how good Romo really is. I think some judge him solely on the numbers he puts up. Others look at game situations and come up with a different conclusion. This analysis highlights a different view of how impactful a QB is. And yes, it does support my views on Romo. But its not just an arbitrary opinion. Its based on formulas that look at the game in a different way.

Is it perfect? No. But in a true team sport like football its very difficult to assign success or failure based only on stats. Its not like baseball where stats are purely of an individual basis that can be clearly seen and recorded.

 

[BlindFaith]

What were the 2013 rankings in the category of defensive QBR?

Doesn't exist. Why?

 

[BlindFaith]

The system has Kaepernick above Brady. No. Not last year. Not ever.

It includes rushes. It's to show how impactful a QB is overall.

 

[percyhoward]

Doesn't exist. Why?

You could use QBR differential (offensive QBR - defensive QBR) to see how the stat correlates to wins.

 

[Afigueroa22]

You could use QBR differential (offensive QBR - defensive QBR) to see how the stat correlates to wins.

I"d like to see this.

 

[BlindFaith]

You could use QBR differential (offensive QBR - defensive QBR) to see how the stat correlates to wins.

86% of the time, according to ESPN, the QB with the higher QBR in a game wins the game.

It would take some time to list the QBR of both teams and see if the QBR of the winning QB was actually higher.

 

[BlindFaith]

You could use QBR differential (offensive QBR - defensive QBR) to see how the stat correlates to wins.

Its actually pretty interesting. On the site you can look week by week.

Week one Carson Palmer had the 7th highest QBR and lost to the Rams.

Palmer - QBR=79.3 - Passer Rating=96.6
Bradford - QBR=50.4 - Passer Rating=100.7

Bradford had the higher Passer Rating but his QBR was much lower. Looking at the Passer Rating only you would have thought he had a good game. But he threw an interception from his own 7 that was returned for a TD. Jared Cook also had 141 yards receiving with much of that yardage being RAC. Marginal production from the rest of the WRs.

Palmer spread the ball out more. Less RAC. Higher QBR.

The Rams did manage to win the game. A close game. 24-27. The Rams D sacked Palmer 4 times. So the D helped out Bradford to pull out the win.

It would take forever, but it would be interesting to do a complete comparison for every game every week like this.

 

[wileedog]

They use Probability of Winning to weight stats. Just means they don't give as much credit for TDs and Passing yards when its very probable that the defense is playing soft, trying to run out a game or has back ups in. If the score then becomes close, they get more credit. Without knowing exactly how much the weighting impacts it, we are only guessing.

Baseball has nothing to compare to this. The game is over when the final out is made. Doesn't matter how long it takes. And the defense doesn't adjust how it plays. The pitching doesn't change. Nothing changes about the game depending on the score.

The problem with football stats is and always will be sample size. Baseball stats are more meaningful because they have such a huge range of data. Football has only 16 games a year when a given player will only be on the field half the time.

But when you look at something like this QBR rating for 1 season and Josh McCown is better than Payton Manning, even though, and precisely because, he only played 8 games and happened to play way above his talent for a few games, you know its all pretty much bunk.

Oh, and by the way, crappy relievers are sent into blowout baseball games too. Then there's September games when teams are out of it and playing AAA prospects etc too. Its not like there isn't "garbage time" stats in baseball.