Benny Friedman NPR, and a word on statistical completeness
Posted: Thu Nov 18, 2021 4:41 pm
I recently picked up my copy of Rupert Patrick's excellent A Statistical History of Pro Football. Being quite a fan of his Normalized Passer Rating system, and since his work begins with the 1932 NFL season, I decided to calculate it for pro football's first undisputed GOAT, Benny Friedman, for each season of his career and his career as a whole. All this only took me a day to do, but I hope it is sufficiently comprehensive; get ready for a lot of numbers and talk about what they mean. I presume this information will prove useful to the early NFL enthusiasts here.
For those who don't have Rupert's book, first of all, you should, and second, Normalized Passer Rating (NPR) is an era-adjusted metric that modifies the classic passer rating formula to compare QBs against league averages in specific years, rather than a constant baseline which has become obsolete. Therefore, NPR balances QBs from all eras relatively equally, instead of heavily favoring modern passers. It does tend to favor great QBs of earlier eras, and I'll explain why shortly; for this reason, I came in expecting Friedman to land at number one all-time.
Stats used to calculate these ratings were obtained from jt-sw.com. They are to be considered incomplete for the majority of these seasons, but I will address this issue extensively in this post.
1927: 101.6
1928: 106.71
1929: 143.98
1930: 137.85
1931: 123.3
1932: 60.32
1933: 123.27
Career: 121.95
All of the figures used for calculation can be seen on this spreadsheet:
https://docs.google.com/spreadsheets/d/ ... p=drivesdk
So, according to NPR, this makes Benny Friedman by far the most efficient quarterback of all time relative to his contemporaries. He's actually the only passer rated over 100 in this system; the #1 career NPR since 1932 is Otto Graham (96.1), or Sid Luckman (94.4) if you prefer to isolate Graham's NFL stats. He is so far ahead that his entire career would be the third-highest rated season in history, behind 1943 and 1941 Luckman. On that note, Friedman's best two seasons replace Luckman's best two on top of that leaderboard, although by not as enormous a margin as he leads the career ranking. 1932 was his only season below 100 (I didn't bother doing 1934 separately because he barely played, but it's in his career stats), but I was definitely surprised to see 1933 rate above 1927 and 1928.
Those who understand how era-adjusted passing metrics work will likely not be altogether shocked by much of this, because speak of the devil, the Luckman Effect is, of course, very much operative in all of it. That is, in earlier and smaller eras of the NFL, years with one or two excellent QBs and otherwise mostly terrible ones were frequent, and the abysmal league averages in those years tend to artificially inflate the NPR of anyone who's any good at all. Friedman, even more than Luckman, was a very big fish in a league full of very small fish at QB, heavily inflating the distance between him and the league average. Luckman and Baugh at least had each other in the league to keep the average somewhat afloat, but Friedman was truly the only game in town, and the average without him was utterly destitute. So that is one reason why it is hard to take NPR at face value in this instance. I expected to see Friedman land at a comfortable #1 for this very reason, but I have to admit the magnitude of the result did surprise me. I can't say it changes my perspective on him, though, given that I already considered him a legend.
It is also unclear how much of Friedman's recorded stats are incomplete, which obviously complicates any attempt to calculate ratio and efficiency stats for such players. However, we can estimate this factor by comparing the leaguewide stats in the unofficial years with those of the official years, and interpolating the difference to just Friedman. So I decided to compare the total numbers for 1927-31 to those in 1932-34, to estimate how much is missing from the former. Per the method for deriving NPR, Friedman's output was removed originally from the league stats for 1927-34, so he has been added back in for this exercise.
However, there is a clear problem here: teams often played different amounts of games during this era of football, and there were also different numbers of teams in the league each year. So I painstakingly counted up every individual game played between two NFL teams for these years, and multiplied the result by two to get an accurate count of "team-games" played per season; the figures are listed in the spreadsheet, under the games played column. With this, we can estimate how incomplete 1927-31 stats are, by noting how much lower the per-game passing stats are for those years compared to 1932-34, assuming per-game passing volume did not change notably over this span. Here is what I got:
Completions: 4.18 C/G 1927-31 (662 games), 4.35 C/G 1932-34 (330 games). Therefore, about 0.17 completions per game, or around 110 total leaguewide, are missing from the record. Friedman accounted for about 13.4% of all completions in this time, so we're missing maybe 15 of his completions, total. Insignificant.
Attempts: 10.48 A/G 27-31, 12.91 A/G 32-34. This is very significant. We're missing around 2.5 attempts per game from 1927-31, around 1600 total - and almost all of them are incompletions. This makes sense, because complete passes would be much more likely to be noted in post-game recaps. Friedman accounted for about 10% of total pass attempts in these years, so we're missing about 160 of his attempts, 145 of which should be incomplete. His true Comp% is not as high as shown. Adding 15 completions and 145 incompletions to his stats gives a career percentage of 44.6 instead of 50.3.
Yards: 67.30 Y/G 27-31, 63.68 Y/G 32-34. Not significant in terms of incompleteness, which is expected considering the minuscule difference in completions. The weirdness of the result seems to be due to a passing volume drop in 1932 specifically. Impossible to estimate how many yards Friedman is missing, but it ain't much. If we multiply his career Y/C by the 15 or so lost completions, it's only about 260 more yards. His Y/A, however, will be lower due to the extra incompletions; 7.66 instead of 8.74, specifically, quite a big difference.
Pass TDs: 0.52 TD/G 27-31, 0.47 TD/G 32-34. One would expect TD data to be mostly complete, and this appears to be the case; they are the only stats considered official by PFR for Friedman, so I think we can assume they aren't missing anything. Again, 1932 is to blame for the discrepancy, as is Friedman running up 20 TDs in 1929, which was about 30% of the league total that year (Ruthian indeed).
Interceptions: 1.39 INT/G 27-31, 1.65 INT/G 32-34. Again significant. Unless QBs were just better in 27-31 (hint: they weren't, besides Friedman), we're missing about one INT per 4 team-games, or 165 INTs leaguewide over these 5 years. Friedman was a much better QB than the rest of these JAGsters, however, and only accounted for about 5.6% of all picks from 27-31, so only about 9 of those missing ones should be his. His career INT% actually decreases from 7.77 to 7.55, accounting for the missing pass attempts.
So overall, one may expect the NPR calculations for Friedman above to decrease as more data is uncovered, but not to a particularly large extent, especially since all of the adjustments I just applied to Friedman also affect the league as a whole. Sure enough, filling in the estimated incomplete stats for both Friedman, and the league minus Friedman, gives him an NPR of 119.17, a very small decrease. Either way, by all educated guesses, the 1927-31 pasaing stats are somewhere around 90 percent recorded, so the provided NPR figures for Friedman are very close to accurate. Oh yeah, dude could run, too.
I hope you enjoyed, and that you gained a new appreciation of Friedman, if you didn't already have one. So what did this exercise convince you more of: that Tom Brady couldn't carry Benny Friedman's cleats, or that NPR is ill-equipped to analyze him due to the horrible league context he played in? Discuss away.
For those who don't have Rupert's book, first of all, you should, and second, Normalized Passer Rating (NPR) is an era-adjusted metric that modifies the classic passer rating formula to compare QBs against league averages in specific years, rather than a constant baseline which has become obsolete. Therefore, NPR balances QBs from all eras relatively equally, instead of heavily favoring modern passers. It does tend to favor great QBs of earlier eras, and I'll explain why shortly; for this reason, I came in expecting Friedman to land at number one all-time.
Stats used to calculate these ratings were obtained from jt-sw.com. They are to be considered incomplete for the majority of these seasons, but I will address this issue extensively in this post.
1927: 101.6
1928: 106.71
1929: 143.98
1930: 137.85
1931: 123.3
1932: 60.32
1933: 123.27
Career: 121.95
All of the figures used for calculation can be seen on this spreadsheet:
https://docs.google.com/spreadsheets/d/ ... p=drivesdk
So, according to NPR, this makes Benny Friedman by far the most efficient quarterback of all time relative to his contemporaries. He's actually the only passer rated over 100 in this system; the #1 career NPR since 1932 is Otto Graham (96.1), or Sid Luckman (94.4) if you prefer to isolate Graham's NFL stats. He is so far ahead that his entire career would be the third-highest rated season in history, behind 1943 and 1941 Luckman. On that note, Friedman's best two seasons replace Luckman's best two on top of that leaderboard, although by not as enormous a margin as he leads the career ranking. 1932 was his only season below 100 (I didn't bother doing 1934 separately because he barely played, but it's in his career stats), but I was definitely surprised to see 1933 rate above 1927 and 1928.
Those who understand how era-adjusted passing metrics work will likely not be altogether shocked by much of this, because speak of the devil, the Luckman Effect is, of course, very much operative in all of it. That is, in earlier and smaller eras of the NFL, years with one or two excellent QBs and otherwise mostly terrible ones were frequent, and the abysmal league averages in those years tend to artificially inflate the NPR of anyone who's any good at all. Friedman, even more than Luckman, was a very big fish in a league full of very small fish at QB, heavily inflating the distance between him and the league average. Luckman and Baugh at least had each other in the league to keep the average somewhat afloat, but Friedman was truly the only game in town, and the average without him was utterly destitute. So that is one reason why it is hard to take NPR at face value in this instance. I expected to see Friedman land at a comfortable #1 for this very reason, but I have to admit the magnitude of the result did surprise me. I can't say it changes my perspective on him, though, given that I already considered him a legend.
It is also unclear how much of Friedman's recorded stats are incomplete, which obviously complicates any attempt to calculate ratio and efficiency stats for such players. However, we can estimate this factor by comparing the leaguewide stats in the unofficial years with those of the official years, and interpolating the difference to just Friedman. So I decided to compare the total numbers for 1927-31 to those in 1932-34, to estimate how much is missing from the former. Per the method for deriving NPR, Friedman's output was removed originally from the league stats for 1927-34, so he has been added back in for this exercise.
However, there is a clear problem here: teams often played different amounts of games during this era of football, and there were also different numbers of teams in the league each year. So I painstakingly counted up every individual game played between two NFL teams for these years, and multiplied the result by two to get an accurate count of "team-games" played per season; the figures are listed in the spreadsheet, under the games played column. With this, we can estimate how incomplete 1927-31 stats are, by noting how much lower the per-game passing stats are for those years compared to 1932-34, assuming per-game passing volume did not change notably over this span. Here is what I got:
Completions: 4.18 C/G 1927-31 (662 games), 4.35 C/G 1932-34 (330 games). Therefore, about 0.17 completions per game, or around 110 total leaguewide, are missing from the record. Friedman accounted for about 13.4% of all completions in this time, so we're missing maybe 15 of his completions, total. Insignificant.
Attempts: 10.48 A/G 27-31, 12.91 A/G 32-34. This is very significant. We're missing around 2.5 attempts per game from 1927-31, around 1600 total - and almost all of them are incompletions. This makes sense, because complete passes would be much more likely to be noted in post-game recaps. Friedman accounted for about 10% of total pass attempts in these years, so we're missing about 160 of his attempts, 145 of which should be incomplete. His true Comp% is not as high as shown. Adding 15 completions and 145 incompletions to his stats gives a career percentage of 44.6 instead of 50.3.
Yards: 67.30 Y/G 27-31, 63.68 Y/G 32-34. Not significant in terms of incompleteness, which is expected considering the minuscule difference in completions. The weirdness of the result seems to be due to a passing volume drop in 1932 specifically. Impossible to estimate how many yards Friedman is missing, but it ain't much. If we multiply his career Y/C by the 15 or so lost completions, it's only about 260 more yards. His Y/A, however, will be lower due to the extra incompletions; 7.66 instead of 8.74, specifically, quite a big difference.
Pass TDs: 0.52 TD/G 27-31, 0.47 TD/G 32-34. One would expect TD data to be mostly complete, and this appears to be the case; they are the only stats considered official by PFR for Friedman, so I think we can assume they aren't missing anything. Again, 1932 is to blame for the discrepancy, as is Friedman running up 20 TDs in 1929, which was about 30% of the league total that year (Ruthian indeed).
Interceptions: 1.39 INT/G 27-31, 1.65 INT/G 32-34. Again significant. Unless QBs were just better in 27-31 (hint: they weren't, besides Friedman), we're missing about one INT per 4 team-games, or 165 INTs leaguewide over these 5 years. Friedman was a much better QB than the rest of these JAGsters, however, and only accounted for about 5.6% of all picks from 27-31, so only about 9 of those missing ones should be his. His career INT% actually decreases from 7.77 to 7.55, accounting for the missing pass attempts.
So overall, one may expect the NPR calculations for Friedman above to decrease as more data is uncovered, but not to a particularly large extent, especially since all of the adjustments I just applied to Friedman also affect the league as a whole. Sure enough, filling in the estimated incomplete stats for both Friedman, and the league minus Friedman, gives him an NPR of 119.17, a very small decrease. Either way, by all educated guesses, the 1927-31 pasaing stats are somewhere around 90 percent recorded, so the provided NPR figures for Friedman are very close to accurate. Oh yeah, dude could run, too.
I hope you enjoyed, and that you gained a new appreciation of Friedman, if you didn't already have one. So what did this exercise convince you more of: that Tom Brady couldn't carry Benny Friedman's cleats, or that NPR is ill-equipped to analyze him due to the horrible league context he played in? Discuss away.