Have we considered digitizing the data in game books?

[Throwin_Samoan]

Hi -

I have - from somewhere, I cannot remember the source - a bunch of .csv files with NFL play-by-play data from 2002-2012. By the looks of it, it may have been scraped from PFR PBP or somewhere else.

Have we ever considered a committee or working group or volunteer effort to take old game books and digitize that data, to make it easier for research purposes? Like if I have a question about something that happened in 1965 (as I did recently), I have to meticulously go through all the extant game books and keep track. If that data were in spreadsheet form, it would be easier to parse.

Big effort, obviously, as just for the 1960s we're talking about more than 1600 regular season games, and we don't have all the game books and some of them are tough to make out.

Just wondered if we had considered this as a project. It might be an outrageous idea, granted.

[Mark]

I thought they had wanted to do it years ago but didn't have enough volunteers to do it. Maybe I am thinking of something else though...I just know it had to do with play by play or gamebooks.

[TodMaher]

I thought they had wanted to do it years ago but didn't have enough volunteers to do it. Maybe I am thinking of something else though...I just know it had to do with play by play or gamebooks.

It was called the Football Project and was run by Patrick Gilligan was now runs the Football Database website.

The problem wasn't a lack of volunteers and it was a lack of volunteers who actually wanted to do the drudgery of entering the play-by-plays. So only three or four of us were doing that and the whole thing conked out.

I also understand the gamebook committees has acquired additional gamebooks over the years but they just haven't been added to the PFRA website.

[TodMaher]

Well, my memory isn''t what it used to be. I looked on the Internet Wayback Machine and found the old website for The Football Project and it says was founded in 1998 by Sean Lahman and Todd Grenier.

I think Patrick Gilligan was involved at some point and created the software for entering the play-by-play data???

[SeahawkFever]

Regarding play by play, last I checked, it’s complete on pro football reference since the 1994 season.

On FTN Fantasy, Aaron Schaatz recently released DVOA team rankings for the 1978 season, which is a fully play-by-play based statistic. So there’s probably a lot of play-by-play from that season and going forward.

As for before then, not sure how many game books and play-by-play accounts can be obtained.

Either way, it will be interesting to see how far back they can go with the play-by-play they can find, and I agree that game books should be digitized for those who research to see.

[CraigRye]

I think this is a very good idea, and in fact I have already started to do this with the Miami Dolphins. I have been working through from 1966 and I am currently working on the 1984 season. My version of digitising is by recreating each gamebook into the modern version. I use MS Word (then into PDF) and check the player totals etc via an Excel spreadsheet.
I would be interested to see the software or spreadsheet that was used in this other project if anyone has a copy.
I would also like to suggest that we do start to do this again and I am willing to co-ordinate as Co-chair of the Gamebook committee.
I do realise that there are a couple of stumbling blocks to getting all the info (enough volunteers / all the gamebooks) but I still feel it would be good to give it a go.
My suggestion would be to digitise each team, so if anyone wants to volunteer message me and let me know which team you would like to digitise.
On a connected note if anyone has any contacts at any of the teams to enable us to contact them to get any of the missing gamebooks as shown in the Gamebooks page that would be really helpful.

[Throwin_Samoan]

Massive project but I think it would be worthwhile. I don’t know that there are enough years left to get it done.

One big issue I see is that there was no consistent format for these things for years. They would not only require digitization but translation into a consistent format. The data would be most useful if all of it were formatted in the same way.

And I think (check me if I’m wrong), the bulk of these books came from the teams, right? Gaps may not be fillable if the teams themselves do not have the actual books in question.

[Throwin_Samoan]

FWIW, This is what the spreadsheets I have look like, which is very close but not identical to the PBP on PF-R. Again, I don't know where I got these, exactly, or who did them (surely someone here did), but consistency would be something to consider.

Screenshot 2025-04-08 120249.png (96.8 KiB) Viewed 3036 times

[TodMaher]

FWIW, This is what the spreadsheets I have look like, which is very close but not identical to the PBP on PF-R. Again, I don't know where I got these, exactly, or who did them (surely someone here did), but consistency would be something to consider.


Screenshot 2025-04-08 120249.png

That looks the play-by-play data from https://www.advancedfootballanalytics.c ... -data.html, which covers the years 2002-2012
2013-2024 is available from https://nflsavant.com/about.php
And 1994-2001 is available from Pro Football Reference.

The problem with the PFR play-by-plays and is that they don't tell you which team is in possession of the ball on each play - which I'm sure they did on purpose

[Throwin_Samoan]

That's very helpful, thank you.

If you could export PF-R's data, it wouldn't be that hard to add who has possession.

So we can basically get at 1994-2024, which is great.

Data is spotty pre-1960. But 1960-1993 would be a worthwhile cause, I feel.