U.S. Patents - Harvester and Parser
The code here performs two tasks:
- Downloads U.S. patent documents from the USPTO website and saves them to local disk.
- Parses stored patent documents to a machine-readable format. (JSON objects)
Python packages required:
This is used to generate the dataset release here: http://patents.tttran.net/
Building the Datasets
To use this code, you need to know the range of patent numbers you wish to download. That is, you can't choose specific date ranges, but you can download a range of patent numbers corresponding to the desired date range.
For example, to download all of the 2010 patents you need to know that the first patent in 2010 is numbered 7640598 and the last patent in 2010 is numbered 7861316. You can then run the following code to harvest the files:
python patent_harvest.py --start=7640598 --end=7861316
Likewise, for 2010 the patent range is 7640598 and 8087093:
python patent_harvest.py --start=7861317 --end=8087093
Once downloaded, run the following command to generate the 2010 dataset:
python patent_parse.py --start=7640598 --end=7861316 --outfile=patent_dataset_2010.jsonl
And for the 2011 dataset:
python patent_parse.py --start=7861317 --end=8087093 --outfile=patent_dataset_2011.jsonl
Files are saved in JSON LINES format. More info here:
Please note that the parser provided is not designed to be able to parse all patent documents released by the USPTO website. Some of the older patents have inconsistent formatting (and broken HTML) that makes parsing difficult, but not impossible. Some adjustment to the code might be needed to handle older (and possibly the newest) patents.
tung.tran [at] uky.edu