As of 2012-05-28, the EPSRC "Grants on the Web" database contains grants dating back to 1985:
- 43,317 grants
- £10,935,828,250 of committed funding
This repository contains tools for scraping and manipulating this data into a useful format.
Instead of using the code in this repository, why not download the database generated by it? A listing of these can be found at CKAN.
To get up and running, you will need
pip. Instructions for installing
python are beyond the scope of this document, but you should be able to get
pip if you don't have it by running:
$ easy_install pip
If you use
virtualenv, you might want to create an environment to play around in:
$ virtualenv --distribute pyenv $ source pyenv/bin/activate
Now install the scraper and its dependencies:
$ pip install -e .
Create the database (or migrate your existing database to the current schema version) by running:
$ ./migrate.sh epsrc.db 4
This scraper operates on a local mirror of the EPSRC GOW website. You must first make this mirror (this will take a long time to complete, often several days):
$ wget -c -m gow.epsrc.ac.uk
Then you can use the
epsrc-scrape tool to parse this data:
$ epsrc-scrape epsrc.db ./gow.epsrc.ac.uk
Each grant has a unique grant reference, e.g.
GR/J50118/01. We can check that the example grant above made it into the database with the following command
$ sqlite3 -line epsrc.db "select * from grants where id='GR/J50118/01'" id = GR/J50118/01 title = USE OF CATALYTIC MEMBRANES FOR IN SITU REACTION AND SEPARATION value = 114608 principal_investigator_id = 40792 department_id = 5273 created_at = 2011-06-16 12:36:35.164790 modified_at = 2011-06-16 12:36:35.164800
Be careful with the data this scraper generates. Care has been taken to sanitize the data, but as with any scraper, there may be inconsistencies or missing entries. Please let me know if you find any problems.