Skip to content
A scrape of data from the UK Engineering and Physical Sciences Research Council "Grants on the Web" service.
Python JavaScript Shell
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
epsrc
schema
.gitignore
README.mkd
migrate.sh
openspending-model.js
setup.py

README.mkd

EPSRC Past Grant portfolio

As of 2011-03-08, the EPSRC "Grants on the Web" database contains grants dating back to 1985:

  • 35,966 grants
  • £6,055,173,655 of committed funding

This repository contains tools for scraping and manipulating this data into a useful format.

Quick access

Instead of using the code in this repository, why not download the database generated by it? A listing of these can be found at CKAN.

Setup

To get up and running, you will need python and pip. Instructions for installing python are beyond the scope of this document, but you should be able to get pip if you don't have it by running:

$ easy_install pip

If you use virtualenv, you might want to create an environment to play around in:

$ virtualenv --distribute pyenv
$ source pyenv/bin/activate

Now install the scraper and its dependencies:

$ pip install -e .

Create the database (or migrate your existing database to the current schema version) by running:

$ ./migrate.sh epsrc.db 3

Usage

To get a database of basic grants information for the financial year beginning 1 Apr 1993, run:

$ epsrc-scrape --basic --year 1993 epsrc.db

More usefully, to download all the information in the EPSRC GOW repository (this will take a long time to complete):

$ epsrc-scrape epsrc.db

Each grant has a unique grant reference, e.g. GR/J50118/01. We can check that the example grant above made it into the database with the following command

$ sqlite3 -line epsrc.db "select * from grants where id='GR/J50118/01'"
                       id = GR/J50118/01
                    title = USE OF CATALYTIC MEMBRANES FOR IN SITU REACTION AND SEPARATION
                    value = 114608
principal_investigator_id = 40792
            department_id = 5273
               created_at = 2011-06-16 12:36:35.164790
              modified_at = 2011-06-16 12:36:35.164800

Warnings

Be careful with the data this scraper generates. Care has been taken to sanitize the data, but as with any scraper, there may be inconsistencies or missing entries. Please let me know if you find any problems.

Something went wrong with that request. Please try again.