CKAN Datastorer Extension
- The CKAN Datastorer Extension provides a Celery task for automatically
- saving CKAN resources that link to csv and excel files into the datastore.
After activating your pyenv, install the sources via pip:
$ (pyenv) pip install -e git+git://github.com/okfn/ckanext-datastorer.git#egg=ckanext-datastorer
Install the requirements:
$ (pyenv) pip install -r ckanext-datastorer/pip-requirements.txt
Add the datastorer plugin to your configuration ini file:
ckan.plugins = datastorer <rest of plugins>...
Start the celery daemon. This can be done in development by:
paster celeryd # this is assuming a development.ini file
In production the daemon should be run with a different ini file and be run as an init script. The simplest way to do this is to install supervisor:
apt-get install supervisor
You can use this file as a template and add it to /etc/supservisor/conf.d:
A paster command is available, that lets you archive all resources or just those belonging to a specific package. The command is as follows:
paster datastorer update [package-id]
To queue the update to run in celery, use:
paster datastorer queue [package-id]
You can run the test suite from the ckanext-datastorer directory. The tests require nose, so install it first if you have not already done so:
$ pip install nose
To run the tests, you will need to be running a CKAN instance, and provide the API key of a sysadmin user on the tests configuration file located on:
Note: Make sure that celery is not running during the tests. Otherwise strange errors will occur!
Then, run nosetests from the ckanext-datastorer directory
$ nosetests ckanext/datastorer/tests