This is a lightweight platform to easily publish and distribute datasets. It was created to be the base for Central de Dados, a repository of data packages related to Portugal. It also powers the Open Food Hackdays portal by the School of Data Switzerland.
It uses Open Knowledge's excellent Data Packages specification as a common format to provide datasets. See the Frictionless Data vision document to understand why it's crucial to think about dataset packaging and distribution.
The main design principle when coming up with Data Central was portability and simplicity of deployment. It is a framework-less approach, using Python scripts to gather and compile all datasets, creating a static HTML web site. Static sites might not be terribly sexy, but they're extremely useful for some purposes:
- data workshops
- offline work
- easy deployment
- portability and replication
We informally refer to this project as "the poor man's CKAN".
Data Central even exposes a static JSON 'API', so that developers have an easy access to the available datasets and their metadata on the portal.
Installation and Usage
Install dependencies. After cloning the repository, ensure that you have virtualenv installed with this command:
$ pip show virtualenv
If it's not there, you can install it with:
$ pip install --user virtualenv
to the project directory and run
make install. This will
create a local virtualenv and install the necessary
dependencies; it shouldn't be necessary to create a virtualenv
make commands are all set to work with the venv
make install creates inside the Data Central dir.
Edit settings. Edit the newly created
settings.confto set your options and point to your data package repositories.
Add content. The sidebar is a good place to tell your visitors what this site is about in a few paragraphs. You can edit this in
content/welcome_text.md. There is also a dedicated About page you can modify. Look and feel can be customised by editing the default theme or adding your own to the
themesfolder and changing settings.
Generate the HTML output. Just run
Push the static HTML output somewhere!. The generated site is placed at the
_outputdirectory. Just copy the contents to your webserver, everything's included.
Run a web server to see the output. While developing, you can also run
make serveto run a simple webserver, and then open the site by pointing your browser to
Upload to a remote web server to publish. Using rsync, your portal contents are compressed and uploaded to a remote server with a command like
SSH_PATH="my.server.org:/my/remote/path" make deploy.
Datacentral uses Nose for testing. After installing it on your system or virtualenv, just run