Tools for generating portable data portals
Python CSS HTML Makefile JavaScript
Latest commit 22aa191 Feb 16, 2017 @rlafuente rlafuente committed on GitHub Merge pull request #26 from schoolofdata-ch/datapackage-support
Data Package download support
Permalink
Failed to load latest commit information.
content Reverted welcome_text Feb 7, 2017
plugins csv2ods conversion plugin stub Jun 2, 2014
themes/centraldedados Refer to source file, not GitHub Feb 7, 2017
.gitignore Merge templates with updated versions from centraldedados.pt Jul 9, 2016
Makefile
README.md Cleanup schoolofdata changes Feb 7, 2017
datacentral.py
requirements.txt
settings.conf.sample Commented sample settings file Jul 9, 2016
tests.py
utils.py Update remote DP Feb 7, 2017

README.md

Data Central

This is a lightweight platform to easily publish and distribute datasets. It was created to be the base for Central de Dados, a repository of data packages related to Portugal.

It uses Open Knowledge's excellent Data Packages specification as a common format to provide datasets. See the Frictionless Data vision document to understand why it's crucial to think about dataset packaging and distribution.

The main design principle when coming up with Data Central was portability and simplicity of deployment. It is a framework-less approach, using Python scripts to gather and compile all datasets, creating a static HTML web site. Static sites might not be terribly sexy, but they're extremely useful for some purposes:

  • data workshops
  • offline work
  • easy deployment
  • portability and replication

We informally refer to this project as "the poor man's CKAN".

Data Central even exposes a static JSON 'API', so that developers have an easy access to the available datasets and their metadata on the portal.

Installation and Usage

  1. Install dependencies. After cloning the repository, ensure that you have virtualenv installed with this command:

    $ pip show virtualenv

If it's not there, you can install it with:

$ pip install --user virtualenv

Now, change to the project directory and run make install. This will create a local virtualenv and install the necessary dependencies; it shouldn't be necessary to create a virtualenv since the make commands are all set to work with the venv that make install creates inside the Data Central dir.

  1. Edit settings. Edit the newly created settings.conf to set your options and point to your data package repositories.

  2. Add content. The sidebar is a good place to tell your visitors what this site is about in a few paragraphs. You can edit this in content/welcome_text.md. There is also a dedicated About page you can modify. Look and feel can be customised by editing the default theme or adding your own to the themes folder and changing settings.

  3. Generate the HTML output. Just run make build!

  4. Push the static HTML output somewhere!. The generated site is placed at the _output directory. Just copy the contents to your webserver, everything's included.

  5. Run a web server to see the output. While developing, you can also run make serve to run a simple webserver, and then open the site by pointing your browser to localhost:8002.

  6. Upload to a remote web server to publish. Using rsync, your portal contents are compressed and uploaded to a remote server with a command like SSH_PATH="my.server.org:/my/remote/path" make deploy.

Running tests

Datacentral uses Nose for testing. After installing it on your system or virtualenv, just run

nosetests tests.py

TODO

  • Set up an English language base theme for the HTML output
  • Use Hyde or Pelican for a more solid static generator back-end