A CKAN extension to create visualizations based on the uploaded datasets.
Please follow installation instructions of the software below if needed. Also, take a look inside the Makefile
to understand what's going on under the hood:
docker
docker-compose
/etc/hosts
contains the127.0.0.1 ckan-dev
line
For building static assets and running end-to-end tests Node.js is required and can be installed with these commands:
$ nvm install 10
$ nvm use 10
$ npm install
Clone the ckanext-querytool
repository (assuming that we're inside the projects
directory):
$ git clone git@github.com:ViderumGlobal/ckanext-querytool.git
$ cd ckanext-querytool
This is a CKAN extension and docker compose setup for local development. It's designed to support live development of extensions. The only one requirement is that the folder with the project should be named ckanext-querytool
.
The whole docker setup is inside the docker
directory. You can tweak any CKAN instance's aspects there (e.g. patches/cron/etc). To add other CKAN extensions to the work - add its folders to docker/docker-compose.dev.yml
(see ckan-dev
volumes).
Pull the latest ckan-base/dev
images and build the project's images:
$ make docker
Let's start the development server. It's recommended to run this command in an additional terminal window because you neede it running during the work. All changes to connected extensions will trigger live-reloading of the server:
$ make start
# see CKAN logs here
Now we can visit our local ckan instance at (you can login using ckan_admin@test1234
):
http://ckan-dev:5000/
To pass the end-to-end tests you have to add required applictions from the staging site to your development server. See notes for every single test
We write and store unit tests inside the ckanext/querytool/tests
directory and end-to-end tests inside the tests
directory. Prefer to name test files after feature/bug names. To run the tests you should have the development server up and running:
$ make test
$ npx nightwatch tests/<testname>.js # for a single test
See the how to write E2E tests
guide:
Put your scripts/fonts/etc inside the ckanext/querytool/fanstatic
folder and images inside the ckanext/querytool/public
folder. It can be used as usual ckan fanstatic
and public
contents. At the same time, we use CSS preprocessor (LESS) to build our styles. Put your styles inside the ckanext/querytool/fanstatic/less
and build it:
$ make assets
Processed styles will be put to the ckanext/querytool/fanstatic/css
folder.
To extract i18n messages and compile the catalog we have to have our development server running. In another terminal window run a command:
$ make i18n
See CKAN documentation for more on i18n management.
To issue commands inside a running container:
$ make shell
Now you can tweak the running ckan-dev
docker container from inside. Please take into account that all changes will be lost after the next container restart.
To update this readme's table of contents run:
$ make readme
There are a few useful docker commands:
$ docker ps -aq # list all containers
$ docker stop $(docker ps -aq) # stop all containers
$ docker rm $(docker ps -aq) # remove all containers
$ docker rmi $(docker images -q) # remove all images
It will destroy all your projects inside docker!!!
If you want to start everything from scratch there is a way to prune your docker environment:
$ docker system prune -a --volumes
The app allows to configure different layouts for an application's visualizations. We can test expected behavious in half-automated way using Chrome console in the brower:
- Go to /querytool/public/detailed-mortality-by-cause
- Copy-past and start in the console script from
bin/test-layouts.js
To install ckanext-querytool:
- Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
- Install the ckanext-querytool Python package into your virtual environment:
pip install ckanext-querytool
-
Add
querytool
to theckan.plugins
setting in your CKAN config file (by default the config file is located at/etc/ckan/default/production.ini
). -
Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
These are the required configuration options used by the extension:
- Add config for application groups in a comma separated key:value pairs:
ckanext.querytool.groups = brazil:Brazil,china:θ₯ε
»,mexico:Mexico
- Add config for map item base layer:
ckanext.querytool.map_osm_url = https://cartodb-basemaps-{s}.global.ssl.fastly.net/light_nolabels/{z}/{x}/{y}{r}.png
ckanext.querytool.map_osm_attribute = © <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a> © <a href="http://cartodb.com/attributions">CartoDB</a>
- Add config for the base public breadcrumb name:
ckanext.querytool.public_breadcrumb_name = Health Topics
- Add config for visibility of navigation bar in the public query tools:
ckanext.querytool.allow_nav_bar = False
To install ckanext-querytool for development, activate your CKAN virtualenv and do:
git clone https://github.com/ViderumGlobal/ckanext-querytool.git
cd ckanext-querytool
python setup.py develop
pip install -r dev-requirements.txt
All code MUST follow PEP8 Style Guide. Most editors have plugins or integrations and automatic checking for PEP8 compliance so make sure you use them.
You should add a pre-commit hook that will check for PEP8 errors. Follow the next steps to enable this check.
- Make sure you have installed the PEP8 style checker:
$ pip install pycodestyle
- In the
.git/hooks
folder which is located inside the project's root directory, create a file namedpre-commit
and inside put this code. - Make
pre-commit
executable by running this command:
$ chmod +x ckanext-querytool/.git/hooks/pre-commit
Now, every time you commit code, the pre-commit hook will run and check for PEP8 errors.
This extension uses LESS for styles. All changes must be made in one of the LESS
files located in the ckanext-querytool/ckanext/querytool/fanstatic/less
folder.
Gulp.js is used for building CSS assets.
In order to build all CSS assets run node_modules/gulp/bin/gulp.js
once. Gulp.js is watching for changes in the source LESS assets and will rebuild CSS on each change. If you have gulp.js installed globally on the system, you can just type gulp
to run it.
To run the tests, first make sure that you have installed the required
development dependencies in CKAN, which can be done by running the following
command in the CKAN's src
directory:
pip install -r dev-requirements.txt
After that just type this command to actually run the tests in the extension.
nosetests --ckan --with-pylons=test.ini
To run the tests and produce a coverage report, first make sure you have coverage installed in your virtualenv (pip install coverage) then run:
nosetests --nologcapture --with-pylons=test.ini --with-coverage --cover-package=ckanext.querytool --cover-inclusive --cover-erase --cover-tests