Skip to content
HURUmap is an interactive web platform that helps journalists and civic watchdogs easily explore and download census or other demographic data. The toolkit includes shareable and customisable infographics. Accessible at https://hurumap.org
Branch: master
Clone or download
Latest commit 967d62f Jul 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Update issue templates Jul 29, 2018
contrib Revert loaddata changes Jun 18, 2019
dominion Merge pull request #199 from CodeForAfrica/fix/za_nw_2016_population_… Jul 12, 2019
elimu_yangu Remove python 2.7 iter* methods Apr 2, 2019
hurumap_global Update hurumap image version & format code Feb 22, 2019
hurumap_ke fix exception handling for ke Jul 8, 2019
hurumap_land catch exception on hurumap_land Jul 12, 2019
hurumap_ng Change insight Jul 19, 2019
hurumap_tz fix exception catching Jul 8, 2019
hurumap_ug fix hurumap zw zm ug Apr 2, 2019
hurumap_zm fix exception handling for zm Jul 8, 2019
hurumap_zw fix exception handling for zw Jul 8, 2019
pesayetu Remove python 2.7 iter* methods Apr 2, 2019
.gitignore revert gitigonre changes Jun 27, 2019
CHANGELOG.md add changelog Jan 15, 2019
CHECKS clean up Jul 30, 2018
CONTRIBUTING.md Documentation update. Dec 7, 2017
Dockerfile Upgrade HURUmap to v0.6.5 Jul 4, 2019
Makefile fix elimu_yangu Jan 24, 2019
README.md Update README.md Mar 25, 2019
VERSION
docker-compose.yml Update to dominion + hurumap 0.5 Mar 25, 2019
manage.py Use the new HURUmap Docker (#105) Nov 25, 2018
runtime.txt Upgrade the runtime. Oct 25, 2017
utils.py fix key ordering for some indicators Feb 18, 2019

README.md

HURUmap

What’s the data behind the story? HURUmap gives infomediaries like journalists and civic activists an easy ‘plug & play’ toolkit for finding and embedding interactive data visualisations into their storytelling.

HURUmap’s underlying data is quality-checked, from reputable official sources including the government Census, PEPFAR and Uwezo.

The project is built on Wazimap, an open source platform by OpenUp and Media Monitoring Africa for making census data more understandable.

Development

We use Docker Compose to simplify development.

To get started, set the HURUmap App you want to work on and spin up the container like so:

export HURUMAP_APP=hurumap_ke
make web

You can create a db and load initial data by running the the following commands;

make createdb
make loaddata

# (Optional) If sqlalchemy.exc.NoSuchTableError error thrown:
export HURUMAP_APP=hurumap_land
# 1. Local Docker DB
docker-compose up -d db
cat $HURUMAP_APP/sql/*.sql | docker-compose exec -T db psql $HURUMAP_APP
# 2. Remote DB option
cat $HURUMAP_APP/sql/*.sql | docker-compose exec -T -e PGPASSWORD=<pass> db psql -h <db.host.com> -U <user> $HURUMAP_APP

elimu_yangu .sql files

olevel_student_performance.sql has been split into 2 files by year because as a single file, it surpasses the maximum file size allowed by Github

Import Data into HURUmap

TODO: Needs to use docker-compose, test, and QA

  1. Ensure ${HURUMAP_APP}/tables.py has a FieldTable that has exactly the columns that you're importing. If there are multiple tables with exactly the same columns, perhaps because their Universes are different, then be sure to take note of the table id.

  2. Do a dry-run of the import, using the table name if necessary.

     python manage.py importsimplecsv yourfile.csv --dry-run [--table TABLENAME]
    
  3. If it all looks good, run it without --dry-run.

  4. Update (or create) the raw SQL data:

     python manage.py dumppsql --table TABLENAME > sql/TABLENAME.sql
    
  5. Commit to git.

  6. All done!

Downloading / Archive Data:

We make the data available in our repository for added availability in two primary ways:

  1. Django fixtures (primarily wazimap_geography)
  2. FieldTables as SQL files.

To do this, run the following command:

export HURUMAP_APP=<hurumap_app>
make dumpdata

Deployment

We use dokku to deploy on our own servers. It's awesome like sliced bread or chapati. Check out their docs on getting started: https://dokku.viewdocs.com/dokku

Once set up, you'll have to do a couple of things:

# Create app
dokku apps:create hurumap-ke

# Set environment variables
dokku config:set hurumap-ke \
  HURUMAP_APP=hurumap_ke \
  DJANGO_SETTINGS_MODULE=hurumap_ke.settings \
  DATABASE_URL=postgresql://hurumap_ke:hurumap_ke@localhost/hurumap_ke

After ensuring your ssh key is added, from your local machine you should now run:

git remote add dokku dokku@hurumap.org:hurumap-ke
git push dokku

NOTE: You'll have to set up the database before deployment. Either self-hosted or managed.

Checks

Dokku allows for checks that make sure you have zero-downtime deployments. We currently only check for DB errors but should allow for better checks in the future.

Contributing

If you'd like to contribute to HURUmap, check out the CONTRIBUTING.md file on how to get started.


Importing data into HURUmap

  1. Ensure hurumap_ke/tables.py has a FieldTable that has exactly the columns that you're importing. If there are multiple tables with exactly the same columns, perhaps because their Universes are different, then be sure to take note of the table id.

  2. Do a dry-run of the import, using the table name if necessary.

     python manage.py importcsv yourfile.csv --dry-run [--table TABLENAME]
    
  3. If it all looks good, run it without --dry-run.

  4. Update (or create) the raw SQL data:

     python manage.py dumppsql --table TABLENAME > sql/TABLENAME.sql
    
  5. Commit to git.

  6. All done!

To dump all data tables at once, run

for t in `ls hurumap_ke/sql/[a-z]*.sql`
do
    echo $t
    pg_dump "postgres://hurumap_ke:hurumap_ke@localhost/hurumap_ke" \
        -O -c --if-exists -t $(basename $t .sql) \
      | egrep -v "(idle_in_transaction_session_timeout|row_security)" \
      > hurumap_ke/sql/$(basename $t .sql).sql
done

Adding Google Analytics

HURUmap supports the use of both single and multiple tracking ids on a single page.

For the case of a single tracking id (or when one tracking id needs to be identified as the "default" tracking id), the variable ga_tracking_id should be set to the string value of the tracking id.

For example:

[hurumap_ke/settings.py]

...
HURUMAP['ga_tracking_id'] = 'UA-44795600-8'
...

And in those cases were multiple tracking ids need to be set, ga_tracking_ids (with an s) should be set as a list of tracking ids.

For example:

[hurumap_ke/settings.py]

...
HURUMAP['ga_tracking_ids'] = ['UA-44795600-1', 'UA-44795600-2', 'UA-44795600-3']
...

NOTE: By default, HURUMAP['ga_tracking_id'] = 'UA-44795600-8'. If you're not using ga_tracking_id at all (such as in those situation where you're only using multiple tracking ids or you're not using Google Analytics altogether), remember to set this variable to blank i.e. HURUMAP['ga_tracking_id'] = ''

Tests

?


License

MIT

You can’t perform that action at this time.