The South African instance of Wazimap, a Django application for exploring census and other similar data.
- clone the repo
virtualenv --no-site-packages env
pip install -r requirements.txt
WAZI_PROFILE environment variable to the instance you are working on, e.g.
DEFAULT_GEO_VERSION environment variable if you don't want to default to the latest, e.g. for youth and ecd
You will need a Postgres database for the instance you are running:
psql create user wazimap_<instance_name> with password wazimap_<instance_name>; create database wazimap_<instance_name>; grant all privileges on database wazimap_<instance_name> to wazimap_<instance_name>;
Import the data into the new database.
cat sql/*.sql | psql -U wazimap_<instance_name> -W wazimap_<instance_name>
Run migrations to keep Django happy:
python manage.py migrate
Import the fixtures for the django models:
python manage.py loaddata fixtures/<instance_name>/wazimap_django_models.json
Create an admin user:
python manage.py createsuperuser
Start the server:
python manage.py runserver
Updating django model data:
Be sure to dump the data to the appropriate fixture when making changes to django models data:
python manage.py dumpdata wazimap > fixtures/<instance_name>/wazimap_django_models.json
See the Wazimap deployment docs for all basic Wazimap configuration.
Set the profile
dokku config:set wazimap_za WAZI_PROFILE=<profile name> dokku config:set wazimap_za DEFAULT_GEO_VERSION=<geography version>
Add dokku as a remote, and then deploy:
git push dokku
Adding new census data
python manage.py importcsv command to import CSV data exported from StatsSA using the SuperWEB or SuperCROSS packages.
Let's suppose you want to add a new table with two fields: favourite colour and gender. We'll describe the process for SuperWEB, but SuperCROSS is similar.
Downloading from SuperWEB
- Visit SuperWEB
- Open the data collection that contains your data (Generally under
Community Profiles > Census 2011) by double-clicking the database.
- Add the geography details as rows by clicking the Geography heading and choosing the levels you want to import. To keep the amount of data sane, export wards separately to other levels such as municipality and province.
- Check the boxes alongside the levels you're interested in (or click the arrow to the right of Geography and choose the correct child level)
- Click the
Add to Rowbutton
- Find the field for the first field you're interested in (gender, in our example) and ensure its children are selected.
- Click the
Add to Columnbutton.
- Do the same for the second field (if appropriate): favourite colour in our example.
- Note: when adding more than one column, be sure to turn off automatic totals for columns by clicking the little blue Sigma symbol to the right of the field name (eg. Gender) in the column.
- Click the big play button
Retrieve datato load data into the table.
- Download the table as
Comma Separate Value (.csv)(not CSV string value).
Importing into Wazimap
FieldTablethat has exactly the columns that you're importing. If there are multiple tables with exactly the same columns, perhaps because their Universes are different, then be sure to take note of the table id.
Do a dry-run of the import, using the table name if necessary.
python manage.py importcsv yourfile.csv --dry-run [--table TABLENAME]
If it all looks good, run it without
Update (or create) the raw SQL data:
python manage.py dumppsql --table TABLENAME > sql/TABLENAME.sql
Commit to git.
To dump all data tables at once, run
for t in `ls sql/[a-z]*.sql` do pg_dump "postgres://wazimap_za@localhost/wazimap_za" \ -O -c --if-exists -t $(basename $t .sql) \ | egrep -v "(idle_in_transaction_session_timeout|row_security)" \ > sql/$(basename $t .sql).sql done