Aid Transparency Tracker
A data quality measurement tool for international aid data.
Run the following commands to bootstrap your environment:
git clone --recursive --branch 2020tracker https://github.com/pwyf/aid-transparency-tracker.git cd aid-transparency-tracker
Setup a virtual environment, and install dependencies:
python3 -m venv venv echo "export FLASK_APP=iatidataquality/__init__.py" >> venv/bin/activate source venv/bin/activate pip install -r requirements.txt
Copy and edit the config.py.tmpl, including pointing at a postgres database:
cp config.py.tmpl config.py
Finally, run the setup script to populate your database:
This will prompt you to create a new admin user.
Fetching and testing data
You can download a dump of today’s IATI data with:
The data will be downloaded to the iatikitcache directory by default, or you can add an iatikit.ini file to specify a different location.
The relevant data (according to the organisations in your database) should then be moved into place using:
This will move files into the
IATI_DATA_PATHspecified in your config.py
Tests are run on this data using:
The complete output of this is stored as CSV files in the
IATI_RESULT_PATHspecified in your config.py
Finally, you can refresh the aggregate data shown in the tracker using:
This step will destructively populate the
aggregateresulttable of your database.
You can run a development server with:
The survey component currently requires the existence of three files (could be abstracted in future). Move them from the tests directory to the DATA_STORAGE_DIR you specified in config.py. E.g., if you set the directory to be /home/me/data/:
cp tests/2012_2013_organisation_mapping.csv /home/me/data/ cp tests/2012_indicators.csv /home/me/data/ cp tests/2012_results.csv /home/me/data/
If at any time you need to reset, you can drop all tables using:
Then follow the installation instructions to reinitialise.
Updating sampling poisoning