Extract / Transform / Load Scripts for databases used in Dataviva Project.
The sources of Dataviva can be found at Dataviva GitHub.
How it Works
Databases are released by Brazil government as raw data and there is some analysis to extract the information needed for the project.
For each database we have a Wiki documentation that explain all the ETL process.
- You'll need libmagic for this project:
brew install libmagic
Ubuntu (BSD Linux) installation
apt-get install libmagic
- Install requirements
pip install -r requirements.txt
See the Wiki for more information.