Publicos.pt is an open source Django website and API to query and analyse data of the portuguese state. Thanks for checking it out.
The problem we aim to solve
This website aims to:
- Provide a consistent way to query portuguese public data using Django ORM
- Interrelate different public data
- Extract and present statistical features of the data
and consists in 4 components:
- A set of crawlers and validators that retrieve information from official databases and store them in Django models
- A set of Django models (in Django Apps) to store the data in a database
- A database with read access to anyone so anyone can query it by git-cloning this code.
- A website that uses the above points to provide some statistical features of the databases
We focus on three aspects of the portuguese state:
- Public Contracts: contracts between entities with a value and other fields.
- Members of the Parliament: Persons that have mandates in legislatures of the parliament.
- Laws: documents that are officially published as laws.
We use Django ORM for the API and database and d3.js for visualisations of statistical quantities of the database. The official website is written in English and translated to portuguese (via i18n), also hosted here.
The code is licenced under BSD.
The API and the crawler are documented.
Pre-requisites and installation
The installation depend on what you want to do:
Access and query the database
1- Install Django, psycopg2 and django-treebeard:
pip install -r api_requirements.txt
2- Download the source
git clone firstname.lastname@example.org:jorgecarleitao/public-contracts.git
3- Enter in the project's directory:
4- Run the example:
python -m contracts.tools.example
you should see two numbers. You now have full access to the database.
Deploy the website locally
1- Install the requirements:
pip install -r website_requirements.txt
2- Start the server:
python manage.py runserver
3- Enter in the url
You should see the website as it is in http://publicos.pt.
Some plots will not be displayed right away because they take some time to be
127.0.0.1:8000/contratos should show the latest contracts.
Crawl the official sources
pip install -r production_requirements.txt
If something went wrong, please report an issue so we can help you and improve these instructions.