OpenSpending is a project to make government finances easier to explore and understand. It started out as "Where does my money go", a platform to visualize the United Kingdom's state finance, but has been renamed and restructured to allow arbitrary financial data to be loaded and displayed.
The main use for the software is the site openspending.org which aims to track government finance around the world.
OpenSpending's code is licensed under the GNU Affero Licence except where otherwise indicated. A copy of this licence is available in the file LICENSE.txt.
OpenSpending is a microservices platform made up of a number of separate apps, each maintained in their own git repository. This repository contains docker-compose files that can be used run an instance of Openspending for development, or as the basis for a production deployment. This repository also acts as a central hub for managing issues for the entire platform.
Most applications that make up the OpenSpending platform are maintained in their own repositories, with their own Dockerfiles, built and pushed to the OpenSpending organisation on Docker Hub:
This repository maintains docker-compose files used to help get you started with the platform.
docker-compose.base.yml: This is the main docker-compose file for OpenSpending specific services. All installations will use this as the basis for running the platform.
docker-compose.dev-services.yml: This defines backing services used by the platform, such as Redis, ElasticSearch, and PostgreSQL. This file also includes fake-s3 in place of AWS S3, so you don't have to set up an S3 bucket for development. It is not recommended to use this for production.
docker-compose.data-importers.yml: This defines the services used for the separate os-data-importers application. They depend on services defined in
docker-compose.dev-services.yml. Unless you are working on the data-importers or its associated source-spec files, it's not necessary to run this file.
docker-compose.local.yml: Create this file to add additional services, or overrides for the base configuration. It is ignored by git.
Dockerfiles/*: Most services are maintained in their own repositories, but a few small custom services used by the platform are maintained here.
os-nginx-frontend is a basic frontend nginx server and configuration files to define resource locations for the platform. This will be build and run directly by
Define the environmental variables that applications in the platform need. The easiest way to do this is to create a
.env.exampleas a template).
docker-compose upto start the platform from the
dev-services, and optionally
$ docker-compose -f docker-compose.base.yml -f docker-compose.dev-services.yml [-f docker-compose.local.yml] up
localhost:8080in your browser.
You can use
volumes to map local files from the host to application files in the docker containers. For example, say you're working on OS-Conductor, you'll add an override service to
docker-compose.local.yml (create this file if necessary).
- Checkout the os-conductor code from https://github.com/openspending/os-conductor into
~/src/dockerfiles/os-conductoron your local machine.
- Add the following to
version: "3.4" services: os-conductor: environment: # Force python not to use cached bytecode PYTHONDONTWRITEBYTECODE: # Override CMD and send `--reload` flag for os-conductor's gunicorn server command: /startup.sh --reload # Map local os-conductor app files to /app in container volumes: - ~/src/dockerfiles/os-conductor:/app
- Start up the platform with
dev-services, and your
$ docker-compose -f docker-compose.base.yml -f docker-compose.dev-services.yml -f docker-compose.local.yml up
Now you can start working on os-conductor application files in
~/src/dockerfiles/os-conductor and changes will reload the server in the Docker container.
In Openspending, the os-data-importers application provides a way to import data and create fiscal datapackages from source-spec files. You can either work on the app independently, by following the README in the os-data-importers repository, or within the context of an Openspending instance, by using the included
docker-compose.data-importers.yml file, and starting Openspending with:
$ docker-compose -f docker-compose.base.yml -f docker-compose.dev-services.yml -f docker-compose.data-importers.yml up
This will start Openspending locally as usual on port
:8080, and the pipelines dashboard will be available on port
That's fine, just add the relevant resource locator to the .env file. E.g., you're using a third-party ElasticSearch server:
Great! There are many ways to orchestrate Docker containers in a network. E.g. for openspending.org we use Kubernetes. Use the
docker-compose.base.yml file as a guide for networking the applications together, with their appropriate environment variables, and add resource locators pointing to your backing services for Postgres, ElasticSearch, Redis, memcached, AWS S3 etc. See the
.env.example file for the required env vars you'll need to set up.
You'll also need to set up OAuth credentials for OS-Conductor (see https://github.com/openspending/os-conductor#oauth-credentials), and AWS S3 bucket details.
You can find the old OpenSpending v2, and the complete history for the codebase to that point, in the