IMPORTANT! Project is discontinued and will no longer be maintained.

OS2BOS

Installation

TL;DR: To get a running development environment run:

git clone git@git.magenta.dk:bevillingsplatform/bevillingsplatform.git
cd bevillingsplatform
docker-compose up -d

You can now reach the frontend at http://localhost:8080. The frontend will proxy all request to the backend.

Run backend tests with:

docker-compose exec bev pytest

Docker

The repository contains a Dockerfile. This is the recommended way to install bevillingsplatform both as a developer and in production.

All releases are pushed to Docker Hub at magentaaps/bevillingsplatform under the latest tag.

To run bevillingsplatform in docker you need a running docker daemon. To install docker we refer you to the official documentation.

To configure the django inside the image, add your setting to /code/settings.ini. This is easiest done by binding a file from the host machine to this file.

The container requires a connection to a postgres database server. It is configured with the DATABASE_* settings. The database server must have a user and a database object. It can be created with the help of /docker/postgres-initdb.d/20-create-db-and-user.sh. This file can easily be mounted into /docker-entrypoint-initdb.d/ in the official postgres docker image.

You can start the container with:

docker run -p 8000:8000 -v $PWD/dev-settings.ini:/code/settings.ini magentaaps/bevillingsplatform:latest

This will pull the image from Docker Hub and starts a container in the foreground. The -p 8000:8000 binds port 8000 of the host machine to port 8000 on the container. The -v $PWD/dev-settings.ini:/code/settings.ini binds the dev-settings.ini file into the container at the location where django will pick it up.

If successful you should see the container migrating the database and finally

[2019-06-13 09:18:48 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)

when the gunicorn server starts up. You should now be able to reach the server from the host at http://localhost:8000.

If you continue to see 01/30 Unable to connect to database. your database configuration most likely wrong. Remember if you set DATABASE_HOST=localhost the container will try to connect to a database in the same container, not the host machine.

Static files

The docker image will serve the static files for the Vue frontend, Django REST framework and Django Admin from gunicorn both in development and in production. Normally it is not good practice to serve static files from gunicorn for security and performance reasons. We use whitenoise to address most of these concerns and generally don't expect many users. If you still want to serve it from another service, all the files are copied to /static on container startup. This can easily be mounted to a webserver.

Logs

For logging we use the builtin logging module used by Django. Logs are written to the /log/ directory inside the container.

The /log/ folder is mapped to the host systems log/ folder in local development and /var/log/os2bos/ anywhere else.

We log the following:

• Error log - the gunicorn error log is output on STDERR as well as in /log/error.log. It can be inspected with docker logs.
• Access log - The gunicorn access log is output on STDOUT as well as in /log/access.log. It can be inspected with docker logs.
• Django debug log - The django log is written to /log/django-debug.log.
• Audit log - We log all non-idempotent requests (POST/PUT/PATCH/DELETE) in the audit log specifying the request endpoint and user making the request. The log is written to /log/audit.log.
• Export to Prism log - Log related to the PRISM exports management command. The log is written to /log/export_to_prism.log.
• Mark Payments Paid log - Log related to the marking payments paid management command. The log is written to mark_payments_paid.log.
• Generate Payments Report log - Log related to generating the payment reports. The log is written to /log/generate_payments_report.log.
• Cron mail logs - Logs related to the Django mailer app. The logs are written to cron_mail.log, cron_mail_deferred.log, cron_mail_purge.log.

User permissions

The Dockerfile creates and runs the application as the bev user. This user will own all the files generated by the application. This user has a UID and GID of 72050.

If you want to use another UID/GID, you can specify it as the --user=uid:gid overwrite flag. for the docker run command or in docker-compose. If you change the UID/GID, the /log and /static volumes may not have the right permissions. It is recommended to only use bind if you overwrite the user and set the same user as owner of the directory you bind.

If some process inside the container needs to write files to locations other than /static or /log, you need to mount a volume with the right permissions. An example is ./manage.py makemigrations trying to write to /code/backend/core/migrations. If you bind /code to your host system, make sure that the user with UID 72050 have write permissions to backend/core/migrations. This can be done with chmod o+w migrations on your host where you grant all user permission to write.

Test

All the requirements for tests included in the docker image. You can run the test from inside a container with pytest.

tox

tox is also installed, but it tries to create a virtual environments inside the container. This is messy and will fail because the application user does not have permission to write files. Don't use tox inside the container.

Docker-compose

You can use docker-compose to start up bevillingsplatform and related services such as postgres and postfix.

A docker-compose.yml for development is included. It includes the settings to connect them. It starts the following services:

• frontend: the vue frontend reachable at http://localhost:8080
• bev: the django backend
• db: a OS2BOS specific postgres database server
• bev-cron: supercronic, a job runner running our cronjobs
• idp: a local version of the IdP running our version of SimpleSAMLphp
• postfix: a postfix email server

Normally the backend image also serves the frontend code, but to ease frontend development, we include a frontend service that run vue-cli-service serve. The frontend proxies requests to the backend. The exact list of proxied endpoints can be seen in frontend/vue.config.js.

docker-compose.yml also mounts the current directory in the container and automatically restarts the server on changes to the backend files. This enables you to edit the backend files and the server will be reloaded automatically.

To pull the images and start the services run:

docker-compose up -d --build

The -d flag move the services to the background. You can inspect the output of them with docker-compose logs <name> where <name> is the name of the service in docker-compose.yml. The --build flag builds the newest docker image for bevillingsplatform from the local Dockerfile.

To stop the service again run docker-compose stop. This will stop the services, but the data will persist. To completely remove the containers and data run docker-compose down -v.

Tests and shell access

To run the backend test, execute: docker-compose exec bev ./manage.py test. It will connect to the running docker container and execute the tests.

To get shell access to the backend run docker-compose exec bev bash.

If you want to write files from inside the container, make sure the bev user have permission to do so. See User permissions.

Tests can also be executed locally with tox:

tox -e test

Code coverage

We adhere to a code coverage of 100%.

After running the test-suite a coverage report can be generated locally with tox:

tox -e coverage

Documentation

The documentation exists at Read the Docs and can be generated locally with tox:

tox -e docs

When changes are introduced to the Django models, update and commit the database model graph for use in documentation:

tox -e graph

Code standards

The Python code is enforced with the following standards:

• black
• flake8
• pydocstyle (PEP257)

Adherence to these standards can be checked locally with tox:

tox -e lint

Licensing

Copyright (c) 2019 Magenta Aps

Bevillingsplatform is free software; you may use, study, modify and distribute it under the terms of version 2.0 of the Mozilla Public License. See the LICENSE file for details. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

All source code in this and the underlying directories is subject to the terms of the Mozilla Public License, v. 2.0.

The core version of the code is located here: https://github.com/OS2bos/os2bos/.