Skip to content
A configuration service for the Performance Platform
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.travis_scripts Silence output on push Mar 29, 2017
assets
doc/arch
etc
examples Update stagecraft port to 3103 Jul 21, 2015
log Paas migration Aug 7, 2017
requirements
stagecraft fix tests for q4 2017 Sep 21, 2018
tools Project upgrade May 2, 2017
.coveragerc
.gitignore Bump performanceplatform-collector to 0.2.7 Nov 3, 2015
.travis.yml Add the FLOWER_BASIC_AUTH secrets for staging and production to travis. Aug 21, 2018
LICENCE.txt Correct copyright notice Feb 16, 2015
README.md Remove GOV.UK VM stuff from the README. Apr 24, 2019
do_dashboard_migration.py Make migration script log correctly Sep 23, 2014
import_schemas.py Capture details about validation failures Jun 1, 2015
manage.py Move too-deep stagecraft/stagecraft directory up Jan 27, 2014
manifest.yml
one_off_migrate_to_transactions_by_channel.py Fix print for python 3 Jul 30, 2014
pep-it.sh
requirements.txt
run_development.sh Update stagecraft port to 3103 Jul 21, 2015
run_tests.sh Paas migration Aug 7, 2017
runtime.txt Upgrade the version of the Python buildpack to latest Apr 24, 2019
start-app.sh

README.md

Stagecraft

Build Status Code Health

Application for managing performance platform.

Installing

You'll need a virtualenv and if you're developing locally you'll want to use requirements/development.txt:

mkdir venv
virtualenv venv
source venv/bin/activate
pip install -r requirements/development.txt

Set Django environment

Because we use environment-specific settings, you need to specify which settings module to use in your environment. You'll need this for all commands you run through manage.py.

export DJANGO_SETTINGS_MODULE=stagecraft.settings.development

Initialising database

Before the server is run for the first time the database needs to be synced.

Note: The first time you migrate, you'll be asked to setup a superadmin.

python manage.py migrate

This project uses django-reversion to provide version control functionality. Whenever a model is added it should be registered with the VersionAdmin class (for an example, see here ), and

python manage.py createinitialrevisions

should be used to populate the version database with an initial set of model data. Depending on the number of rows in the database, this command can take a while to execute.

Need to setup a new superadmin?

python manage.py createsuperuser --username=<ADMIN_USER_NAME> --email=<YOUR_EMAIL_ADDRESS>

Need to delete all tables and start again?

./tools/drop-tables.sh

Adding a new model

After creating a model, create the initial migration script with:

python manage.py schemamigration [app_name] --initial

and then after making any changes to the model, create new migration scripts with:

python manage.py schemamigration [app_name] --auto

Apply migration scripts with:

python manage.py migrate [app_name]

Running

To run the development server:

python manage.py runserver 0.0.0.0:3103

or use the Pinfile in https://github.com/alphagov/pp-puppet/tree/master/development:

bowl stagecraft

You should now be able to access the admin control panel at http://stagecraft.development.performance.service.gov.uk/admin/.

Running tests

A convenience script for running tests is included. Remember to prefix this with the appropriate settings environment variable, e.g.:

DJANGO_SETTINGS_MODULE=stagecraft.settings.development ./run_tests.sh

More on reversions

NOTE: Reversions are not compatible with schema changes. Without special migrations, revisions saved before schema changes will be lost after these changes

In contexts other than the Admin app there are things you must keep in mind in order to get reversions:

  • When the operation is part of a request, the reversion.middleware.RevisionMiddleware middleware class will ensure that database modifying operations are accompanied by revisions on registered models (e.g. models with reversion.register(ModelName)).
  • NOTE: When doing a request you will need to test this - this is our current understanding but we could be wrong...

However

'Warning: Due to changes in the Django 1.6 transaction handling, revision data will be saved in a separate database transaction to the one used to save your models, even if you set ATOMIC_REQUESTS = True. If you need to ensure that your models and revisions are saved in the save transaction, please use the reversion.create_revision() context manager or decorator in combination with transaction.atomic().'

In Addition

Outside of requests (e.g. in scripts) you will need to use the following:

  • import reversion.
  • Ensure the model is registered with reversion.register(DataGroup) if used in a context where the admin registration has not run.
  • Run all database modifying operations in functions with the @reversion.create_revision() decorator or in the reversion.create_revision(): context manager.

See the docs for more info

Migrating to new data set schemas.

As more schemas are added to describe data types we may find ourselves needing to migrate data sets to these new schemas. This is the purpose of stagecraft/libs/mass_update/copy_dataset_with_new_mapping.py.

To use this define a mapping of the format:

mapping = {
    #finds an old data set to copy config and data from.
    'old_data_set': {
        'data_group': "carers-allowance",
        'data_type': "weekly-claims"
    },
    #a new data set with the data group and type here will be created
    #all config will be copied from the old data set
    #except for anything specified here which will override this config
    #(in this case this is the auto_ids field)
    'new_data_set': {
        'auto_ids': '_timestamp,channel',
        'data_group': "carers-allowance",
        'data_type': "transactions-by-channel"
    },
    'data_mapping': {
        #this specifies a mapping of keys in the old data set data which will be changed
        'key_mapping': {
            "key": "channel",
            "value": "count"
        },
        #this specifies a mapping of values in the old data set data which will be changed
        'value_mapping': {
            "ca_clerical_received": "paper",
            "ca_e_claims_received": "digital"
        }
    }
}

Run it with:

migrate_data_set(mapping['old_data_set'],
                 mapping['new_data_set'],
                 mapping["data_mapping"])

This will not delete the old data set. This should be done later when you are happy with the result.

Tests which further specify the behaviour of this can be found in test_copy_dataset_with_new_mapping.py.

You can’t perform that action at this time.