Smoke tests for GOV.UK
Gherkin Ruby Python Shell
Clone or download
Latest commit 8ffc20a Jun 18, 2018

GOV.UK Smoke Tests

Automated tests that describe high level user journeys which touch multiple applications within the GOV.UK stack.

These are used to verify releases and also to provide Nagios alerts for major features.

Technical documentation

The smoke tests are based on Cucumber. We use feature files to describe single applications (eg whitehall, frontend).

Running the tests

Run the suite with:

bundle exec rake

or against a single feature:

bundle exec cucumber features/frontend.feature

The tests run in/against the integration environment by default but require additional configuration to run successfully. This set of options should allow you to run the tests successfully from your development machine:

SIGNON_EMAIL="<email-address>" \
SIGNON_PASSWORD="<password>" \
AUTH_USERNAME="<username>" \
AUTH_PASSWORD="<password>" \
bundle exec cucumber \
--format pretty \
--tags ~@pending \
--tags ~@local-network \
--tags ~@notintegration

Test configuration

You can use the following environment variables to configure the tests:

    • Default: The value returned by plek for draft-origin.
    • Required by tests tagged with @draft.
    • Default: Blank
    • Used to construct URLs in the #application_base_url method.
    • Default: Blank
    • Set the HTTP Basic username required to access GOVUK_WEBSITE_ROOT.
    • Default: Blank
    • Set the HTTP Basic password required to access GOVUK_WEBSITE_ROOT.
    • Default: Blank
    • Email address of a user with a Signon account in the environment the tests are being run in.
    • Default: Blank
    • Password of a user with a Signon account in the environment the tests are being run in.

Spoof target domain

Set the SPOOF_TARGET_DOMAIN environment variable to run tests against a alternative URL, while maintaining the Host header as set by the GOVUK_WEBSITE_ROOT environment variable.

This may be useful when you wish to test against the FQDN of a site where the DNS does not yet resolve to the correct IP.

An alternative method would be to update /etc/hosts on the client running the tests.

Debugging the tests

Set the POLTERGEIST_DEBUG environment variable to see Poltergeist debug output when running the tests:

$ POLTERGEIST_DEBUG=true bundle exec rake

Adding new tests

Tests that are supposed to be run by icinga also have to be added to the file modules/monitoring/manifests/checks/smokey.pp in our Puppet repository. For example, the test frontend.feature is added to icinga like this:

icinga::check_feature {
  'check_frontend':          feature => 'frontend';
  #other feature tests

Prioritising scenarios

Because we integrate Nagios with the output from these tests, we provide a set of tags which match with how important we consider a scenario to be. @high and above will trigger pager alerts.

Each scenario can and should be prioritised by using the @urgent, @high, @normal or @low cucumber tags. For example, the frontend scenario "check guides load" can be prioritised like this:

Scenario: check guides load
  When I visit "/getting-an-mot/overview"
  Then I should see "Getting an MOT"


This master branch of this Smokey project is automatically deployed by Jenkins at about 9am each day.