Test plans for load testing GOV.UK frontend apps using Gatling.
- Terminology
- Methods
2.1 GOV.UK Gatling
2.2 Virtual Machine
2.3 Rental of an enterprise Gatling Instance via AWS Marketplace - Configuration Options
- Simulation Plans
- Uploading
- Combining concurrent test results
- Troubleshooting
-
$GATLING_HOMEis the directory where Gatling is installed to. For example, if you download version 3.0.0-RC4 of the Gatling bundle zip and extract it in your~/Downloadsfolder,$GATLING_HOMEis~/Downloads/gatling-charts-highcharts-bundle-3.0.0-RC4 -
simulation plan: a set of scenarios, where each scenario represents how a user will interact with the website.
There are 3 main methods to install and run Gatling:
- terraform a gatling instance using GOV.UK tools and run Gatling
- install and run Gatling on a virtual machine which can reside on your laptop or a AWS instance
- rent an enterprise Gatling instance via AWS Marketplace
To provision a Gatling instance, deploy
Terraform
for app-gatling in the blue stack in the required environment.
-
SSH into the Gatling instance
-
Switch to the
gatlinguser and into the right directorysudo su - gatling cd ~/govuk-load-testing -
Export the required environment variables for your load test
-
Run Gatling with
/usr/local/bin/gatling/bin/gatling.sh -sf src/test/scala -
Learn what each simualtion plan tests and choose one:
Choose a simulation number: [0] govuk.Frontend [1] govuk.WhitehallPublishing [2] govuk.WhitehallPublishingCollections -
Now, the Gatling web UI is available at
https://gatling.<environment>.govuk.digital. There's an index of all Gatling runs made since provisioning, and you can see the results of a particular run.
Manually "stop" the Gatling instance in the AWS Console UI.
The installation steps are:
- Install a JDK, Gatling needs at least version 8
- Clone this repository into your
~/govuk/directory - Download and extract Gatling, these test plans are written for version 3
- Rename and move the extracted Gatling directory to
~/govuk/gatlingto make it available in/var/govuk/gatlingin your Virtual Machine - Set the needed environment variables:
-
$ export 'JAVA_OPTS=<required-options>'(see Configuration Options) -
$ export 'GATLING_USERNAME=<test-user-email>' -
$ export 'GATLING_PASSWORD=<test-user-password>'Note:
GATLING_USERNAMEandGATLING_PASSWORDare only needed for test plans using Signon. These are the credentials for the test user that has been set up in Signon in the environment it's going to test in, e.g.staging.
In order to run a simulation plan, Gatling provides a wrapper script to compile and launch test plans in its user-files directory.
You can run Gatling by:
- running the following command:
$ $GATLING_HOME/bin/gatling.sh -sf src/test/scala
Note: This command must be run from within ~/govuk/govuk-load-testing (or /var/govuk/govuk-load-testing if you are using the Virtual Machine).
- selecting the simulation plan number that you wish to run.
Choose a simulation number:
[0] govuk.BusinessReadinessFinder
[1] govuk.DynamicLists
[2] govuk.DynamicListsEmailSignup
[3] govuk.Frontend
[4] govuk.PublishToPublishingApi
[5] govuk.WhitehallPublishing
[6] govuk.WhitehallPublishingCollections
A description of relevant simulation plans available for the gov.uk website is available here
Note: Gatling FrontLine instances cost $9/hour, so it's important to switch off the instance while it's not in use.
To launch a Gatling instance, follow the instructions below. You may find there is already an existing Gatling instance available in EC2, in which case you can just start it without having to create a new one.
- First make sure you are logged in to AWS and have switched your role to production
- Go to "AWS Marketplace Solutions" from the services list (you may need to switch region to N. Virginia).
- Select "Manage" on Gatling FrontLine, click on "Actions" and then "Launch new instance."
- Set the region to "eu-west-1" and click "Continue to Launch".
- Select your desired EC2 Instance Type (recommended
t2.2xlarge). - Choose the
vpc-07069e8dd026cc725VPC,subnet-00103e6927dd1fb36subnet andgovuk_gatling_accesssecurity group. - Click "Launch" and you will be given a link to the EC2 instance.
- Find the Public DNS name for that instance and go to it in your browser. It should provide you with a wizard to complete the set up of the instance.
- You may find it useful to rename the instance in AWS to
gatlingso you can find it again easily. - If running the test cross-environment (e.g. running a load test in staging from a production EC2 instance), edit the
govuk_cache_external_elb_accesssecurity group in the environment you are testing to permit access on port 443 from the public IP address of your Gatling EC2 instance.
Once you have a Gatling FrontLine EC2 instance, you can use it to load a plan.
- Click on "Create" at the top left to create your plan.
- Give it a name, and choose the classname that corresponds to the test plan class you want. For example,
govuk.Frontendfor Frontend.scala. See this section for a list of plans. - Click next and choose "Build from sources" (should be the default option). Enter
git clone https://github.com/alphagov/govuk-load-testing.gitinto the repository command box, chooseSBT Projectin the build command drop down and click next. - If you're only using this one instance, choose the "Local" pool with a weight of 100%.
- Click on "More options" and here you can enter the
JAVA_OPTSvalue in the second box. For example, to use 100 workers, you would enter-Dworkers=100. Please see section for further configuration options. - Now you can click save and your plan should appear in the list. Click the play button to build it.
- Once your plan has built, it will go ahead and run it for you. You can click on the icon of a graph to view live updating results from the load test.
We use Java properties to pass options to the script which we don't want to hard-code. These can be set using the JAVA_OPTS environment variable:
$ export JAVA_OPTS="-Dkey1=value1 -Dkey2=value2 ..."
The following property is required:
baseUrl, prepended to all requests, at the least it should include the scheme and domain name
The following properties are necessary depending on the environment or scenario:
username, the HTTP basic auth usernamepassword, the HTTP basic auth password
The property signonUrl and environment variables GATLING_USERNAME and GATLING_PASSWORD are required for scenarios authenticating with a signon application.
The following properties are optional:
dataDir(default: "src/test/resources/test-data"), the directory to look in for test data filesrateLimitToken(default: no header sent), the value of theRate-Limit-Tokenheaderworkers(default: 1), the number of threads making requestsramp(default: 0), the duration, in seconds, over which the workers are startedbust(default: false), whether to pass a unique cache-busting string with every request or notmaxTime(default: 3600), the longest a test can run, defaulted to 1 hour
The simulation plans for gov.uk are located in the src/test/scala directory of this repository
while their data files live in the src/test/resources directory.
-
govuk.BusinessReadinessFinder
Data files: business-readiness-paths.csv
Optional:
factor(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits, each worker requestsbase_pathceil(hits * factor / workers)times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.DynamicLists
Data files: get-ready-brexit-check_paths.csv
Optional:
factor(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits, each worker requestsbase_pathceil(hits * factor / workers)times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.DynamicListsEmailSignup
Data files: get-ready-brexit-check-email-signup_paths.csv
Optional:
factor(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits, each worker requestsbase_pathceil(hits * factor / workers)times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time. -
govuk.Frontend
Data files: paths.csv
Optional:
factor(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits, each worker requestsbase_pathceil(hits * factor / workers)times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time.If you are having difficulty running the entire test plan on a single machine within your desired duration, try splitting up the data file and running multiple instances of Gatling simultaneously on different machines.
-
govuk.PublishToPublishingApi
Requires:
BEARER_TOKENenvironment variables.Note: The Publishing API is not accessible from the outside, you'll need to set up a proxy into our infrastructure to connect to it from Gatling.
For example, for staging:
$ ssh publishing-api-1.staging -CNL 8443:publishing-api.staging.publishing.service.gov.uk:443
You'll then need to add the line
127.0.0.1 publishing-api.staging.publishing.service.gov.ukto/etc/hoststo make sure the HTTPS server name matches.Then you can set your
baseUrltohttps://publishing-api.staging.publishing.service.gov.uk:8443.Since the load testing is now limited by your SSH proxy, you may want to increase the
ulimitto a high number such asulimit -n 4096. This means SSH is able to cope with more open sockets.Steps:
- Put content
- Patch links
- Publish
-
govuk.WhitehallPublishing
Requires:
signonUrlproperty.GATLING_USERNAMEandGATLING_PASSWORDenvironment variables.Optional:
scheduleproperty will schedule publication.
This value must be a timestamp in the formatyyyy-MM-ddTHH:mm(eg.2019-01-10T17:30). The value must be at least 15 minutes before the test run as Whitehall enforces this rule for scheduled publishing.Optional:
maxTryAttemptproperty will set how many times each step will be attempting before logging as a failure. This is to stimulate a user refreshing or reattempting after receiving an error response. If nothing is passed, it will default to 1.Example:
$ export JAVA_OPTS="-DbaseUrl=https://whitehall-admin.staging.publishing.service.gov.uk/ -Dworkers=1 -DsignonUrl=https://signon.staging.publishing.service.gov.uk/"Steps:
- Authenticates with signon
- Drafts a publication
- Attaches an HTML attachment
- Tags to taxonomy
- Force publishes or force schedules
-
govuk.WhitehallPublishingCollections
Requires:
signonUrlproperty.GATLING_USERNAMEandGATLING_PASSWORDenvironment variables.Optional:
documentSearchesproperty - How many document searches to make when adding to the collection.Steps:
- Authenticates with signon
- Drafts a collection
- Searches for Gatling Test publications
- Adds search results to collection
- Tags to taxonomy
- Force publishes
-
govuk.Search
Data files: search.csv
Optional:
factor(default: 1), the multiplier to apply to the amount of desired traffic For an entrybase_path,hits, each worker requestsbase_pathceil(hits * factor / workers)times, with no delay between requests. Each worker proceeds through the csv in order.Optional:
duration(default: 0), if set the test will last for the given number of seconds. Any workers that have not started or completed will be halted at this point. Conversely, any workers that finish ahead of this point will be restarted to ensure the test lasts for the given time.If you are having difficulty running the entire test plan on a single machine within your desired duration, try splitting up the data file and running multiple instances of Gatling simultaneously on different machines.
See how to upload results for more information on how to upload results so they are easy to access in the future.
It may be useful to run multiple concurrent Gatling tests to simulate more load than a single Gatling instance could generate. See how to combine results from concurrent tests.
-
My requests are being rate limited
Set the
rateLimitTokenproperty, and make sure the token is valid for the environment you're testing. These tokens live in the encrypted hieradata in govuk-secrets. -
Authentication issues
- Ensure the right permissions have been set for your test user
- Disable 2FA
GATLING_USERNAMEneeds to be an email address
-
No such file or directory
If you see an error such as:
15:16:48.524 [ERROR] i.g.a.Gatling$ - Run crashed java.io.FileNotFoundException: test-data/lorem-ipsum.txt (No such file or directory)Ensure you run gatling in this repo,
govuk-load-testing:export GATLING_HOME=/Users/username/gatling/ $GATLING_HOME/bin/gatling.sh -
Exit scenario
This may be useful to remove workers that fail at any step:
.exec{ //CODE }.exitHereIfFailed -
Output html
.exec(session => { val response = session("BODY").as[String] println(s"Response body: \n$response") session })