This repository has been archived by the owner on Nov 3, 2022. It is now read-only.
Fix E2E tests not working with Chrome 73.0.3683.75 #284
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://ci.integration.publishing.service.gov.uk/job/publishing-e2e-tests/job/test-against/20341/consoleFull
Chapter 1: The Long Day
Previously our E2E tests started failing with the following error.
This issue was limited to the CI agent machines. In order to debug the
issue, we SSH'd to ci-agent-8, became the jenkins user and changed to
one of the workspace directories where the E2E tests are run. We than
ran the E2E tests manually in order to reproduce the issue, as follows.
Once inside the container where the tests are run, we were able to
reproduce the issue with 'bundle exec rspec'. In order to investigate
further, we then installed vim, in order to install the irb gem and
start an irb console using 'bundle exec irb -Ispec'. Then we did
Running the visit method twice yields the same error as when running the
tests. Using 'docker exec' to start another bash console and inspect the
running processes shows that Chrome itself is failing to start.
Running chrome manually with the options from the spec_helper then
yields the following error, even though we hadn't changed these options.
Searching online for this error indicates its related to a new version
of Chrome, as per the following issue on the puppeteer repo.
Unfortunately it's not possible for us to downgrade Chrome, since Google
only provide the latest version in their package repo, and the E2E tests
are being run in transient containers, which have no older versions
available to downgrade to. This is the point where we lost all hope.
Chapter 2: The New Dawn
In Chapter 1 we experimented with running Chrome manually, based on
https://developers.google.com/web/updates/2017/04/headless-chrome.
The success of this command indicated one of the options specified in
the spec_helper was causing Chrome to crash, and experimentation showed
this was '--disable-dev-shm-usage'. Removing this parameter fixed the
error, but caused Chrome to crash for a different reason.
/dev/shm is a tmpfs partition, but by default it is only 64M in size.
Previously, we had specified the '--disable-dev-shm-usage' option to use
/tmp instead, but the new release of Chrome makes this option unusable
for some reason. The obvious remedy is to increase the size of /dev/shm.
The combination of removing the faulty option and specifying a larger
size for /dev/shm meant we could then run the E2E tests successfully.
And they all lived happily ever after.