Skip to content
This repository has been archived by the owner. It is now read-only.

Minimum viable component guide visual regression test #1

Merged
merged 7 commits into from Sep 18, 2017
Merged

Conversation

@fofr
Copy link
Contributor

@fofr fofr commented Sep 13, 2017

Steps

How to test locally

Fake a webhook, where deployment_payload.json is the file in 210c55a:

curl -X POST -d @deployment_payload.json http://www.dev.gov.uk:4567/run --header "Content-Type:application/json"
  • Rake tasks are also available with REVIEW_DOMAIN and LIVE_DOMAIN environment variables

Part of:
https://trello.com/c/iXflEn6F/106-3-build-automated-visual-regression-proof-of-concept

Follow on from:
alphagov/government-frontend#458
alphagov/government-frontend#472

cc @nickcolley

fofr added 5 commits Sep 11, 2017
* Create a simple sinatra app
* Respond to POST requests to /run
* Capture webhook JSON from Github
* Declare ruby in Gemfile for Heroku
* Heroku doesn’t pick up the `.ruby-version` file
* Update config and runner to take domain params
* Use friendly paths in gallery output
@fofr fofr requested a review from tijmenb Sep 13, 2017
We can’t easily get a list of URLs for a component guide. Instead use
the wraith spider feature and crawl the review domain’s component
guide. Exclude paths that don’t start with /component-guide and only
compare paths that end with `/preview` to limit the number of pages and
speed up the test run.

* Include rake tasks for running locally
* Use Heroku runner in app
@fofr fofr force-pushed the trigger-builds branch from 8b2bed1 to d1cf92a Sep 14, 2017
@fofr fofr temporarily deployed to govuk-visual-regression Sep 14, 2017 Inactive
@fofr
Copy link
Contributor Author

@fofr fofr commented Sep 14, 2017

This doesn't cope well with multiple jobs running in parallel:

  • Generated files are not unique per job
  • Multiple jobs exceed Heroku memory limits (memory bumped up)
* Keep number of pages to test down to a minimum
* Only test the pages that show all previews on one page

eg:
"/component-guide/contents-list/preview"
but not:
"/component-guide/contents-list/long_text/preview"
@fofr fofr temporarily deployed to govuk-visual-regression Sep 18, 2017 Inactive
@fofr
Copy link
Contributor Author

@fofr fofr commented Sep 18, 2017

Added e9ff6ac to speed up each component guide test run.

@fofr fofr merged commit 1edf9d1 into master Sep 18, 2017
@fofr fofr deleted the trigger-builds branch Sep 18, 2017
@tijmenb
Copy link

@tijmenb tijmenb commented Sep 18, 2017

💯 🎉

fofr added a commit that referenced this pull request Jan 12, 2018
Use PR description from PR #1 as README
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.