Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
[Spike] Automated visual regression testing #458
Spike into automated visual regression testing
It's possible to deploy to a Heroku review app and to use a
I tried putting the generated gallery into the /public directory so the Rails app would be able to render it and so we'd have a consistent URL to check the output of. However – by running the script in the
Running wraith in a different context, ie on the core dyno, isn't possible. Using something like the rake task buildpack or by running a task on Rails initialise, these tasks always run before the app is available at the specified Heroku URL, or before phantomJS has been installed by Heroku.
This will be an issue for whatever visual regression tool we use if we run it on Heroku – either as part of the deployed review app or a separate app running elsewhere on Heroku that somehow gets notified of the review app’s deployment (this is probably possible using Github deployment webhooks)
Spectre with client is a tool for managing and viewing screenshots. You can set up runs and post screenshots to each run. It's supposed to also run on Heroku but there are reports of images being deleted when doing so. I also can't see how its secure – ie anyone can post any screenshot, from what I can tell. There are no tokens.
percy.io was a suggested alternative. Their pricing will be an issue. For each component we will be generating 3 images per fixture, per environment, per PR/PR sync. That's ~200 screenshots per run. Running the tests against a set of live content pages will be a further ~200 images per run. We will quickly get through a quota of snapshots.
Something like a