Consider integrating visual regression testing

[cvrebert]

Via e.g.

• https://github.com/bfirsh/needle (uses Selenium)
• https://github.com/facebook/huxley (uses Selenium, image diff-er might need work)
• http://crossbrowsertesting.com (many browsers, $27/month on yearly plan)
• https://github.com/Huddle/PhantomCSS (Firefox + Chrome, uses PhantomJS, supposedly supports screen width tests)
• https://github.com/bbc-news/wraith (Firefox + Chrome, uses PhantomJS, supports screen width tests)
• http://browsershots.org (not fully automated, uses Selenium)
• https://npmjs.org/package/grunt-photobox (Chrome, uses PhantomJS, supports screen width testing)
• https://github.com/cburgmer/csscritic (Chrome, uses PhantomJS)
• https://github.com/bslatkin/dpxdt
• https://github.com/garris/BackstopJS

For storing the images, http://git-annex.branchable.com/special_remotes/ ?

For visual diff inspection, http://huddle.github.io/Resemble.js/ ?

[mdo]

Still thinking we need this or should we pass? Seems the options have been rather limited since originally opening this issue.

[cvrebert]

It's still on my radar, but there's no definite timeline. Punted to the next milestone for now.

[cvrebert]

Okay, we have https://github.com/cvrebert/grunt-sauce-screenshots to take the screenshots, and a Python script to upload them to S3.
Now we just need a web app to run the comparisons and let humans review+approve differences as they arise.

[XhmikosR]

Maybe https://github.com/giakki/chai-resemble ?

[cvrebert]

@XhmikosR (a) That doesn't seem to be cross-browser (b) We just need generic image comparison, since we already have something to take the screenshot images in the first place.

[XhmikosR]

That compares 2 pages itself; it runs in node.js. I haven't used it myself just thought it might help.

[mdo]

Anyone want to own this? If not, I'll say fuck it and pass.

[cvrebert]

I own this. Gonna punt it to post-3.2.

[mdo]

My man. <3

[thomaswelton]

I'd be interested to see which direction you go with this.
On development on a new company frontend toolkit we have visual regression testing where we are using https://github.com/Huddle/PhantomCSS in full. And also using PhantomCSS just to do the visual diff comparisons (the screenshots are taken in java using Selenium)

But https://github.com/cvrebert/grunt-sauce-screenshots looks good.

[mdo]

Closing this because it's been forever and it's a nice-to-have thing.

[markelog]

Prompted by discussion in jquery chassis repo - jquery-archive/css-chassis#25

It seems @cvrebert and i indentified two candidates for this task: Gemini and WebdriverCSS. Both provide same amount of features, but in different ways.

@mdo if you still interested in this and if @cvrebert doesn't mind, i'd like to volunteer to try my luck with gemini. If implementation would be successful, it could create a good example of how this could be done for chassis and topcoat frameworks.

You can get the idea of the end result with this repo:
config - https://github.com/bem/bem-components/blob/v2/.gemini.yml
test example - https://github.com/bem/bem-components/blob/v2/gemini/button.gemini.js
screenshots example - https://github.com/bem/bem-components/tree/v2/gemini/screens/button

What do you think?

[cvrebert]

I don't mind at all.