Don't recommend committing the baseline images in git

[necolas]

Commit/push these baseline images with your normal tests

This is going to result in a huge git repo, as git doesn't play well with binaries.

[jamescryer]

I don't know enough about Git and how it handles binaries to know how this approach will scale. I can however report that Huddle's PhantomJS test suite contains 252 baseline images, all in Git - with no problems to date.

Is there an alternative?

[necolas]

Every time you update an image, git stores a whole new binary in its history. So the git repo balloons over time. The bigger the app – and the more often it changes – the more it benefits from perceptual diff-ing, but then you have 1000's of baseline images that need to be updated many times a day.

I don't know what a robust solution would be. Having the tool generate the baseline locally each time is problematic because you might not be 100% sure that it's identical to the baseline other developers would generate. The images could be perhaps stored in an SVN repo instead.

[kpdecker]

Throwing in another data point:

We have a so-so solution that we've moved to. We're still using git but have a separate repo that matches our branching structure of the app repo. It's not ideal but it keeps the checkout time reasonable on the app repo and lets us continue to version the files.

We do not change our app nearly at the scale that @necolas is mentioning though.

Running all of the inputs through ImageOptim helps a bit as well.

Previous attempts at dropbox were an unmitigated disaster as we never really trusted that the files were safe and unmodified.

[jamescryer]

Interesting! Perhaps generating some kind of JSON formatted 'fingerprint' for comparison might help... But using a seperate repo is probably a better idea. Being able to see old and new screenshots is very very useful, like the hover(and click)overs in this experimental report http://huddle.github.io/PhantomFlow/demo/phantomFlowReport.

[jamescryer]

Tools like https://github.com/schacon/git-media or https://github.com/joeyh/git-annex might be of use?

[jamescryer]

I've updated the readme.