Use TravisCI + Saucelabs to run seltest

[ihodes]

This change has two main parts:

1. ./scripts/update-screenshots.sh starts a test server locally, and then updates the seltest screenshots via a browser on SauceLabs
2. ./scripts/travis-run-pdiff-tests.sh runs the screenshot tests on Travis via a browser on SauceLabs

Fixes #644.

To do:

• Run seltest tests on SauceLabs from TravisCL.
• Run screenshot updates with a local Cycledash test server in a remote browser via SauceLabs
• Figure out how to handle Sauce Labs credentials locally (right now require SAUCE_USERNAME and SAUCE_ACCESS_KEY to be in ENV, but we don't want this in the repo (other than encrypted, like they are in the travis-test.sh script).
• Use hammerlab credentials.
• Start sauce connect in a better way (no sleep allowed—Travis seems to do this well).
• Update docs including information about SauceLabs.
• Upload failed screenshots to Imgur.

filed #759

[danvk]

Just a few comments. Thanks for putting in all the work to figure this out! The vertically truncated screenshots issue is unfortunate, but hopefully we can find a workaround.

---

Reviewed 29 of 29 files at r1.
Review status: all files reviewed at latest revision, 6 unresolved discussions, all commit checks successful.

---

^{.travis.yml, line 11 [r1] (raw file):}
say something about where these came from

---

^{cycledash/static/scss/style.scss, line 31 [r1] (raw file):}
Is this a merge issue or an intended change?

---

^{DEVELOP.md, line 157 [r1] (raw file):}
consistency (sp)

---

^{scripts/travis-run-pdiff-tests.sh, line 3 [r1] (raw file):}
on Travis→on a Travis worker, as written this could be read as "this kicks off a Travis job".

---

^{scripts/travis-run-pdiff-tests.sh, line 26 [r1] (raw file):}
What does the error look like if either of these environment variables doesn't exist? Might be clearer to add an explicit check for them at the top of the script, like you do in update-screenshots.sh.

Speaking of which, how do these get set?

---

^{tests/pdifftests/test.py, line 127 [r1] (raw file):}
removed→removes

---

Comments from the review on Reviewable.io

[ihodes]

I agree. Chrome team has been punting on this for years, though. One solution may be to use Firefox. We can also try running Chrome in a headless GUI.

---

Review status: all files reviewed at latest revision, 2 unresolved discussions, all commit checks successful.

---

^{cycledash/static/scss/style.scss, line 31 [r1] (raw file):}
merge, nixed

---

^{scripts/travis-run-pdiff-tests.sh, line 26 [r1] (raw file):}
The error is super explicit (says the keys were not set/set to none).

They're set in the .travis.yml file (now commented), so they'll always be there.

---

Comments from the review on Reviewable.io

[danvk]

LGTM, feel free to merge on green.

After this goes in, can you send a quick message to everyone who works on Cycledash summarizing what we need to do (i.e. register for Sauce Labs, set environment variables) to stay up to date?

---

Review status: 24 of 29 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending.

---

Comments from the review on Reviewable.io