Build testing: Broken webdriver task

[eKoopmans]

There's a bug in the testing process with the webdriver task - nobody's pull requests are passing. I've tracked down the problem to initBrowser in tests/utils.js - the browser is never initialized (the .init(settings) promise never resolves). I believe that the selenium server that is needed to perform the tests is not being started.

The browser object is created using wd, which explains that:

WD.js does not start the selenium server. You may use the selenium-standalone package to install and start a selenium server.

Starting the selenium server

I've tried installing and running selenium-standalone on my machine, and after removing port: 4445 (line 35 of utils.js), I got the webdriver test to get a bit further - several browsers pop up on my machine before the test fails.

html2canvas tests using Travis, so I'm guessing this guide is relevant. This would make sense since utils.js uses the environment variables SAUCE_USERNAME and SAUCE_ACCESS_KEY as described in those instructions. Where I'm stuck is where they describe starting a web server, which I believe isn't happening in our tests. Just not sure how to set that up here.

It's a shame, I think many/most current PRs are failing just because of this problem!

Related issues/pull requests

#782, #941, #1084, #1085, #1086, #1087

[eKoopmans]

I found the problem - the webdriver task can't be performed on pull requests, because we don't have access to the Sauce Labs environment variables. This is by design - those variables are meant to be kept private - so the only solution is to bypass the webdriver task on pull requests.

#1089 solves the problem. If you have made a pull request and want it to pass the tests, I recommend rebasing off of that branch. Here's what you would do:

git fetch https://github.com/eKoopmans/html2canvas.git bugfix/webdriver
git checkout your-branch
git rebase bugfix/webdriver

That will place the bugfix/webdriver commit before your branch, which should fix the issue. You can then update your repo on GitHub:

git push -f

[niklasvh]

Moving onwards, the whole testing infrastructure should be rethought a bit. I would hope to move to more accurate and consistent automated tests and limit the webdriver tests purely just for parsing (as the rendering should be consistent if the parsed values are the same across browsers).

[niklasvh]

Fixed