Support for alternate browsers

[sheltonial]

Would be great to have the option of running tests in an alternate browser such as Chrome. Our application has reached a point where we have so many tests that PhantomJs crashes. Seems that many others are having this issue with PhantomJs also and the solution that has worked for us is to run via Chrome.

[ArtemGovorov]

We have had the same issue with one of our apps as well.
Though wallaby.js had solved it because it does the initial run in parallel, so there are much less tests to run in each individual PhantomJs instance.

Have you already encountered PhantomJs crashes when using wallaby.js for your tests?

Anyway, we do plan to support Chrome runner at some point.

[sheltonial]

Interesting, perhaps it's an issue with our tests. I'll look into it some more. Seems to be hangining around the the 1650th test. Turning debug on within the wallaby config reveals the following output:

Wed, 04 Mar 2015 00:04:56 GMT project Sandbox [rvgd1] is responsive, closing it
Wed, 04 Mar 2015 00:04:59 GMT project Test executed: should have "..." on scope
Wed, 04 Mar 2015 00:04:59 GMT project Test executed: should have "..." on scope
Wed, 04 Mar 2015 00:04:59 GMT project Test executed: should have "..." on scope
Wed, 04 Mar 2015 00:05:01 GMT project Signal killed phantomjs #1: null, exit code: 3221225477

[ArtemGovorov]

I have set up wallaby to run angular.js project tests (around ~3500), works ok. But I know there's an issue with phantomJs that you're talking about.

Before we add the support for Chrome runner, there's one thing you can try (apart from trying to find out whether there's something can be done with the tests).

As I have said, wallaby is using multiple instances of PhantomJs to run tests, especially initially. So, if you try to run your tests using more parallel instances initially - it may help. Have a look into http://wallabyjs.com/docs/config/workers.html and try to increase workers.initial setting, and then re-running wallaby. If your tests are not in a single huge file, it may help.

[sheltonial]

Increasing the initial workers has definitely helped. I suspect it could be a memory issue with Phantom processors as if I decrease the number of initial workers to 1 it fails much sooner. Tests are split up over approx 200 files. I'm finding now it completes the tests but fails merging tests results. Output is:

Wed, 04 Mar 2015 00:47:20 GMT project Run 143 test(s), skipped 0 test(s)
Wed, 04 Mar 2015 00:47:20 GMT project Sandbox [4mtc9] is responsive, closing it
Wed, 04 Mar 2015 00:47:20 GMT project Merging parallel test run results
Finished executing: 1839 tests
Wed, 04 Mar 2015 00:47:32 GMT project Test run finished
Wed, 04 Mar 2015 00:48:18 GMT project Failed to process test run results Invalid string length
Failed to process test run results Invalid string length

[ArtemGovorov]

Nice, we are getting somewhere. Did you set "instrument: false" for libraries and other large files (patterns), that you don't really edit much and don't need coverage for, in your files list? Not only it should speed up the tests execution, but significantly decreases the memory footprint.

[sheltonial]

Setting instrument to false for library files combined with increasing the number of initial workers has solved the problem. Thanks Artem

[ArtemGovorov]

Awesome, the issue with "Invalid string length" happens when wallaby tries to JSON.stringify a huge object with coverage to send it to IDE. Excluding libraries makes the object way more compact, as coverage for them is not collected anymore. You may probably also observe some test execution speed improvement.

If you don't mind, I'll keep the issue opened, as the title and the request for other browsers support is valid.

[jamesshore]

+1 for supporting alternate browsers (and better yet, multiple browsers; and better still, integrating with Karma).

[elgalu]

+1 for Chrome support, PhantomJS is buggy while using Protractor

[vlazar]

+1 for supporting multiple real browsers even if tests are not instant but "just" much faster, than say in karma

[cResults]

+1 for supporting multiple real browser chrome, IE, FF, etc. I agree with vlazar, the results of this doesn't need to be instant. Although they are rare, karma has pointed out tests that pass on many browsers, but fail on one (usually ie), so this is unfortunately a necessary requirement for us.