Add cross browser testing #3198

Merged
merged 6 commits into from Oct 25, 2016

Projects

None yet

6 participants

@denar90
Member
denar90 commented Sep 24, 2016

Proposed changes

Link to the issue:
#2353

Sometimes test are not passed because of ngrok failing with error - to many connections.
Another questions about platforms. Which one do we need. Currently added:

    ["Windows 7", "internet explorer", "9.0"],
    ["Windows 8", "internet explorer", "10.0"],
    ["Windows 8.1", "internet explorer", "11.0"],
    ["Windows 10", "MicrosoftEdge", "latest"],
    ["Windows 10", "chrome", "latest"],
    ["Windows 10", "firefox", "latest"],
    ["OS X 10.11", "chrome", "latest"],
    ["OS X 10.11", "firefox", "latest"],
    ["OS X 10.11", "safari", "9.0"]
@denar90 denar90 added the in progress label Sep 24, 2016
package.json
@@ -54,6 +55,7 @@
"backbone": "1.2.1 - 1.3.x",
"chai": "3.4.0",
"chai-jq": "0.0.9",
+ "easy-sauce": "^0.3.5",
@rafde
rafde Sep 25, 2016 Member

would it be ideal to lock down the version in case of something randomly breaking

@denar90
denar90 Sep 25, 2016 edited Member

Sounds reasonable.

test/setup/browser.js
- global.mocha.run();
- require('./setup')();
+ var runner = global.mocha.run();
+ require('./setup')(runner);
@rafde
rafde Sep 25, 2016 Member

strange, the CI doesn't seem to get runner passed to it.

@denar90
denar90 Sep 25, 2016 Member

Fixed this one.

@coveralls

Coverage Status

Coverage remained the same at 100.0% when pulling 825e630 on denar90:cross-browser-testing into 7a2af5d on marionettejs:master.

@denar90
Member
denar90 commented Sep 25, 2016

Found out why problems with too many connections appear. easy-sauce lib ping status of each test to know if it's ok or not. If we have to many platforms for testing then ping requests increase (in our case to ~70). ngrok allow only 20 per 1 minute. Try to solve this issue by changing easy-sauce API - philipwalton/easy-sauce#7
Another thing is that easy-sauce supports node >v6 ... Probably we should say no for using it, because we run our CI on different node env, lower then v6...

@paulfalgout
Member

@denar90 #3199 open an issue about Node with the issues around upgrading there.

@rafde
Member
rafde commented Sep 26, 2016

So no version lock down of "easy-sauce": "^0.3.5"?

@denar90
Member
denar90 commented Sep 26, 2016

@rafde I think we should wait until new version with fixes for stuff I wrote above will be added. Then of course lock version.

@rafde
rafde approved these changes Sep 27, 2016 View changes

👍

@denar90 denar90 referenced this pull request Oct 17, 2016
Merged

Use node v6 #3234

denar90 added some commits Sep 24, 2016
@denar90 denar90 add sauce runner af8091e
@denar90 denar90 improve mocha results processing 26bfbd8
@denar90 denar90 update cross browser testing
04e7f75
@coveralls

Coverage Status

Coverage remained the same at 100.0% when pulling 04e7f75 on denar90:cross-browser-testing into 7f31924 on marionettejs:master.

@denar90 denar90 cleanup
4562059
@coveralls

Coverage Status

Coverage remained the same at 100.0% when pulling 4562059 on denar90:cross-browser-testing into 7f31924 on marionettejs:master.

@denar90
Member
denar90 commented Oct 22, 2016

First, I removed livereload for test runner.html page (it was not used anyway).
Second, badge from sauce labs was added, so readme will have a bit updated look https://monosnap.com/file/hDOvt131ydbmwDeaBGfhqul1qtDnKT
Third, TravisEnvironment Variables with saucelabs credentials were added for running cross browser testing.

Moreover, I wanna say thanks to @philipwalton for reviewing/improving/releasing easy-sauce.

@scott-w

Looks great 👍

gulp/test-browser.js
+ name: 'Marionette.js',
+ username: process.env.SAUCE_USERNAME,
+ key: process.env.SAUCE_ACCESS_KEY,
+ tests: '/test/runner.html',
@philipwalton
philipwalton Oct 22, 2016

This is supposed to be testPath in easy-sauce 0.4.0, so either this isn't working or I did something wrong in my recent upgrade. Let me check.

@denar90
denar90 Oct 22, 2016 Member

I tested with all 3 services, everything was fine.

@philipwalton
philipwalton Oct 22, 2016

Yeah, looks like I missed a few instances in my find-and-replace. I've fixed it and released easy-sauce 0.4.1 with the changes.

@denar90
denar90 Oct 23, 2016 Member

Thanks, updated.

@paulfalgout
Member

I removed livereload for test runner.html page (it was not used anyway).

Pretty sure it is used. It you run gulp test-browser it'll spawn a browser window.. then if you make any chances to the code or the tests, the browser will run the tests again. Is there no way to keep this for development?

@denar90 denar90 revert livereload removing, update easy-sauce version
edcbeba
@denar90
Member
denar90 commented Oct 23, 2016

@paulfalgout I got back livereload.

@coveralls

Coverage Status

Coverage remained the same at 100.0% when pulling edcbeba on denar90:cross-browser-testing into 7f31924 on marionettejs:master.

gulp/test-browser.js
@@ -43,47 +44,47 @@ function bundle() {
exclude: 'node_modules/**'
})
]
- }).then(function (bundle) {
+ }).then(bundle => {
@scott-w
scott-w Oct 23, 2016 Member

Strictly speaking you don't need the excess { and return here:

.then(bundle => bundle.write({
...
})).then(...);
@denar90
denar90 Oct 23, 2016 Member

Thanks 👍

@denar90 denar90 cleanup
6956a36
@coveralls

Coverage Status

Coverage remained the same at 100.0% when pulling 6956a36 on denar90:cross-browser-testing into 7f31924 on marionettejs:master.

@denar90 denar90 removed the in progress label Oct 24, 2016
@paulfalgout paulfalgout merged commit 6a72aff into marionettejs:master Oct 25, 2016

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage remained the same at 100.0%
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment