Unable to connect to older browsers on SauceLabs

[jhildenbiddle]

This is happening with multiple repos, all of which previously worked without issue.

• macOS 10.15.5
• node v12.15.0
• karma v5.1.0
• karma-sauce-launcher v4.1.5
• mocha v4.1.0 (req'd for IE9 tests)
• chai v4.2.0

Connections are made as expected when later versions of Chrome, Edge, Firefox, IE, and Safari are used (although with a deprecation warning as mentioned in #209). Easier versions of every browser fail consistently. For example, here is the console output from testing with Chrome 26 on Windows 10:

16 06 2020 11:53:18.572:INFO [karma-server]: Karma v5.1.0 server started at http://0.0.0.0:9877/
16 06 2020 11:53:18.573:INFO [launcher]: Launching browsers sl_chrome with concurrency unlimited
16 06 2020 11:53:18.583:INFO [launcher]: Starting browser chrome 26.0 (Windows 10) on SauceLabs
(node:28716) DeprecationWarning: Got: "options.rejectUnauthorized" is now deprecated, please use "options.https.rejectUnauthorized"
16 06 2020 11:54:18.589:WARN [launcher]: chrome 26.0 (Windows 10) on SauceLabs have not captured in 60000 ms, killing.
16 06 2020 11:54:18.590:ERROR [SaucelabsLauncher]: Could not quit the Saucelabs selenium connection. Failure message:
16 06 2020 11:54:18.590:ERROR [SaucelabsLauncher]: TypeError: Cannot read property 'deleteSession' of null
    at SaucelabsLauncher.<anonymous> (/Users/john/Dev/canvas-size/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:77:26)
    at Generator.next (<anonymous>)
    at /Users/john/Dev/canvas-size/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/john/Dev/canvas-size/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:4:12)
    at SaucelabsLauncher.<anonymous> (/Users/john/Dev/canvas-size/node_modules/karma-sauce-launcher/dist/launcher/launcher.js:74:31)
    at SaucelabsLauncher.emit (events.js:223:5)
    at SaucelabsLauncher.EventEmitter.emit (domain.js:475:20)
    at SaucelabsLauncher.emitAsync (/Users/john/Dev/canvas-size/node_modules/karma/lib/events.js:52:10)
    at SaucelabsLauncher.BaseLauncher.kill (/Users/john/Dev/canvas-size/node_modules/karma/lib/launchers/base.js:61:27)
2020-06-16T16:56:51.472Z ERROR webdriver: Request failed with status 400 due to Error: Missing or invalid 'desiredCapabilities'
2020-06-16T16:56:51.473Z ERROR webdriver: Error: Missing or invalid 'desiredCapabilities'
    at getErrorFromResponseBody (/Users/john/Dev/canvas-size/node_modules/webdriver/build/utils.js:121:10)
    at WebDriverRequest._request (/Users/john/Dev/canvas-size/node_modules/webdriver/build/request.js:148:56)
    at processTicksAndRejections (internal/process/task_queues.js:94:5)
    at async startWebDriverSession (/Users/john/Dev/canvas-size/node_modules/webdriver/build/utils.js:41:16)
    at async Function.newSession (/Users/john/Dev/canvas-size/node_modules/webdriver/build/index.js:44:23)
    at async Object.remote (/Users/john/Dev/canvas-size/node_modules/webdriverio/build/index.js:66:20)
16 06 2020 11:56:51.473:ERROR [SaucelabsLauncher]: Error: Failed to create session.
Missing or invalid 'desiredCapabilities'
    at startWebDriverSession (/Users/john/Dev/canvas-size/node_modules/webdriver/build/utils.js:45:11)
    at processTicksAndRejections (internal/process/task_queues.js:94:5)

The error descriptions read to me like errors with karma-sauce-launcher:

16 06 2020 11:54:18.590:ERROR [SaucelabsLauncher]: Could not quit the Saucelabs selenium connection. Failure message:
16 06 2020 11:54:18.590:ERROR [SaucelabsLauncher]: TypeError: Cannot read property 'deleteSession' of null

2020-06-16T16:56:51.472Z ERROR webdriver: Request failed with status 400 due to Error: Missing or invalid 'desiredCapabilities'
2020-06-16T16:56:51.473Z ERROR webdriver: Error: Missing or invalid 'desiredCapabilities'

16 06 2020 11:56:51.473:ERROR [SaucelabsLauncher]: Error: Failed to create session.
Missing or invalid 'desiredCapabilities'
    at startWebDriverSession (/Users/john/Dev/canvas-size/node_modules/webdriver/build/utils.js:45:11)
    at processTicksAndRejections (internal/process/task_queues.js:94:5)

I was able to come up with a list of browser/platform combinations that work reliably. Any browser version earlier than the ones listed below will fail to connect.

• Chrome 75 / Windows 10
• Edge 13 / Windows 10
• Firefox 55 / Windows 10
• IE10 / Windows 7
• Safari / OS X 10.13

One thing I noticed on SauceLabs' Platform Configurator page is an option to select either the "WebDriver (W3C) - Selenium 3/4, Webdriver.io" or "Legacy - Selenium 2/3" API.

Perhaps the issue is related a configuration for the new API being incorrectly being sent to browser/platform combinations that are require the older API?

Thanks!

[longlho]

I got the same issue, last working version was 4.1.4

[windwinkel]

For Version 4.1.5 this change was made:
7e75e17?branch=7e75e171d695e07af2c4c2e207a5dc579abcfb4e&diff=split

It seems to me, that sauce:options is always passed to sauce labs with this change.
According to sauce labs however, when sauce:options is passed, it always assumes that the new protocol W3C is used. However older browsers need to use the legacy JSONWP protocol instead.

[web-padawan]

Major dependency bump in a patch release is not that the users expect.

@rcebulko please consider reverting #207 on master and releasing 4.1.6.
Then there should be 5.0.0 major release including saucelabs bump.

[rcebulko]

Hi @web-padawan ,

Our project has moved away from Sauce Labs, and I am no longer actively involved on this repo. I briefly joined to try and fix some issues, but was unable to do so given the commits built up over a couple years without release. I believe I still have merge rights, so if you'd like to submit PR rolling by I'll approve it

[christian-bromann]

Major dependency bump in a patch release is not that the users expect.

@web-padawan I don't see major dependency bumps problematic as long as they don't change the behavior of the software. In this case it actually did for people that still test on old browser that don't support the WebDriver protocol. I see two options:

1. apply the sauce:option capability only if provided capabilities suggest new browser
2. revert all changes up until v4.1.4, make a new patch release, reapply changes and release as v5

What would you all suggest?

[web-padawan]

Thank you for your feedback!

apply the sauce:option capability only if provided capabilities suggest new browser

Sounds goos to me in case if this would help to fix the errors with older browsers 👍

[zyv]

@web-padawan I don't see major dependency bumps problematic as long as they don't change the behavior of the software. In this case it actually did for people that still test on old browser that don't support the WebDriver protocol. I see two options:

1. apply the sauce:option capability only if provided capabilities suggest new browser

This sounds like a great solution to me @christian-bromann !

[pchorus]

@christian-bromann Sounds great!

[wswebcreation]

New release that fixes this will come asap

[karmarunnerbot]

🎉 This issue has been resolved in version 4.3.1 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)

Your semantic-release bot 📦🚀

[jhildenbiddle]

Great to see this fixed. Thank you!