Update dependency on selenium-webdriver to 3.7.0 or newer

[tbowmo]

I have been seeing a lot of random errors like

- Error: ECONNREFUSED connect ECONNREFUSED 127.0.0.1:62416
at ClientRequest.<anonymous> (C:\AzureAgentUI14\_work\4\s\src\Gandalf.Web\node_modules\selenium-webdriver\http\index.js:238:15)
at emitOne (events.js:116:13)
at ClientRequest.emit (events.js:211:7)
at Socket.socketErrorListener (_http_client.js:387:9)
at emitOne (events.js:116:13)

While exploring this error I found that it is selenium-webdriver that throws the error (as also noted in the error dump above). Protractor relies on selenium-webdriver@3.6.0.

However, in selenium-webdriver@3.7.0 they fixed this issue and implemented a more generic retry feature in case of any network issues with the webdriver socket.

Is there a possibility to update the dependencies on selenium-webdriver? As this would fix some of our problems with unstable tests.

[qiyigg]

I don't think there's a selenium-webdriver@3.7.0 for javascript, as far as I know, the next version is 4.0.0 which will break all the tests that are using control-flow.

[tbowmo]

ahh yes, I discovered that as well..

I solved it by using the patch-package npm package.. So we can apply a patch on our build agents on the fly, that forces selenium-webdriver to retry on the ECONNREFUSED error (as well as a couple of other errors).

[simonua]

@tbowmo, that sounds pretty interesting, and I may have a similar issue. Are you at liberty to share some of your solution, please? Thank you!

[tbowmo]

@simonua
I ended up using https://github.com/ds300/patch-package to patch the http client in selenium-webdriver. Basically just copying over this file to node_modules/selenium-webdriver/http, and create a patch using the patch-package module

You have to edit the index.js after you copied it over, as it doesn't do retries on ECONNREFUSED out of the box

[simonua]

@tbowmo, that's awesome! Thanks a lot for the information! I'll give it a go.

Cheers,

Simon

[rafalf]

@tbowmo can you post it how you patched it. I am having some problems

[tbowmo]

@rafalf what parts of the patching process are you having issues with?

[rafalf]

@tbowmo I copied the file, modified package and added the script line and then .. tried to run yarn to patch but got errors.

patch-package: you have both yarn.lock and package-lock.json
Defaulting to using npm
You can override this setting by passing --use-yarn or deleting
package-lock.json if you don't need it

☑ Creating temporary folder
☑ Building clean node_modules with npm
npm WARN deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated jade@0.26.3: Jade has been renamed to pug, please install the latest version of pug instead of jade
npm WARN deprecated to-iso-string@0.0.2: to-iso-string has been deprecated, use @segment/to-iso-string instead.
npm WARN deprecated minimatch@0.3.0: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated graceful-fs@3.0.11: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated minimatch@2.0.10: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated graceful-fs@1.2.3: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN prepublish-on-install As of npm@5, `prepublish` scripts are deprecated.
npm WARN prepublish-on-install Use `prepare` for build steps and `prepublishOnly` for upload-only.
npm WARN prepublish-on-install See the deprecation note in `npm help scripts` for more information.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! protractor@5.4.0 prepublish: `gulp prepublish`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the protractor@5.4.0 prepublish script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

16185 info lifecycle protractor@5.4.0~prepublish: protractor@5.4.0
16186 verbose lifecycle protractor@5.4.0~prepublish: unsafe-perm in lifecycle true
16187 verbose lifecycle protractor@5.4.0~prepublish: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB/node_modules/.bin:/usr/local/lib/node_modules/protractor/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
16188 verbose lifecycle protractor@5.4.0~prepublish: CWD: /private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB
16189 silly lifecycle protractor@5.4.0~prepublish: Args: [ '-c', 'gulp prepublish' ]
16190 silly lifecycle protractor@5.4.0~prepublish: Returned: code: 1  signal: null
16191 info lifecycle protractor@5.4.0~prepublish: Failed to exec prepublish script
16192 verbose stack Error: protractor@5.4.0 prepublish: `gulp prepublish`
16192 verbose stack Exit status 1
16192 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:285:16)
16192 verbose stack     at EventEmitter.emit (events.js:182:13)
16192 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
16192 verbose stack     at ChildProcess.emit (events.js:182:13)
16192 verbose stack     at maybeClose (internal/child_process.js:947:16)
16192 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:236:5)
16193 verbose pkgid protractor@5.4.0
16194 verbose cwd /private/var/folders/vp/f4s9240s0wx2t0r91vpcqdkc0000gn/T/tmp-60871hvF7dbpUr4JB
16195 verbose Darwin 15.6.0
16196 verbose argv "/usr/local/Cellar/node/6.7.0/bin/node" "/usr/local/bin/npm" "i"
16197 verbose node v10.0.0
16198 verbose npm  v5.6.0
16199 error code ELIFECYCLE
16200 error errno 1
16201 error protractor@5.4.0 prepublish: `gulp prepublish`
16201 error Exit status 1
16202 error Failed at the protractor@5.4.0 prepublish script.
16202 error This is probably not a problem with npm. There is likely additional logging output above.
16203 verbose exit [ 1, true ]

would you be able to post end to end how you did it

[tbowmo]

I can't really remember the exact steps that I did, but I believe that I forced it to use yarn (--use-yarn option), as that is our preferred package manager currently.

But install patch-package as per the description on their site, then update the files needed under selenium-webdriver, and then

yarn patch-package selenium-webdriver --use-yarn

That should create a patch file for selenium...

[rene-leanix]

Thanks for pointing us to the fix in selenium-webdriver, @tbowmo! All that's needed is the line added in this commit.

(I also wasn't able to use patch-package (with npm) for this, as it failed on a post-install script.)

[tbowmo]

@rene-leanix
In our case we needed to add ECONNREFUSED to the retry list as well, as we got frequent issues with that on our azure CI agents.

[simonua]

@tbowmo, same has been true for my organization.

[rafalf]

@simonua @rene-leanix @tbowmo How did you update this file?
I am on 5.4 protractor and looking at /usr/lib/node_modules/protractor/node_modules/selenium-webdriver/http/.index.js
and I dont have any of these functions:
shouldRetryRequest
isRetryableNetworkError

[tbowmo]

@rafalf I gave a recipe in reply no. 4 (if my calculations are right)