Too many ECONNRESET errors after upgrading to 8.12.0 from (8.11.4) #23237

kunalkukreja · 2018-10-03T07:06:16Z

After upgrading nodejs version to 8.12.0, we have seen multiple 'ECONNRESET' socket errors. Same calls are successful in 8.11.4.

This happens when there are bulk requests being executed parallelly. The calls at the other peer are successful.

Version: 8.12.4
Platform: 64-bit (Windows)
Subsystem: https/net/tls

These calls originate from nodejs application to rails server (behind nginx with self signed certificate).
Same calls gives success on retry. On debugging further the error is thrown by net module from this line https://github.com/nodejs/node/blob/v8.12.0/lib/net.js#L622

Script to reproduce the issue:

var request = require('request'), 
fs = require('fs'), 
path = require('path'), 
ca = fs.readFileSync(path.join( 'client', 'my-private-root-ca.cert.pem')) , 
url = process.argv[3] || 'https://xyz.com/some_api/sompath/15', 
ciphers = '!aNULL:kECDH+AESGCM:ECDH+AESGCM:RSA+AESGCM:kECDH+AES:ECDH+AES:RSA+AES';

var headers = { 
     'Accept-Version': '10160205',
     Cookie: '_session_id=7157182c19271bc66108644e2531303a; path=/; secure; HttpOnly',
     Accept: 'application/json' 
};
var reqOpts = {
    url: url,
    headers: headers,
    method: 'GET',
    strictSSL: false,
    time: true,
    gzip: true,
    ca: ca,
    ciphers: ciphers
};

function makeRequest(){
    request(reqOpts, function(e,r,b){
           if(e) console.log(e);
    })
}

for(var i = 0; i < 100; i++){ makeRequest(); }

Above script works fine on 8.11.4 but thows error on 8.12.0. See below errors

> for(var i = 0; i < 100; i++){ makeRequest(); }
undefined
> { Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
    at TLSWrap.onread (net.js:622:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }

I will share NODE_DEBUG logs for net, tls modules

addaleax · 2018-10-03T14:32:02Z

/cc @mafintosh @nodejs/http2

mcollina · 2018-10-04T07:17:15Z

Can you verify that the same behavior applies to the latest Node 10? We are in the process of backporting a lot of work on HTTP2 for Node 8, as the one on Node 8 was left behind.

A repro script would be extremely useful to track this down.

The behavior of that commit is correct: the socket is being abruptly interrupted. My question right now is how a ECONNRESET is being generated in the first place if all requests are arriving correctly.

kunalkukreja · 2018-10-06T08:22:08Z

Running it with NODE_DEBUG=tls,net mode shows following logs(only pasted relevant logs for brevity)

NET 4396: connect: find host xyz.com
NET 4396: connect: dns options { family: undefined, hints: 0 }
NET 4396: _read
NET 4396: _read wait for connection
NET 4396: afterConnect
TLS 4396: start
NET 4396: _read
NET 4396: Socket._read readStart
TLS 4396: secure established
NET 4396: afterWrite 0
NET 4396: afterWrite call cb
TLS 4396: secure established
NET 4396: afterWrite 0
NET 4396: afterWrite call cb
NET 4396: onread 771
NET 4396: got data
NET 4396: _read
NET 4396: _final: not ended, call shutdown()
NET 4396: onread -4077
NET 4396: destroy
NET 4396: close
NET 4396: close handle
NET 4396: afterShutdown destroyed=true ReadableState {
  objectMode: false,
  highWaterMark: 16384,
  buffer: BufferList { head: null, tail: null, length: 0 },
  length: 0,
  pipes: null,
  pipesCount: 0,
  flowing: true,
  ended: false,
  endEmitted: false,
  reading: true,
  sync: false,
  needReadable: true,
  emittedReadable: false,
  readableListening: false,
  resumeScheduled: false,
  destroyed: true,
  defaultEncoding: 'utf8',
  awaitDrain: 0,
  readingMore: false,
  decoder: null,
  encoding: null }

Note the folowing log which triggers socket destroy.

NET 4396: onread -4077

Let me know if more details are required from my side.

kunalkukreja · 2018-10-16T11:51:39Z

cc @nodejs Please update or add the concerned person who can help.

AyushG3112 · 2018-10-16T13:06:56Z

@kunalkukreja would you have a repro script by any chance?

kunalkukreja · 2018-10-16T13:08:35Z

@AyushG3112 see this comment #23237 (comment) for script

kunalkukreja · 2018-12-06T12:31:26Z

The issue persists in v8.14.0 too.

On debugging further the issue is being caused by gzip option in the request. Passing the header 'Accept-Encoding' = 'deflate' fixes the issue. Both 'gzip' and 'gzip, deflate' values throw the error. The default value set by request library for 'Accept-Encoding' is 'gzip, deflate'. Refer this

This happens only on windows.

bnoordhuis · 2018-12-07T14:04:13Z

It sounds like this is a server issue, if it doesn't like the 'gzip' value in the Accept-Encoding header.

Are you sure the Node.js version is the only factor? E.g., perhaps you also wittingly or unwittingly updated request when you upgraded Node.js and it now contains some changes that exacerbate it?

Can you reproduce standalone? I.e., no request, no third-party server?

gs-ajain · 2018-12-28T06:51:21Z

Facing this issue on Windows. Node - v8.12.0

My workaround: Upgraded Nodejs to v10.15.0 and the issue disappeared.

mindfulme · 2018-12-30T21:27:26Z

I have the same issue on node.js v8.12.0 and v10.15.0 too. I have client node.js application running through nginx trying to access public api wss://mainnet.infura.io/ws/. The initial connection is okay but the error appears on event streams

mindfulme · 2018-12-31T10:53:40Z

Actually, after deleting nginx completely the error disappeared, though I was using the node.js server as a client to receive data from a remote socket.

mindfulme · 2018-12-31T10:53:58Z

So the solution is deleting nginx in my case

Actually, after deleting nginx completely the error disappeared, though I was using the node.js server as a client to receive data from a remote socket.

luispeerez · 2019-03-05T04:46:18Z

@mindfulme what did you have in your nginx config? proxy config?

targos · 2020-06-13T09:16:17Z

I'm closing this issue because of its lack of recent activity. Feel free to reopen or ask that we reopen if you think that it should still be addressed.

addaleax added the http2 Issues or PRs related to the http2 subsystem. label Oct 3, 2018

This was referenced Oct 24, 2018

Angular project fails e2e on webdriver-manager step angular/webdriver-manager#329

Closed

update failing with --ignore_ssl in Node 8.12.0 angular/webdriver-manager#326

Open

Blackbaud-BobbyEarl mentioned this issue Feb 6, 2019

Testing Travis + Appveyor on Node 6, 8, and 10. blackbaud/skyux-builder#513

Closed

targos closed this as completed Jun 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Too many ECONNRESET errors after upgrading to 8.12.0 from (8.11.4) #23237

Too many ECONNRESET errors after upgrading to 8.12.0 from (8.11.4) #23237

kunalkukreja commented Oct 3, 2018 •

edited

addaleax commented Oct 3, 2018

mcollina commented Oct 4, 2018

kunalkukreja commented Oct 6, 2018

kunalkukreja commented Oct 16, 2018

AyushG3112 commented Oct 16, 2018

kunalkukreja commented Oct 16, 2018

kunalkukreja commented Dec 6, 2018 •

edited

bnoordhuis commented Dec 7, 2018

gs-ajain commented Dec 28, 2018 •

edited

mindfulme commented Dec 30, 2018

mindfulme commented Dec 31, 2018

mindfulme commented Dec 31, 2018 •

edited

luispeerez commented Mar 5, 2019

targos commented Jun 13, 2020

Too many ECONNRESET errors after upgrading to 8.12.0 from (8.11.4) #23237

Too many ECONNRESET errors after upgrading to 8.12.0 from (8.11.4) #23237

Comments

kunalkukreja commented Oct 3, 2018 • edited

addaleax commented Oct 3, 2018

mcollina commented Oct 4, 2018

kunalkukreja commented Oct 6, 2018

kunalkukreja commented Oct 16, 2018

AyushG3112 commented Oct 16, 2018

kunalkukreja commented Oct 16, 2018

kunalkukreja commented Dec 6, 2018 • edited

bnoordhuis commented Dec 7, 2018

gs-ajain commented Dec 28, 2018 • edited

mindfulme commented Dec 30, 2018

mindfulme commented Dec 31, 2018

mindfulme commented Dec 31, 2018 • edited

luispeerez commented Mar 5, 2019

targos commented Jun 13, 2020

kunalkukreja commented Oct 3, 2018 •

edited

kunalkukreja commented Dec 6, 2018 •

edited

gs-ajain commented Dec 28, 2018 •

edited

mindfulme commented Dec 31, 2018 •

edited