Error: read ECONNRESET

[Gcaufy]

I have a the code like this.

var lists = [.......];  // There are 78 items in the array
let execFunc = (i) => {
    if (i === lists.length)
        return;
    rp.get('http://xxxx/get/' + lists[i]).then((d) => {
        execFunc(i + 1);
    });
};
execFunc(0);

and I get the error:

Unhandled rejection RequestError: Error: read ECONNRESET
    at new RequestError (/Users/apple/mine/code/project/chong_detection/node_modules/request-promise-core/lib/errors.js:14:15)
    at Request.plumbing.callback (/Users/apple/mine/code/project/chong_detection/node_modules/request-promise-core/lib/plumbing.js:87:29)
    at Request.RP$callback [as _callback] (/Users/apple/mine/code/project/chong_detection/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at self.callback (/Users/apple/mine/code/project/chong_detection/node_modules/request/request.js:187:22)
    at emitOne (events.js:90:13)
    at Request.emit (events.js:182:7)
    at Request.onRequestError (/Users/apple/mine/code/project/chong_detection/node_modules/request/request.js:813:8)
    at emitOne (events.js:90:13)
    at ClientRequest.emit (events.js:182:7)
    at Socket.socketErrorListener (_http_client.js:306:9)
    at emitOne (events.js:90:13)
    at Socket.emit (events.js:182:7)
    at emitErrorNT (net.js:1249:8)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

No idea what's wrong with it.

[analog-nico]

You have two issues you have to look at separately:

1. Error: read ECONNRESET is the primary cause and says that your connection/request got cancelled on the TCP/IP level. This may have many reasons but basically your request is configured correctly. It might just be the server you are sending the requests to which is resetting the connection. All you can do then is retrying the request. It might also be the general configuration of node.js but for that you would need to dig into the docs of the http module. Under the hood this module is used to send requests. Also see How to set max concurrent connections and a queue? #133 about the HttpAgent.
2. Unhandled rejection RequestError is an error that occurred afterwards because you are not catching the first error. What you need to do is the following – which is standard promise usage:

rp.get('http://xxxx/get/' + lists[i])
    .then((d) => {
        execFunc(i + 1);
    })
    .catch((err) => {
        // Handle the error here
    });

Hope this helps!

[analog-nico]

Hi @Gcaufy , I think it is safe to close this issue now. Let me know if you have more questions.

[Gcaufy]

Sorry, forget to close this issue. Seems I had fixed it, but I don't remember how. -.-!

[analog-nico]

No problem. :)

[Anaphase]

For people who stumble upon this issue looking for a solution, try adding the Connection: 'keep-alive' header to the request. I was having issues with a stupid server always instantly closing connections unless this header was specified.

[Fire-Brand]

@Anaphase adding Connection: 'keep-alive' header to the request it didn't work me, any other ideas?

[Anaphase]

¯\_(ツ)_/¯ could be any number of issues, try your luck on stack overflow?

[Fire-Brand]

I did, but could find anything that worked...

At the end that was just an issue on my local environment (mac), when it was up on the server (Ubuntu) everything worked fine.

[ankitappuria]

I am also getting the above issue. please anybody can help me how they solved it?

[Getitdan]

Just to add to @analog-nico's answer:

rp.get('http://xxxx/get/' + lists[i])
    .then((d) => {
        execFunc(i + 1);
    })
    .catch((err) => {
        if(err.message === 'read ECONNRESET'){
            // wait and rerun, or handle connection and rerun
        } else {
            throw err;
        }
    });

[cybairfly]

This issue was in my case caused by this missing request option: gzip: true

[mitchmike]

I had the same error. The cause for me was over-aggressive scraping of a site, hitting it with too many requests in a short space of time.
I found the below article very helpful. Limiting maximum concurrent sockets slows down your requests and makes your scraping a bit more gentle/responsible.
https://www.gregjs.com/javascript/2015/how-to-scrape-the-web-gently-with-node-js/

[LuisSevillano]

@mitchmike and @cybairfly thank you very much for your response! It took me hours to find a solution!

[thisisthemurph]

I found that the solution for me was including headers.
I used something like:

let options = {
  uri: baseurl,
  headers: {
    Connection: 'keep-alive',
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
  }
}