EADDRINUSE on connect #210

Closed
konsumer opened this Issue Mar 19, 2012 · 18 comments

Comments

Projects
None yet
10 participants
@konsumer

I am trying to write a little script to replicate an elasticsearch index to another elasticsearch index.

Basically, I need a HTTP connection to a source db, and to target db. I thought that my connections would be pooled automatically, but after a short time of running this code on 200,000 records I get a

{ [Error: connect EADDRINUSE] code: 'EADDRINUSE', errno: 'EADDRINUSE', syscall: 'connect' }

Here are the important bits of my code:
https://gist.github.com/2128623

What am I doing wrong?

@drypot

This comment has been minimized.

Show comment Hide comment
@drypot

drypot May 10, 2012

I get EADDRINUSE exception after 16000 request to elasicsearch server.

Using OS X, node 0.6.17.

Get similar error with superagent.js, too, after 16000 request.

node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: connect EADDRINUSE
at errnoException (net.js:670:11)
at connect (net.js:548:19)
at net.js:607:9
at Array.0 (dns.js:88:18)
at EventEmitter._tickCallback (node.js:192:40)

drypot commented May 10, 2012

I get EADDRINUSE exception after 16000 request to elasicsearch server.

Using OS X, node 0.6.17.

Get similar error with superagent.js, too, after 16000 request.

node.js:201
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: connect EADDRINUSE
at errnoException (net.js:670:11)
at connect (net.js:548:19)
at net.js:607:9
at Array.0 (dns.js:88:18)
at EventEmitter._tickCallback (node.js:192:40)

@drypot

This comment has been minimized.

Show comment Hide comment

@mikeal mikeal closed this Jul 25, 2012

@sylvain-hamel

This comment has been minimized.

Show comment Hide comment
@sylvain-hamel

sylvain-hamel Mar 6, 2013

I get the same error, only on Windows. After about 16000 requests, the program starts getting EADDRINUSE errors. I have both a Mac and a PC. The program runs fine on my Mac.

[Error: connect EADDRINUSE] code: 'EADDRINUSE', errno: 'EADDRINUSE', syscall: 'connect'

I don't set any network connection related options in my program. Any idea where to start the investigation?

I get the same error, only on Windows. After about 16000 requests, the program starts getting EADDRINUSE errors. I have both a Mac and a PC. The program runs fine on my Mac.

[Error: connect EADDRINUSE] code: 'EADDRINUSE', errno: 'EADDRINUSE', syscall: 'connect'

I don't set any network connection related options in my program. Any idea where to start the investigation?

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Mar 7, 2013

Owner

@isaacs was there an issue in the Agent by chance where we stepped on the socket in a strange way?

Owner

mikeal commented Mar 7, 2013

@isaacs was there an issue in the Agent by chance where we stepped on the socket in a strange way?

@sylvain-hamel

This comment has been minimized.

Show comment Hide comment
@sylvain-hamel

sylvain-hamel Mar 14, 2013

Hi, I updated to node 0.10.0 and request@2.14.0 and I still have the same issue. Should I log a new issue or can we reopen this one? Or is this issue with node?
Thanks

Hi, I updated to node 0.10.0 and request@2.14.0 and I still have the same issue. Should I log a new issue or can we reopen this one? Or is this issue with node?
Thanks

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Mar 14, 2013

Owner

the link above to the OS X issue wasn't the cause?

Owner

mikeal commented Mar 14, 2013

the link above to the OS X issue wasn't the cause?

@sylvain-hamel

This comment has been minimized.

Show comment Hide comment
@sylvain-hamel

sylvain-hamel Mar 14, 2013

No, it works fine on OSX. The problem is on Windows.

No, it works fine on OSX. The problem is on Windows.

@sylvain-hamel

This comment has been minimized.

Show comment Hide comment
@sylvain-hamel

sylvain-hamel Mar 14, 2013

I have just reproduced the problem without the request module. I'll cleanup my repro project and post it to the node project. Thanks

I have just reproduced the problem without the request module. I'll cleanup my repro project and post it to the node project. Thanks

@tp

This comment has been minimized.

Show comment Hide comment
@tp

tp Jan 27, 2014

@sylvain-hamel Do you have a bug for this already? I ran into similar issues on Azure (Windows), and would like to get to the bottom of it.

tp commented Jan 27, 2014

@sylvain-hamel Do you have a bug for this already? I ran into similar issues on Azure (Windows), and would like to get to the bottom of it.

@sylvain-hamel

This comment has been minimized.

Show comment Hide comment
@sylvain-hamel

sylvain-hamel Jan 27, 2014

@tp: No, I did not investigate the issue further. I really think it's specific to Windows. Back then I was trying to batch import hundreds of thousands of documents into a Couch database. The program was starting to fail after about 16000 requests to couch (documents). This was for a one-shot import for me, so I ended-up pausing the program for 5 minutes after each 15000 requests to let the OS do its thing. This solved the problem for me.

@tp: No, I did not investigate the issue further. I really think it's specific to Windows. Back then I was trying to batch import hundreds of thousands of documents into a Couch database. The program was starting to fail after about 16000 requests to couch (documents). This was for a one-shot import for me, so I ended-up pausing the program for 5 minutes after each 15000 requests to let the OS do its thing. This solved the problem for me.

@Torwori

This comment has been minimized.

Show comment Hide comment
@Torwori

Torwori Apr 19, 2014

I keep getting this error at around 3900 requests on Windows Server 2003 (and it isn't a one time import for me), any idea how I should handle this?

Torwori commented Apr 19, 2014

I keep getting this error at around 3900 requests on Windows Server 2003 (and it isn't a one time import for me), any idea how I should handle this?

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Apr 22, 2014

Owner

Is this the Windows version of file handlers being exceeded?

Owner

mikeal commented Apr 22, 2014

Is this the Windows version of file handlers being exceeded?

@Torwori

This comment has been minimized.

Show comment Hide comment
@Torwori

Torwori Apr 22, 2014

I solved my issue by making a batch of 3000 requests every 10 minutes. It's ugly, but it works.

Torwori commented Apr 22, 2014

I solved my issue by making a batch of 3000 requests every 10 minutes. It's ugly, but it works.

@tomhalley

This comment has been minimized.

Show comment Hide comment
@tomhalley

tomhalley Jan 7, 2015

Did anyone ever find a better work around for this? I'm currently sending a huge amount of requests to a node js server with request, and it too dies after about 16000 records. I'm working on windows, and after it times out at the 16000 mark, it takes almost exactly 100 seconds before it starts responding to requests again, even if i reboot the server.

It seems this is related

Did anyone ever find a better work around for this? I'm currently sending a huge amount of requests to a node js server with request, and it too dies after about 16000 records. I'm working on windows, and after it times out at the 16000 mark, it takes almost exactly 100 seconds before it starts responding to requests again, even if i reboot the server.

It seems this is related

@tomhalley

This comment has been minimized.

Show comment Hide comment
@tomhalley

tomhalley Jan 7, 2015

And this also seems to be related

I moved my node script that was acting as a client, as well as the server onto a linux virtual machine, and from there i seems to work fine. I can make any number of requests and this won't happen, confirming this is a windows issue.

I've also tested running the server on the linux virtual box and having the client running from my host machine, and it seemed to work without stopping at 16,000.

In summary, to anyone else who finds this issue through google, it's Windows. You will need to run your server from a Linux box. If you're testing a locally running server, I recommend using Vagrant to create a Linux box and run the server off of that.

And this also seems to be related

I moved my node script that was acting as a client, as well as the server onto a linux virtual machine, and from there i seems to work fine. I can make any number of requests and this won't happen, confirming this is a windows issue.

I've also tested running the server on the linux virtual box and having the client running from my host machine, and it seemed to work without stopping at 16,000.

In summary, to anyone else who finds this issue through google, it's Windows. You will need to run your server from a Linux box. If you're testing a locally running server, I recommend using Vagrant to create a Linux box and run the server off of that.

@hassy

This comment has been minimized.

Show comment Hide comment
@hassy

hassy Jul 22, 2016

This is most likely due to Windows running out of ephemeral ports.

To fix, set MaxUserPort to something like 65000 under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters

Source: https://docs.oracle.com/cd/E26180_01/Search.94/ATGSearchAdmin/html/s1207adjustingtcpsettingsforheavyload01.html

hassy commented Jul 22, 2016

This is most likely due to Windows running out of ephemeral ports.

To fix, set MaxUserPort to something like 65000 under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters

Source: https://docs.oracle.com/cd/E26180_01/Search.94/ATGSearchAdmin/html/s1207adjustingtcpsettingsforheavyload01.html

@danielstaleiny

This comment has been minimized.

Show comment Hide comment
@danielstaleiny

danielstaleiny Feb 7, 2017

I think another option is async library where you can select to wait for request.
async.eachLimit(collection,limit,function(item, callback){},function(errHandler(err)){})

I think another option is async library where you can select to wait for request.
async.eachLimit(collection,limit,function(item, callback){},function(errHandler(err)){})

@ericrini

This comment has been minimized.

Show comment Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment