Get EADDRNOTAVAIL after creating ~16.000 docs

[Nomeasmo]

I've no clue whats wrong here. I played around with some options but didn't succeed.

client = new elasticsearch.Client({
    host: 'localhost:9200',
    agent: true,
    maxSockets: 5,
    connectionClass: "http",
    keepAlive: false
    // ,log: 'trace'
});

Working in OSX, node 0.10.25, es 1.0.1

Elasticsearch ERROR: 2014-02-26T16:03:58Z
  Error: Request error, retrying -- connect EADDRNOTAVAIL
      at Log.error (/Users/oliver/Documents/node/splimporter/node_modules/elasticsearch/src/lib/log.js:213:60)
      at checkRespForFailure (/Users/oliver/Documents/node/splimporter/node_modules/elasticsearch/src/lib/transport.js:185:18)
      at HttpConnector.<anonymous> (/Users/oliver/Documents/node/splimporter/node_modules/elasticsearch/src/lib/connectors/http.js:138:7)
      at ClientRequest.bound (/Users/oliver/Documents/node/splimporter/node_modules/elasticsearch/node_modules/lodash-node/modern/internals/baseBind.js:52:17)
      at ClientRequest.EventEmitter.emit (events.js:95:17)
      at Socket.socketErrorListener (http.js:1547:9)
      at Socket.EventEmitter.emit (events.js:95:17)
      at net.js:441:14
      at process._tickCallback (node.js:415:13)

[spalger]

I believe it is because of the keepAlive: false in your config. This is going to cause new connections to be created for each request, I suggest using keepAlive.

[Nomeasmo]

Nope, I initially tried it with standard configuration. No difference with any keepAlive setting.

client = new elasticsearch.Client({
    host: 'localhost:9200',
});

[Nomeasmo]

Using the bulk works pretty well.

[spalger]

Yes, that's certainly a good idea if you have access to multiple requests at a time.

I'm almost certain that EADDRNOTAVAIL is directly related to opening and closing tcp sockets, but if you are not modifying the keepAlive/maxSockets/minSockets config then it should be opening 10 sockets and keeping them alive, no more, no less. As I walk through the forever-agent module I'm not sure that is the case. Do you have a script you can share that reproduces this?

[Nomeasmo]

This small code does the job

var elasticsearch = require('elasticsearch'),
    async = require('async'),
    _ = require('underscore'),
    client;

client = new elasticsearch.Client({
    host: 'localhost:9200'
});

async.eachSeries(_.range(20000), function (value, done) {
    client.create({index: 'tests', type: 'test', body: {something: true, value: value}}, done);
});

[spalger]

alright, I found the problem and submitted request/forever-agent#16. Working on a solution for the meantime.

[spalger]

Version 1.5.10 includes a fix that should force forever-agent to use the connection pool for all requests. If you have time to confirm I would appreciate it.

[smithatlanta]

I'm using 1.5.11 and I'm still getting this error. Did it get reintroduced?

[spalger]

Well, this is embarrassing. It looks like I didn't test the solution I committed and it didn't actually do what I thought it did, sorry about that.

Using @Nomeasmo's script I was able to reproduce the issue and create a verified fix. Creating a test case for this that can run automatically is pretty tricky, but it is on my todo list.

@smithatlanta or @Nomeasmo if either of you have time to verify that things are working for you too, I would really appreciate it.

https://gist.github.com/spenceralger/9550110

[smithatlanta]

That solved my issue. Loaded 1/4 of a million items with no issues. Thanks.

[spalger]

Glad to hear.

[philberg]

@spalger I am facing the exact same issue running OSX 10.10.2 , node v0.12.7, and ES 1.6.0

I am able to replicate the issue using the sample code from @Nomeasmo above. Could this issue needs to be resolved differently in 0.12 given the different http pooling?