FYI this bug is related to #4153, #4155 and #4158

Any progress on this yet?

I'm trying to implement a workaround for #6314 and this is required. Any update here for a fix?

Hello,

I'm seeing an issue with the bulk request API where I am sending many bulk requests simultaneously.

When my index reaches a certain size and the amount of memory assigned to each shard is increased (i.e. increasing from 56Mb allocated to 128Mb allocated) the pending bulk requests fail to respond (i.e. no call to 'onResponse' or 'onFailure' in the BulkRequest ActionListener) effectively locking my application from sending any additional requests as the semaphore flags acquired before calling 'execute' are never released.

Could this issue relate to a problem that I am seeing with the bulk request API?

Thanks,
Andrew

Hi @awnixon ,
if you mean the semaphore within BulkProcessor, that's now released in a finally block. Which version of elasticsearch are you using though? This reminds me of #4153, it might be your problem if you're using an old version of elasticsearch,

Hi @javanna,

Within my own client application I manage the number of bulk requests which may be sent using a semaphore flag which is acquired before calling execute and released once a response is received.
The issue you referenced does sound similar to my own, however I am using the 2.0.0-SNAPSHOT version (as of the 9th of June).

Also, I do not see any exceptions coming from the elasticsearch node. Simply the statement that the amount of memory allocated to each shard is to be increased at which point the number of pending requests in my client application steadily increases without any response from the server. I am still able to execute curl -XGet requests (i.e. elastisearch cluster state is reported as green on request).

I hope that helps.

All the best,
Andrew

Then the problem can't be in the BulkProcessor as you are not using it. I'd suggest to switch to it though. There might be an issue in your client code that handles the semaphore?

Thanks for the suggestion. I've switched to the BulkProcessor and found the
situation to be the same where I set the level of concurrency to > 1.
I believe that I have found the root cause of my issue however. Whilst
using concurrent concurrent bulk requests I'm reaching (or exceeding) the
specified "indices.memory.max_index_buffer_size" (which I've set to 256Mb
during testing). When I increase the "indices.memory.max_index_buffer_size"
value my issue no longer occurs. Would you expect elasticsearch to handle
this situation by rejecting some/all pending requests or should it be
returning to the current set of pending requests once the
"index_buffer_size" for each shard has been updated?

On 13 June 2014 15:59, Luca Cavanna notifications@github.com wrote:

Then the problem can't be in the BulkProcessor as you are not using it.
I'd suggest to switch to it though. There might be an issue in your client
code that handles the semaphore?

—
Reply to this email directly or view it on GitHub
#5038 (comment)
.