There are no easy way to retry on failures when indexing in bulk

[vytas-dauksa]

Search::Elasticsearch client tracks two error types internally:

• "NoNodes" Cxn errors, which don't clear buffer and are retried immediately at least once (Async::Static::NoPing cxn_pool accepts max_retries, which allows specifying maximum number of retries). However, other cxn errors clear buffer - making it hard to retry on failures.
• Individual action failures - like EsRejectedExecutionException[rejected execution (queue capacity 50) on org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction..] -
  clears the buffer and are not retried.

In either error case it is difficult to retry on failure.

For bulk individual action failures there are handy on_error and on_conflict handlers, however they only provide action, response and index in the callback. I would argue later should not be included - as it only exposes internal workings. I think it would be much more useful to provide the original request, so that client user would be able to requeue it or otherwise handle it.

For cxn errors, it is unclear what other than "NoNodes" errors exist and why they clear the buffer. In case of Async, it is difficult to handle even "NoNodes" errors, for instance user may desire to delay between retries. I haven't checked - but wouldn't ES client in bulk mode auto-flush and keep-dispatching after each action in case of "NoNodes" cxn error and larger queue than max_count or max_size?

[atishbits]

I think it would be much more useful to provide the original request

I agree this would be useful, however this will require the ES-Client to internally cache requests until they have succeeded. This will be an unnecessary memory overhead. This is much better done by the application itself, as in some cases the App may not be bothered if a few requests fail while in other cases it may want to retry. According to the use case, the App can decide to keep the requests in memory until a success response is received from ES-server.