-
Notifications
You must be signed in to change notification settings - Fork 647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Elasticsearch createWithPassThrough sometimes doesn't return WriteResult [1.0-M3] #1566
Comments
Thank you for reporting, that is definitely possible. |
Do you have retrying enabled? When retrying at least order isn't kept. |
Hi, I do use retrying and the problem is not with the order. It seems like some messages are missing. I have a unit test that can be used to produce the issue. |
Ok, so it seems related to retrying. Can you see (eg. in the log) if it actually retries in cases where you are missing |
Please try creating a |
I created a unit test with
|
In addition, I took a look at how |
Sorry for being so quiet, I've been traveling and on vacation. |
I've stumbled on the same issue. In my case there are 2 document updates:
Doc 1 is part of a bulk update along with other documents, while doc 2 is alone. After some time the retry is carried on but only for doc 1. Here are the actual logs:
Looking at the code for
causes the loosing of previous failed messages if a retry doesn't happen before the next failure. I think that the desired behaviour should be:
The same reasoning should also be applied for the
into
Furthermore to correctly handle the retry count for messages failed at different times, instead of having a global I'm going to prepare a patch and test it on my code in the next days. I see that #2031 is targeting the retry logic, will that be a rewrite of it thus solving this issue at the root (and making possible patches moot)? |
- failed messages are treated independently to avoid loosing track of previously failed ones - retry counts are kept per message so to guarantee correct number of retries and intervals - added test case proving fix correctness
I experienced a similar issue. The PR by @nivox seems to fix it. |
@nivox Thanks for the analysis and the PR!
Using an upstream |
- failed messages are treated independently to avoid loosing track of previously failed ones - retry counts are kept per message so to guarantee correct number of retries and intervals - added test case proving fix correctness
Hi,
I upgraded to 1.0-M3 version of Alpakka ES and it seems like there is a bug when using ElasticsearchFlow.createWithPassThrough - I'm not receiving WriteResult for all bulk documents.
Everything worked fine when I used 1.0-M1 version of Alpakka.
I know you have made some changes in ES, and I think it's related.
The text was updated successfully, but these errors were encountered: