-
Notifications
You must be signed in to change notification settings - Fork 24.6k
-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
final_pipeline appears to process twice when using the date match index processor #83653
Comments
Pinging @elastic/es-data-management (Team:Data Management) |
Hello, I've tested this in the latest version of Elasticsearch and the issue is still occurring. Even more, I was able to recreating other scenarios where the issue is happening. Basically, every time you update the This is another version, this time using a script processor:
The following document creation will execute the final_pipeline twice:
Right now I'm working on a bugfix for this issue. |
In case, a processor changes the index of a request and the new index has a final_pipeline, this pipeline executed twice. This happens because `IngestService` overwrites the pipeline information, but it never cleans it up later. This change ensures that `IngestService` does not change the pipeline information for this case using methods without side effects Closes elastic#83653
Elasticsearch Version
7.16.2
Installed Plugins
No response
Java Version
bundled
OS Version
Cloud
Problem Description
When using the
final_pipeline
option and thedate match index
in a pipeline to process data, the final_pipeline appears to run twice. This can be shown by using theappend
processor which adds two of the same values or arename
processor which shows a failure, but it will show the name of a field changed and shows a failure due to theold
named field not being present.Also referenced in a discuss topic to confirm that the example is valid.
Additional note: This seems to run fine on a 7.6.2 environment, so something between that and 7.16.2 may have caused this issue. #69727 or #75047 could be related.
Steps to Reproduce
Append method of showing the
final_pipeline
running twice:Rename processor used to show a failure.
Logs (if relevant)
No response
The text was updated successfully, but these errors were encountered: