Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest: expose reroute inquiry/reset via Elastic-internal API bridge #96958

Merged
merged 3 commits into from Jun 20, 2023

Conversation

yaauie
Copy link
Member

@yaauie yaauie commented Jun 20, 2023

Logstash's Integration filter works directly with the processors, but cannot use the IngestService that is tightly-coupled with cluster state and must therefore emulate the behavior introduced in #94000.

To do so, the additional methods for inquiring about and resetting the reroute state need to be externally-accessible. Exposing them through a clearly-named bridge allows us to avoid making these Elastic-internal bits a part of the public APIs that are subject to years-long stability and deprecation notice policies.

This is an alternative to #96934

Logstash's Integration filter works directly with the processors, but cannot
use the IngestService that is tightly-coupled with cluster state and must
therefore emulate the behavior introduced in elastic#94000.

To do so, the additional methods for inquiring about and resetting the reroute
state need to be externally-accessible. Exposing them through a clearly-named
bridge allows us to avoid making these Elastic-internal bits a part of the
public APIs that are subject to years-long stability and deprecation notice
policies.
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v8.9.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 20, 2023
@yaauie yaauie added :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team and removed needs:triage Requires assignment of a team area label external-contributor Pull request authored by a developer outside the Elasticsearch team v8.9.0 labels Jun 20, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine
Copy link
Collaborator

Hi @yaauie, I've created a changelog YAML for you.

@joegallo
Copy link
Contributor

I like this approach more than #96934 (at least for right now). I agree that the implementation of the reroute processor is tightly coupled with the IngestService itself, and we've chosen a limited visibility for these methods in order to have the compiler enforce that, but I can also see why you need to get around that limitation (and also that your need is very much a special case).

+1, but also noting for future github historians that we can still easily switch paths and revert this PR in favor of the simpler visibility change if there's reason to do so.

@yaauie yaauie merged commit 0a45b13 into elastic:main Jun 20, 2023
12 checks passed
@yaauie yaauie deleted the ingest-stack-bridge branch June 20, 2023 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement Team:Data Management Meta label for data/management team v8.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants