Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add response-handler executor parameter to internal client actions #86765

Open
DaveCTurner opened this issue May 12, 2022 · 4 comments
Open

Add response-handler executor parameter to internal client actions #86765

DaveCTurner opened this issue May 12, 2022 · 4 comments
Labels
:Core/Infra/REST API REST infrastructure and utilities Team:Core/Infra Meta label for core/infra team >tech debt

Comments

@DaveCTurner
Copy link
Contributor

DaveCTurner commented May 12, 2022

Actions executed via the transport client would fork to the LISTENER threadpool to complete their listener to avoid problems with running too much work on a transport thread. In contrast, actions executed internally via NodeClient don't fork by default. We removed this forking behaviour entirely in #53049 as part of the removal of the transport client.

Since then we've seen at least a couple of issues where internal actions end up inadvertently doing nontrivial work on a transport thread:

I think this is trappy, there's not even anywhere to specify which thread to use to complete the listener so it's understandable if a developer might think it's going to be completed somewhere safe, perhaps on the same threadpool as was used to call the action in the first place.

I think we should at least require an executor parameter when executing an action via the client so that callers must make a conscious decision about whether to fork for the response listener or not. Alternatively we could just make them all fork by default (to where? reinstate the LISTENER threadpool?) and allow an opt-in non-forking API for performance-critical cases.

@DaveCTurner DaveCTurner added :Distributed/Network Http and internode communication implementations team-discuss labels May 12, 2022
@elasticmachine elasticmachine added the Team:Distributed Meta label for distributed team label May 12, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@henningandersen
Copy link
Contributor

The es-distributed team discussed this and arrived at requiring an executor parameter, filled in with SAME in the initial commit (possibly picking another one for known cases like the two cases above).

@DaveCTurner DaveCTurner changed the title Should (internal) client actions fork before completing their listeners by default? Add response-handler executor parameter to internal client actions Jun 13, 2022
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Jun 14, 2022
`TaskListener` is almost always a trivial wrapper around an
`ActionListener`. There's a couple of places where that's not true, it
reports the task ID in a log message, but these logging effects can be
achieved in other ways which don't require polluting various widely-used
APIs so much.

This commit replaces all usages of `TaskListener` with `ActionListener`,
reworking the logging effects, and removing the now-unnecessary overrides.

Relates elastic#86765 in that this commit simplifies some of the APIs that
elastic#86765 will need to adjust.
elasticsearchmachine pushed a commit that referenced this issue Jun 24, 2022
`TaskListener` is almost always a trivial wrapper around an
`ActionListener`. There's a couple of places where that's not true, it
reports the task ID in a log message, but these logging effects can be
achieved in other ways which don't require polluting various widely-used
APIs so much.

This commit replaces all usages of `TaskListener` with `ActionListener`,
reworking the logging effects, and removing the now-unnecessary
overrides.

Relates #86765 in that this commit simplifies some of the APIs that
#86765 will need to adjust.
@DaveCTurner DaveCTurner added :Core/Infra/REST API REST infrastructure and utilities and removed :Distributed/Network Http and internode communication implementations Team:Distributed Meta label for distributed team labels Sep 19, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Sep 19, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@DaveCTurner
Copy link
Contributor Author

This is basically the other half of #97916.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/REST API REST infrastructure and utilities Team:Core/Infra Meta label for core/infra team >tech debt
Projects
None yet
Development

No branches or pull requests

5 participants