Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log reason(s) for reroute if exceptional #58259

Open
DaveCTurner opened this issue Jun 17, 2020 · 1 comment
Open

Log reason(s) for reroute if exceptional #58259

DaveCTurner opened this issue Jun 17, 2020 · 1 comment
Labels
:Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >enhancement Team:Distributed Meta label for distributed team

Comments

@DaveCTurner
Copy link
Contributor

Sometimes we call reroute() because something went wrong, and often that "something" is a problem common to multiple shards. For instance, a CircuitBreakingException during shard fetching will likely affect all the shard fetches on that node at once. We want to know about such exceptions in the logs, but there's little value in logging the exception for every single shard.

Today we discussed this as a team (in relationship to #57804 (comment)) and decided that it seems natural to use the batching facility built into the BatchedRerouteService to record examples of these failures in the logs, on a reroute-by-reroute basis, without having to record them all.

@DaveCTurner DaveCTurner added >enhancement :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) labels Jun 17, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Allocation)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >enhancement Team:Distributed Meta label for distributed team
Projects
None yet
Development

No branches or pull requests

2 participants