-
Notifications
You must be signed in to change notification settings - Fork 25.5k
Description
Running a simple/fast search on a large number of indices via an index pattern can result in significant memory usage on the coordinating node for the org.elasticsearch.action.OriginalIndices
instances in the search responses.
When using a simple pattern of e.g. auditbeat-${long}
and searching over 1k indices, this leads to hundreds of MB (and more during heavy load) of duplicate instances on heap that are referenced as such:
I think it should be possible to deduplicate these (or even the full search requests referenced in the responses?).
This somewhat relates to #78164 but doesn't just apply to can_match.
The problem with these instances is that they are not accounted for all that well by the circuit breaker in the response collector and it's trivial to OOM a coordinating node if a search targets a large number of indices at once when the responses come back rapidly.