Skip to content

Search Responses to Many Shards use Excessive Amounts of Memory for OriginalIndices instances  #78314

@original-brownbear

Description

@original-brownbear

Running a simple/fast search on a large number of indices via an index pattern can result in significant memory usage on the coordinating node for the org.elasticsearch.action.OriginalIndices instances in the search responses.

When using a simple pattern of e.g. auditbeat-${long} and searching over 1k indices, this leads to hundreds of MB (and more during heavy load) of duplicate instances on heap that are referenced as such:

image

I think it should be possible to deduplicate these (or even the full search requests referenced in the responses?).
This somewhat relates to #78164 but doesn't just apply to can_match.
The problem with these instances is that they are not accounted for all that well by the circuit breaker in the response collector and it's trivial to OOM a coordinating node if a search targets a large number of indices at once when the responses come back rapidly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Search/SearchSearch-related issues that do not fall into other categories>bugTeam:SearchMeta label for search team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions