Fixing _rollup/data performance for a large number of indices #138305
Conversation
masseyke
added
>bug
:StorageEngine/Rollup
|
Hi @masseyke, I've created a changelog YAML for you. |
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
|
Here's what it looks like calling |
nielsbauman
left a comment
nielsbauman
left a comment
There was a problem hiding this comment.
Left one optional suggestion, other than that LGTM. Thanks Keith :)
| static Map<String, RollableIndexCaps> getCapsByRollupIndex(Collection<String> resolvedIndexNames, Map<String, IndexMetadata> indices) { | ||
| Map<String, List<RollupJobCaps>> allCaps = new TreeMap<>(); | ||
|
|
||
| indices.entrySet().stream().filter(entry -> resolvedIndexNames.contains(entry.getKey())).forEach(entry -> { |
There was a problem hiding this comment.
Do you think it's worth converting this to a regular for-each loop (instead of using the Streams API)?
There was a problem hiding this comment.
It doesn't seem like it would matter much for a single pass through a collection, does it?
There was a problem hiding this comment.
That is, this is only hot code b/c it's in an N^2 for loop -- the method itself isn't called all that often.
3 participants
This avoids performing an N^2 loop through indices when we call
GET /*/_rollup/data. This can save a few seconds per call for clusters with a very large number of indices.