-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Handle indices with zero/missing uptime correctly in write-load calculation #136929
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle indices with zero/missing uptime correctly in write-load calculation #136929
Conversation
| if (someIndicesHadUptime) { | ||
| assertThat(forecastedWriteLoad.getAsDouble(), not(notANumber())); | ||
| } | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could probably have tested this by calling forecastIndexWriteLoad directly (it's exposed for testing). Happy to do that instead if we want to reduce the boilerplate.
|
Hi @nicktindall, I've created a changelog YAML for you. |
|
Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| // that index. It should be safe to extrapolate our weighted average out to the | ||
| // maximum uptime observed, based on the assumption that write-load is roughly | ||
| // evenly distributed across shards of a datastream index. | ||
| assert Double.isNaN(weightedAverageShardWriteLoad) == false : "Invalid average shard write load"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: maybe Double.isFinite instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I changed these assertions to assert on the two values we are adding to the overall totals, please re-check. I think this is a better approach as it's more agnostic of how they are calculated.
See d59c122
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
…lation (elastic#136929) Fixes: ES-13286 (cherry picked from commit 2e340de)
This caused an incident in QA, we will continue to investigate WHY an index might be missing uptime/write load for all shards, but this should protect against it if/when it happens again.
Fixes: ES-13286