Skip to content

Fix testILMDownsampleRollingRestart() test #98889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 6, 2023
Merged

Fix testILMDownsampleRollingRestart() test #98889

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e868e00
Fix testILMDownsampleRollingRestart() test
martijnvg Aug 25, 2023
b45dc41
Merge branch 'main' into downsampel/fix_test/testILMDownsampleRolling…
elasticmachine Sep 4, 2023
459e881
Merge branch 'main' into downsampel/fix_test/testILMDownsampleRolling…
elasticmachine Sep 5, 2023
a32ab1f
Merge branch 'main' into downsampel/fix_test/testILMDownsampleRolling…
elasticmachine Sep 5, 2023
d01664e
Merge branch 'main' into downsampel/fix_test/testILMDownsampleRolling…
elasticmachine Sep 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions ...nternalClusterTest/java/org/elasticsearch/xpack/downsample/ILMDownsampleDisruptionIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,8 +210,14 @@ private void startDownsampleTaskViaIlm(
var request = new UpdateSettingsRequest(sourceIndex).settings(
Settings.builder().put(LifecycleSettings.LIFECYCLE_NAME, POLICY_NAME)
);
// Updating index.lifecycle.name setting may fail due to the rolling restart itself,
// we need to attempt it in a assertBusy(...)
assertBusy(() -> {
try {
if (indexExists(sourceIndex) == false) {
Copy link
Contributor

@salvatore-campagna salvatore-campagna Sep 4, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly this might happen in case there is a background downsample task running that failed for some reason, has been retried, and before we get here, the task completes right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. For example the request to update index settings fails to return because the node got shutdown, but the index settings were updated. The downsampling can be kicked off. When retrying the source index could not longer exist.

logger.info("The source index [{}] no longer exists, downsampling likely completed", sourceIndex);
return;
}
client().admin().indices().updateSettings(request).actionGet(TimeValue.timeValueSeconds(10));
} catch (Exception e) {
logger.warn(() -> format("encountered failure while updating [%s] index's ilm policy", sourceIndex), e);
Expand Down