Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] SnapshotBasedIndexRecoveryIT testCancelledRecoveryAbortsDownloadPromptly failing #107628

Closed
albertzaharovits opened this issue Apr 18, 2024 · 4 comments
Closed

[CI] SnapshotBasedIndexRecoveryIT testCancelledRecoveryAbortsDownloadPromptly failing #107628

albertzaharovits opened this issue Apr 18, 2024 · 4 comments
Assignees
@DiannaHohensee
Labels
:Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. low-risk An open issue or test failure that is a low risk to future releases Team:Distributed Meta label for distributed team >test-failure Triaged test failures from CI

Comments

@albertzaharovits
Copy link
Contributor

Seems to fail once a week: https://es-delivery-stats.elastic.dev/app/r?l=DISCOVER_APP_LOCATOR&v=8.14.0&lz=N4IgjgrgpgTgniAXKSsGJAYwDYEMDO%2BAcrgLZSIAEA9jAOYB0Ue%2BALgJab5S4yYAWDAB4AHXJgDWDfADtcI%2FP2qsARgSgATGFEzUAbrHZR8DbboPwGAZTkKlrAELqNASRkaoQgEo79aFwAqlLjulHLkVKzGrADCIZjM2Jo%2B5mgAgiq0rPgAItQA7jLY1LgaAAow1KQirNhwwaEAZrjsSRqUkTDQlCAANCB4MnQQuHRQSCAS0PAgAL79%2BFlIANrLIAACHORsZCJ9IB74mCAAuif9utgQpDL4K%2Bcg7O6eE40qKhqYAGwAHACcAFoVFAAEwAZgBABYVGCNACfgAGACsAHYAbhcJCESCoH9NAjMEj9k8ojA9LhsBNcBBWNR9o1WqS7ohlg8NLhWLgAGpGfIuDSvd6fX6A4HgqEwuGI1HozHY3H4wn7LZQLwhMZIUCNSqkCYg7GQgEIw0IgCMAVNSMQkJ%2BiDBpoYkJBkIAWsq6Rh9c6jYbTT8LVbIV9EKaUQwvl8kW75iBtNrjPw3KTyZTkCAU9AkF8ETmEf0xBBuEhWF0oLNZkA%3D

Build scan:
https://gradle-enterprise.elastic.co/s/4jpgp3aan356a/tests/:x-pack:plugin:snapshot-based-recoveries:internalClusterTest/org.elasticsearch.xpack.snapshotbasedrecoveries.recovery.SnapshotBasedIndexRecoveryIT/testCancelledRecoveryAbortsDownloadPromptly

Reproduction line:

./gradlew ':x-pack:plugin:snapshot-based-recoveries:internalClusterTest' --tests "org.elasticsearch.xpack.snapshotbasedrecoveries.recovery.SnapshotBasedIndexRecoveryIT.testCancelledRecoveryAbortsDownloadPromptly" -Dtests.seed=305390E34B1BEA94 -Dtests.locale=en-NZ -Dtests.timezone=America/Aruba -Druntime.java=17 -Dtests.fips.enabled=true

Applicable branches:
main, 8.13

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.xpack.snapshotbasedrecoveries.recovery.SnapshotBasedIndexRecoveryIT#testCancelledRecoveryAbortsDownloadPromptly

Failure excerpt:

java.lang.AssertionError: safeAwait: CountDownLatch did not reach zero within the timeout

  at org.junit.Assert.fail(Assert.java:89)
  at org.junit.Assert.assertTrue(Assert.java:42)
  at org.elasticsearch.test.ESTestCase.safeAwait(ESTestCase.java:2101)
  at org.elasticsearch.xpack.snapshotbasedrecoveries.recovery.SnapshotBasedIndexRecoveryIT.testCancelledRecoveryAbortsDownloadPromptly(SnapshotBasedIndexRecoveryIT.java:759)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:568)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:833)
@albertzaharovits albertzaharovits added :Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. >test-failure Triaged test failures from CI labels Apr 18, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@elasticsearchmachine elasticsearchmachine added blocker Team:Distributed Meta label for distributed team labels Apr 18, 2024
@volodk85 volodk85 added medium-risk An open issue or test failure that is a medium risk to future releases and removed blocker labels Apr 22, 2024
@volodk85
Copy link
Contributor

due to failure cadence lowering to medium priority

@DiannaHohensee DiannaHohensee self-assigned this Jun 7, 2024
@DiannaHohensee
Copy link
Contributor

Failed on this test line.

@DiannaHohensee DiannaHohensee added low-risk An open issue or test failure that is a low risk to future releases and removed medium-risk An open issue or test failure that is a medium risk to future releases labels Jun 7, 2024
@fcofdez
Copy link
Contributor

fcofdez commented Jun 25, 2024

This was fixed by #110107

@fcofdez fcofdez closed this as completed Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DiannaHohensee DiannaHohensee

Labels
:Distributed/Recovery Anything around constructing a new shard, either from a local or a remote source. low-risk An open issue or test failure that is a low risk to future releases Team:Distributed Meta label for distributed team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@fcofdez @DiannaHohensee @albertzaharovits @elasticsearchmachine @volodk85