Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] AtomicRegisterCoordinatorTests testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly failing #96273

Closed
original-brownbear opened this issue May 23, 2023 · 2 comments · Fixed by #96490
Closed

[CI] AtomicRegisterCoordinatorTests testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly failing #96273

original-brownbear opened this issue May 23, 2023 · 2 comments · Fixed by #96490
Assignees
@DaveCTurner
Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed Meta label for distributed team >test-failure Triaged test failures from CI

Comments

@original-brownbear
Copy link
Member

original-brownbear commented May 23, 2023 •
edited by idegtiarenko

Build scan:
https://gradle-enterprise.elastic.co/s/xksdnrqjf2zyu/tests/:server:test/org.elasticsearch.cluster.coordination.AtomicRegisterCoordinatorTests/testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly

Reproduction line:

./gradlew ':server:test' --tests "org.elasticsearch.cluster.coordination.AtomicRegisterCoordinatorTests.testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly" -Dtests.seed=D98DE3FF01622475 -Dtests.locale=fr-CH -Dtests.timezone=Europe/Skopje -Druntime.java=20

Applicable branches:
main

Reproduces locally?:
Yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.cluster.coordination.AtomicRegisterCoordinatorTests&tests.test=testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly

Failure excerpt:

java.lang.AssertionError: node4 has no NO_MASTER_BLOCK
Expected: <false>
     but: was <true>

  at __randomizedtesting.SeedInfo.seed([D98DE3FF01622475:636E5D3BCD9A5B7]:0)
  at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
  at org.junit.Assert.assertThat(Assert.java:956)
  at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:593)
  at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:554)
  at org.elasticsearch.cluster.coordination.CoordinatorTests.testFollowerDisconnectionWithoutDisconnectEventDetectedQuickly(CoordinatorTests.java:437)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
  at java.lang.reflect.Method.invoke(Method.java:578)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:1623)
@original-brownbear original-brownbear added :Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. >test-failure Triaged test failures from CI labels May 23, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Distributed Meta label for distributed team label May 23, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@DaveCTurner DaveCTurner self-assigned this Jun 1, 2023
@DaveCTurner
Copy link
Contributor

#96107 disturbed the randomness here so the seed no longer works.

DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Jun 1, 2023
@DaveCTurner
Become candidate on publication failure
db1fdcc 
If the publication fails before starting (e.g. due to one of the
precondition checks in `CoordinationState#handleClientValue`) then we
must explicitly stand down as leader.

Closes elastic#96273
@DaveCTurner DaveCTurner mentioned this issue Jun 1, 2023
Merged
DaveCTurner added a commit that referenced this issue Jun 1, 2023
@DaveCTurner
Become candidate on publication failure (#96490)
c777925 
If the publication fails before starting (e.g. due to one of the
precondition checks in `CoordinationState#handleClientValue`) then we
must explicitly stand down as leader.

Closes #96273
HiDAl pushed a commit to HiDAl/elasticsearch that referenced this issue Jun 14, 2023
@DaveCTurner @HiDAl
Become candidate on publication failure (elastic#96490)
675e5f7 
If the publication fails before starting (e.g. due to one of the
precondition checks in `CoordinationState#handleClientValue`) then we
must explicitly stand down as leader.

Closes elastic#96273
This was referenced Oct 24, 2023
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DaveCTurner DaveCTurner

Labels
:Distributed/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed Meta label for distributed team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Become candidate on publication failure DaveCTurner/elasticsearch
3 participants
@DaveCTurner @original-brownbear @elasticsearchmachine