Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] ThreadPoolTests testScheduledFixedDelayRejection failing #106618

Closed
gmarouli opened this issue Mar 21, 2024 · 5 comments · Fixed by #106630
Closed

[CI] ThreadPoolTests testScheduledFixedDelayRejection failing #106618

gmarouli opened this issue Mar 21, 2024 · 5 comments · Fixed by #106630
Assignees
@DaveCTurner
Labels
:Core/Infra/Core Core issues without another label low-risk An open issue or test failure that is a low risk to future releases Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI

Comments

@gmarouli
Copy link
Contributor

This test was introduced in #106542

Build scan:
https://gradle-enterprise.elastic.co/s/vhbjdbibh2se2/tests/:server:test/org.elasticsearch.threadpool.ThreadPoolTests/testScheduledFixedDelayRejection

Reproduction line:

./gradlew ':server:test' --tests "org.elasticsearch.threadpool.ThreadPoolTests.testScheduledFixedDelayRejection" -Dtests.seed=CA7F3E4D95D9A00F -Dtests.locale=bg-BG -Dtests.timezone=Asia/Anadyr -Druntime.java=22

Applicable branches:
main

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.threadpool.ThreadPoolTests#testScheduledFixedDelayRejection

Failure excerpt:

java.lang.AssertionError: safeAwait: CountDownLatch did not reach zero within the timeout

  at __randomizedtesting.SeedInfo.seed([CA7F3E4D95D9A00F]:0)
  at org.junit.Assert.fail(Assert.java:89)
  at org.junit.Assert.assertTrue(Assert.java:42)
  at org.elasticsearch.test.ESTestCase.safeAwait(ESTestCase.java:2097)
  at org.elasticsearch.threadpool.ThreadPoolTests.lambda$blockExecution$12(ThreadPoolTests.java:555)
  at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
  at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:984)
  at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.lang.Thread.run(Thread.java:1570)
@gmarouli gmarouli added :Core/Infra/Core Core issues without another label >test-failure Triaged test failures from CI labels Mar 21, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine elasticsearchmachine added blocker Team:Core/Infra Meta label for core/infra team labels Mar 21, 2024
@gmarouli gmarouli mentioned this issue Mar 21, 2024
Merged
DaveCTurner added a commit that referenced this issue Mar 21, 2024
@DaveCTurner
AwaitsFix for #106618
93a21e1
DaveCTurner added a commit that referenced this issue Mar 21, 2024
@DaveCTurner
AwaitsFix for #106618
09df993
DaveCTurner added a commit that referenced this issue Mar 21, 2024
@DaveCTurner
AwaitsFix for #106618
d74e9f8
@albertzaharovits
Copy link
Contributor

Started failing recently.
Thank you for muting @DaveCTurner !

@DaveCTurner
Copy link
Contributor

👍 looks flaky since it was introduced yesterday; muted on main, 8.13 and 7.17.

@DaveCTurner
Copy link
Contributor

Ok yeah this test doesn't quite work. The problem is that we need the scheduled task to be rejected, but because it's racing with the blocking tasks it might end up in the queue and therefore it never runs.

DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Mar 21, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection
f7d85c0 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes elastic#106618
@DaveCTurner DaveCTurner mentioned this issue Mar 21, 2024
Merged
@DaveCTurner DaveCTurner added the low-risk An open issue or test failure that is a low risk to future releases label Mar 21, 2024
@DaveCTurner DaveCTurner self-assigned this Mar 21, 2024
@DaveCTurner
Copy link
Contributor

Just a test bug --> low-risk

@lkts lkts mentioned this issue Mar 21, 2024
Closed
DaveCTurner added a commit that referenced this issue Mar 22, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection (#106630)
1fe4946 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes #106618
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Mar 22, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection (elastic#106630)
154d09b 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes elastic#106618
DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Mar 22, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection (elastic#106630)
829f7ac 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes elastic#106618
elasticsearchmachine pushed a commit that referenced this issue Mar 22, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection (#106630) (#106643)
009950d 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes #106618
elasticsearchmachine pushed a commit that referenced this issue Mar 22, 2024
@DaveCTurner
Fix testScheduledFixedDelayRejection (#106630) (#106642)
72aa514 
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.

Closes #106618
This was referenced Mar 25, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DaveCTurner DaveCTurner

Labels
:Core/Infra/Core Core issues without another label low-risk An open issue or test failure that is a low risk to future releases Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix testScheduledFixedDelayRejection DaveCTurner/elasticsearch
4 participants
@albertzaharovits @gmarouli @DaveCTurner @elasticsearchmachine