Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intermittent TestServer.checkMetricsForExecutorService #4982

Closed
romain-grecourt opened this issue Sep 27, 2022 · 7 comments · Fixed by #5103
Closed

Intermittent TestServer.checkMetricsForExecutorService #4982

romain-grecourt opened this issue Sep 27, 2022 · 7 comments · Fixed by #5103
Assignees
Labels
4.x Version 4.x bug Something isn't working intermittent testing

Comments

@romain-grecourt
Copy link
Contributor

java.lang.AssertionError: 
Completed task count after accessing slow endpoint
Expected: is <1>
     but: was <0>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.helidon.metrics.TestServer.checkMetricsForExecutorService(TestServer.java:221)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
@tomas-langer
Copy link
Member

It seems there is an inherent race condition in this test - the update of metrics may happen after the response is returned, causing the next request to be "too early" to see the modified metric.
This should be refactored to use some deterministic appraoch

@tomas-langer tomas-langer added the bug Something isn't working label Sep 30, 2022
@tomas-langer
Copy link
Member

For main branch, we will disable this test until a solution is found

@tjquinno
Copy link
Member

Yes, this is the case in at least one other test as well. I am addressing them.

@danielkec
Copy link
Contributor

Just failed in 4x on top of the fix commit ce733cb

[INFO] Running io.helidon.metrics.TestServer
Error:  Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.559 s <<< FAILURE! - in io.helidon.metrics.TestServer
Error:  io.helidon.metrics.TestServer.checkMetricsForExecutorService  Time elapsed: 1.021 s  <<< FAILURE!
java.lang.AssertionError: 
Completed task count after accessing slow endpoint
Expected: is <1>
     but: was <0>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.helidon.metrics.TestServer.lambda$checkMetricsForExecutorService$1(TestServer.java:223)
	at io.helidon.common.testing.junit5.MatcherWithRetry.retry(MatcherWithRetry.java:76)
	at io.helidon.metrics.TestServer.checkMetricsForExecutorService(TestServer.java:207)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)

@danielkec danielkec reopened this Oct 5, 2022
@danielkec
Copy link
Contributor

@danielkec
Copy link
Contributor

danielkec commented Oct 6, 2022

Tweak with FT #5103

@danielkec danielkec linked a pull request Oct 6, 2022 that will close this issue
@tjquinno
Copy link
Member

tjquinno commented Oct 6, 2022

Re-opening. Daniel had improved on my original retry approach, but retrying is not the right solution here. The test GreetService already has a CountdownLatch which other tests use but this one did not. I'll submit yet another PR that fixes the test so it correctly uses the latch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4.x Version 4.x bug Something isn't working intermittent testing
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

4 participants