Skip to content

Fix flaky MicrometerTest (ConditionTimeoutException) - #4822

Merged
xiang17 merged 1 commit into
microsoft:mainfrom
xiang17:xiang17/fix-micrometer-flakiness
Aug 5, 2026
Merged

Fix flaky MicrometerTest (ConditionTimeoutException)#4822
xiang17 merged 1 commit into
microsoft:mainfrom
xiang17:xiang17/fix-micrometer-flakiness

Conversation

@xiang17

@xiang17 xiang17 commented Aug 4, 2026

Copy link
Copy Markdown
Member

This PR is trying to fix the frequent MicrometerTest is flaky failures.

MicrometerTest waited on AzureMonitorMeterRegistry's background publisher. Micrometer's StepLong/StepDouble only expose a recorded value if it is polled in the very next step (previous = (lastInit == stepTime - 1) ? v : 0), so a single delayed publish tick on a busy CI runner discards the recorded values permanently and the await() can never succeed. Raising the awaitility timeout to 30 seconds (#4746) did not help for that reason - the data is lost, not late; it recently failed again in both shouldCaptureTimer and shouldCaptureDistributionSummary.

Tests now drive the registry with a MockClock and publish explicitly, so results are deterministic and failures are ordinary assertion errors that show expected vs. actual instead of an opaque timeout.

Current failure examples

> Task :agent:instrumentation:micrometer-1.0:test

MicrometerTest > shouldCaptureTimer() FAILED
    org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in MicrometerTest was not fulfilled within 30 seconds.
        at app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1160)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1129)
        at app//MicrometerTest.shouldCaptureTimer(MicrometerTest.java:135)
> Task :agent:instrumentation:micrometer-1.0:test

MicrometerTest > shouldCaptureDistributionSummary() FAILED
    org.awaitility.core.ConditionTimeoutException: Condition with Lambda expression in MicrometerTest was not fulfilled within 30 seconds.
        at app//org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
        at app//org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1160)
        at app//org.awaitility.core.ConditionFactory.until(ConditionFactory.java:1129)
        at app//MicrometerTest.shouldCaptureDistributionSummary(MicrometerTest.java:158)

What a failure would look like after this PR

MicrometerTest > shouldCaptureTimer() FAILED
    java.lang.AssertionError: [non-zero measurements named "test-timer-DEMO", out of everything published: [test-timer=99.0]]
    Expecting actual not to be empty
        at MicrometerTest.getLastMeasurement(MicrometerTest.java:241)
        at MicrometerTest.shouldCaptureTimer(MicrometerTest.java:124)

For significant contributions please make sure you have completed the following items:

  • Design discussion issue #
  • Changes in public surface reviewed
  • CHANGELOG.md updated

MicrometerTest waited on AzureMonitorMeterRegistry's background publisher. Micrometer's StepLong/StepDouble only expose a recorded value if it is polled in the very next step (previous = (lastInit == stepTime - 1) ? v : 0), so a single delayed publish tick on a busy CI runner discards the recorded values permanently and the await() can never succeed. Raising the awaitility timeout to 30 seconds (microsoft#4746) did not help for that reason - the data is lost, not late; it recently failed again in both shouldCaptureTimer and shouldCaptureDistributionSummary.

Tests now drive the registry with a MockClock and publish explicitly, so results are deterministic and failures are ordinary assertion errors that show expected vs. actual instead of an opaque timeout.
@xiang17
xiang17 merged commit 61c3120 into microsoft:main Aug 5, 2026
146 checks passed
@xiang17
xiang17 deleted the xiang17/fix-micrometer-flakiness branch August 5, 2026 04:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants