Skip to content

CAMEL-23223: Stabilize flaky ManagedMessageHistoryAutoConfigIT#24425

Merged
gnodet merged 2 commits into
apache:mainfrom
gnodet:camel-23223-fix-flaky-test-managedmessagehistorya
Jul 5, 2026
Merged

CAMEL-23223: Stabilize flaky ManagedMessageHistoryAutoConfigIT#24425
gnodet merged 2 commits into
apache:mainfrom
gnodet:camel-23223-fix-flaky-test-managedmessagehistorya

Conversation

@gnodet

@gnodet gnodet commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes the intermittent failure of ManagedMessageHistoryAutoConfigIT.testMessageHistory in camel-opentelemetry-metrics, reported across various CI architectures.

Root cause: The test used await().atMost(1000ms).until(handler::hasLogs) which only waited for any log to appear (which could be a non-Camel OTel internal metric), then immediately asserted on all metrics. On slow CI architectures (ppc64le, s390x), the Camel-specific metrics may not have been exported yet within 1 second. Additionally, early metric exports during message processing could contain incomplete data, causing the per-entry assertions to fail.

Fix:

  • Replace the two-step approach (wait for any log → assert immediately) with await().atMost(20s).untilAsserted(...) that retries the full set of metric assertions until the OTel periodic reader exports Camel metrics
  • Assert on the last exported Camel MetricData (which contains the complete accumulated histogram data), rather than asserting on every exported entry
  • Make MemoryLogHandler thread-safe with CopyOnWriteArrayList since publish() is called from the OTel exporter thread while getLogs()/hasLogs() are called from the test thread
  • Remove handler from logger in finally block to prevent handler accumulation across test runs in the same JVM
  • Remove redundant new ArrayList<>() wrapper since getLogs() already returns a defensive copy

Test plan

  • ManagedMessageHistoryAutoConfigIT passes consistently (verified 3 consecutive runs locally)
  • All 89 unit tests + integration tests pass in the module
  • CI passes on all architectures

Claude Code on behalf of @davsclaus

🤖 Generated with Claude Code

Replace the short 1-second Awaitility wait (which only waited for any
log to appear) with untilAsserted using a 20-second timeout that
retries the full set of metric assertions. This handles slow CI
architectures where the OTel periodic reader may not export Camel
metrics within 1 second, and avoids failures from incomplete early
metric exports during message processing.

Also make MemoryLogHandler thread-safe with CopyOnWriteArrayList
since publish() is called from the OTel exporter thread while
getLogs()/hasLogs() are called from the test thread.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Guillaume Nodet <gnodet@gmail.com>
@gnodet gnodet requested review from apupier and davsclaus July 4, 2026 17:18
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • components/camel-opentelemetry-metrics

🔬 Scalpel shadow comparison — Scalpel: 1 tested, 0 compile-only — current: 9 all tested

Maveniverse Scalpel detected 1 affected modules (current approach: 9).

Modules only in current approach (8)
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-launcher-container
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin

Skip-tests mode would test 1 modules (1 direct + 0 downstream), skip tests for 0 (generated code, meta-modules)

Modules Scalpel would test (1)
  • camel-opentelemetry-metrics

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (9 modules)
  • Camel :: JBang :: MCP
  • Camel :: JBang :: Plugin :: MCP
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: TUI
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: Launcher :: Container
  • Camel :: Opentelemetry Metrics
  • Camel :: YAML DSL :: Validator
  • Camel :: YAML DSL :: Validator Maven Plugin

⚙️ View full build and test results

@gnodet gnodet marked this pull request as ready for review July 4, 2026 22:41
@gnodet gnodet requested a review from Copilot July 4, 2026 22:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Stabilizes the ManagedMessageHistoryAutoConfigIT.testMessageHistory integration test in camel-opentelemetry-metrics by making metric assertions resilient to delayed/partial OpenTelemetry periodic exports and by hardening the in-memory log capture against concurrent access.

Changes:

  • Replaces “wait for any export → assert once” with an await().untilAsserted(...) loop that retries the full metric assertions until Camel metrics are available and complete.
  • Asserts against the last exported Camel MetricData rather than every exported entry to avoid transient/incomplete exports during message processing.
  • Makes MemoryLogHandler thread-safe by using CopyOnWriteArrayList for cross-thread log publishing/reading.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
components/camel-opentelemetry-metrics/src/test/java/org/apache/camel/opentelemetry/metrics/integration/messagehistory/ManagedMessageHistoryAutoConfigIT.java Reworks the Awaitility strategy and assertion target to reduce flakiness on slower CI architectures.
components/camel-opentelemetry-metrics/src/test/java/org/apache/camel/opentelemetry/metrics/integration/MemoryLogHandler.java Switches log storage to a thread-safe list to avoid races between exporter and test threads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +105 to +109
await().atMost(Duration.ofSeconds(20)).untilAsserted(() -> {
List<LogRecord> logs = new ArrayList<>(handler.getLogs());
assertFalse(logs.isEmpty(), "No metrics were exported");

MetricData lastCamelMetric = null;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed both points in c26c8fe:

  • Added logger.removeHandler(handler) in a finally block to prevent handler accumulation.
  • Removed the redundant new ArrayList<>() wrapper (and the now-unused ArrayList import) since getLogs() already returns a defensive copy.

- Remove handler from logger in finally block to prevent handler
  accumulation across test runs in the same JVM
- Remove redundant new ArrayList<>() wrapper since getLogs() already
  returns a defensive copy
- Remove unused ArrayList import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Guillaume Nodet <gnodet@gmail.com>
@gnodet gnodet self-assigned this Jul 5, 2026
@gnodet gnodet added the tests label Jul 5, 2026
@gnodet gnodet merged commit 4909881 into apache:main Jul 5, 2026
5 checks passed
@gnodet gnodet deleted the camel-23223-fix-flaky-test-managedmessagehistorya branch July 5, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants