Skip to content

CAMEL-24556: Add LangChain4jAgentObservabilityTest - #25909

Merged
davsclaus merged 2 commits into
apache:mainfrom
atiaomar1978-hub:feature/CAMEL-24556-langchain4j-agent-observability-test-c587
Aug 31, 2026
Merged

CAMEL-24556: Add LangChain4jAgentObservabilityTest#25909
davsclaus merged 2 commits into
apache:mainfrom
atiaomar1978-hub:feature/CAMEL-24556-langchain4j-agent-observability-test-c587

Conversation

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Summary

Adds producer-level GenAI observability test coverage for langchain4j-agent, completing the gap identified during review of #25337 (Phase 1).

  • LangChain4jAgentObservabilityTest — verifies generate_content span emission with gen_ai.operation.name, gen_ai.system, gen_ai.request.model, token usage tags, and result headers when observability is enabled
  • shouldNotEmitGenAiSpanWhenDisabled — verifies no telemetry when camel.aiObservability.enabled=false
  • Test dependencies — adds test-scoped camel-ai-observability, camel-telemetry, and camel-test-junit6 to camel-langchain4j-agent/pom.xml (same pattern as camel-langchain4j-chat)

JIRA

https://issues.apache.org/jira/browse/CAMEL-24556

Testing

./mvnw test -pl components/camel-ai/camel-langchain4j-agent -Dtest=LangChain4jAgentObservabilityTest

Both tests pass.

AI-generated on behalf of atiaomar1978-hub

Add producer-level GenAI observability tests for langchain4j-agent,
mirroring langchain4j-chat and langchain4j-embeddings coverage.

Verifies generate_content spans with gen_ai.* attributes and token usage
when enabled, and no telemetry when camel.aiObservability.enabled=false.

Co-authored-by: Cursor <cursoragent@cursor.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

Automated review (Bugbot + Grok) completed:

  • Bugbot: no issues found
  • Grok: AC satisfied; added follow-up assertions in second commit for TOTAL_TOKEN_COUNT, FINISH_REASON, absent RESPONSE_MODEL header, and disabled-path body verification

AI-generated on behalf of atiaomar1978-hub

Assert total token count, finish reason, and absent response model header
on the enabled path; verify route body still works when observability is
disabled.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • components/camel-ai/camel-langchain4j-agent

🔬 Scalpel shadow comparison — Scalpel: 9 tested, 27 compile-only — current: 9 all tested

Maveniverse Scalpel detected 36 affected modules (current approach: 9).

⚠️ Modules only in Scalpel (27)
  • apache-camel
  • camel-allcomponents
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

Skip-tests mode would test 9 modules (1 direct + 8 downstream), skip tests for 27 (generated code, meta-modules)

Modules Scalpel would test (9)
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-langchain4j-agent
  • camel-launcher-container
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin
Modules with tests skipped (27)
  • apache-camel
  • camel-allcomponents
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • components/camel-ai/camel-langchain4j-agent: 16 test(s) disabled on GitHub Actions
All tested modules (36 modules)
  • Camel :: AI :: LangChain4j :: Agent
  • Camel :: All Components Sync point
  • Camel :: Assembly
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Catalog :: Camel Report Maven Plugin
  • Camel :: Catalog :: Camel Route Parser
  • Camel :: Catalog :: Console
  • Camel :: Catalog :: Dummy Component
  • Camel :: Catalog :: Maven
  • Camel :: Catalog :: Suggest
  • Camel :: Component DSL
  • Camel :: Coverage
  • Camel :: Docs
  • Camel :: Endpoint DSL
  • Camel :: Endpoint DSL :: Support
  • Camel :: Integration Tests
  • Camel :: JBang :: Core
  • Camel :: JBang :: Integration tests
  • Camel :: JBang :: MCP
  • Camel :: JBang :: Main
  • Camel :: JBang :: Plugin :: Edit
  • Camel :: JBang :: Plugin :: Generate
  • Camel :: JBang :: Plugin :: Kubernetes
  • Camel :: JBang :: Plugin :: MCP
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: TUI
  • Camel :: JBang :: Plugin :: Testing
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: Kamelet Main
  • Camel :: Launcher
  • Camel :: Launcher :: Container
  • Camel :: YAML DSL
  • Camel :: YAML DSL :: Deserializers
  • Camel :: YAML DSL :: Maven Plugins
  • Camel :: YAML DSL :: Validator
  • Camel :: YAML DSL :: Validator Maven Plugin

⚙️ View full build and test results

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI-generated review on behalf of Guillaume Nodet

LGTM — clean, well-structured observability test.

This closely follows the established pattern from LangChain4jChatObservabilityTest and LangChain4jEmbeddingsObservabilityTest. All assertions are correct against the actual GenAiAttributes constants, GenAiOperationName enum values, GenAiModelResolver behavior, and the agent producer's header-setting logic. The shouldNotEmitGenAiSpanWhenDisabled test is a valuable addition that the sister tests lack.

Two minor observations (not blocking):

  1. Consider asserting RESPONSE_MODEL absence in span tags (line ~137): The test correctly asserts Headers.RESPONSE_MODEL is null on the exchange header, since the agent producer passes null for responseModel in GenAiUsage.of(). For completeness, you could also verify tags.get(GenAiAttributes.RESPONSE_MODEL) is null — documenting the span-level behavior alongside the header-level one.

  2. Pre-existing pattern debt — RecordingTracer/RecordingSpan duplication: The same inner classes are now copy-pasted across four test classes (GenAiObservabilitySpanTest, LangChain4jChatObservabilityTest, LangChain4jEmbeddingsObservabilityTest, and this one). As observability tests grow, extracting these into a shared test-support class would reduce maintenance burden. Not introduced by this PR — just noting for future consideration.

Nice work filling the observability gap for the agent component!

@davsclaus davsclaus added this to the 4.23.0 milestone Aug 31, 2026
@davsclaus davsclaus added the test label Aug 31, 2026
@davsclaus
davsclaus merged commit 9fc36de into apache:main Aug 31, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants