Skip to content

Activate Java SDK flag eval metrics system tests#6706

Draft
typotter wants to merge 1 commit intomainfrom
typo/activate-java-eval-tests
Draft

Activate Java SDK flag eval metrics system tests#6706
typotter wants to merge 1 commit intomainfrom
typo/activate-java-eval-tests

Conversation

@typotter
Copy link
Copy Markdown

@typotter typotter commented Apr 8, 2026

Summary

  • Enable tests/ffe/test_flag_eval_metrics.py for the Java SDK (was missing_feature)
  • 10/17 tests pass, 7 annotated as bug (FFL-1972) for known DDEvaluator gaps
  • Add OTel SDK deps to Java Spring Boot weblog for eval metrics support

Changes

manifests/java.yml:

  • File-level: missing_featurev1.60.0
  • 7 per-test bug (FFL-1972) annotations for known failures

utils/build/docker/java/spring-boot/pom.xml:

  • Add opentelemetry-sdk-metrics:1.41.0
  • Add opentelemetry-exporter-otlp:1.41.0

Known failures (FFL-1972)

Test Root cause Fix
test_ffe_eval_metric_basic reason: targeting_match instead of static dd-trace-java#10971
test_ffe_eval_reason_split reason: targeting_match instead of split dd-trace-java#10971
test_ffe_eval_metric_type_mismatch silent type coercion dd-trace-java#11036
test_ffe_eval_metric_numeric_to_integer silent NUMERIC→INTEGER coercion dd-trace-java#11036
test_ffe_eval_metric_parse_error_invalid_regex silent invalid regex handling new work needed
test_ffe_eval_metric_parse_error_variant_type_mismatch type_mismatch instead of parse_error new work needed
test_ffe_eval_metric_count OTLP delta temporality points format investigate

Test plan

  • TEST_LIBRARY=java ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_flag_eval_metrics.py → 10 passed, 7 xfailed

Enable tests/ffe/test_flag_eval_metrics.py for the Java SDK.
10 of 17 tests pass. 7 tests annotated as bug (FFL-1972) for
known DDEvaluator gaps:
- 2x reason mapping (targeting_match instead of static/split)
- 2x type mismatch (silent coercion instead of error)
- 2x parse error (invalid regex, variant type mismatch)
- 1x metric count (OTLP delta temporality points format)

Also adds opentelemetry-sdk-metrics and opentelemetry-exporter-otlp
to the Java Spring Boot weblog pom.xml, required for the
dd-openfeature provider's eval metrics OTel counter.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 8, 2026

CODEOWNERS have been resolved as:

manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
utils/build/docker/java/spring-boot/pom.xml                             @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant