Migrate validate jmx-metrics from datadog_checks_dev to ddev by AAraKKe · Pull Request #23652 · DataDog/integrations-core

AAraKKe · 2026-05-10T20:19:24Z

What does this PR do?

Migrates the validate jmx-metrics command from datadog_checks_dev/.../tooling/commands/validate/jmx_metrics.py to ddev/src/ddev/cli/validate/jmx_metrics.py, rewriting the implementation to use ddev's Application output style and ddev's Repository / Integration model. Adds a new Integration.jmx_metrics_file cached property in ddev/src/ddev/integration/core.py so other commands can reuse the same file lookup. The legacy command file is deleted; entry removed from ALL_COMMANDS in datadog_checks_dev's validate group.

Helper gap notes

tooling.utils.get_jmx_metrics_file had no direct ddev equivalent. Added Integration.jmx_metrics_file (cached property returning a Path; caller checks .is_file()). PR 3.5 (meta jmx) is expected to reuse this property.
Integration.is_jmx_check already existed; refactored to delegate to jmx_metrics_file.is_file() for consistency. Behavior unchanged.
tooling.utils.is_jmx_integration (reads conf.yaml.example and looks up init_config.is_jmx) is a different criterion from Integration.is_jmx_check (file presence). To preserve substantive parity with the legacy command, ported the conf-based check inline as a private _is_jmx_integration helper. ddev's native iter_jmx_checks would have changed the matched set (e.g., would have included hazelcast, which legacy excludes). If meta jmx ends up needing the same conf-based criterion, we can lift this helper to a shared module.
tooling.utils.get_default_config_spec → replaced with Integration.config_spec.
tooling.testing.process_checks_option → replaced with app.repo.integrations.iter(selection) plus a 'changed' short-circuit to iter_changed_code().
tooling.commands.console.annotate_error (GitHub Actions annotation): no ddev equivalent. Errors are still emitted via app.display_error and surface via the non-zero exit code; CI annotations are dropped. (Tracked separately: an app.annotate_* primitive will be added in a follow-up infrastructure PR; this command will be retrofitted then.)
read_file / file_exists → Path.read_text() / .is_file() from ddev.utils.fs.Path.

Test plan

ddev validate jmx-metrics --help output identical to master
ddev validate jmx-metrics tomcat produces the same exit code (0) and output as master
Full repo ddev validate jmx-metrics returns the same set of 13 JMX integrations as legacy
ddev --no-interactive test ddev passes (3 unrelated VCR-cassette failures from worktree-path-as-repo-name detection — being fixed separately in a parallel infrastructure PR)
ddev --no-interactive test datadog_checks_dev passes (437/437)
New unit tests at ddev/tests/cli/validate/test_jmx_metrics.py cover happy path, missing include, missing scope, duplicate beans, missing config spec, and missing JMX templates in spec

Motivation

Part of the datadog_checks_dev → ddev CLI migration wave (PR 1.10). Every CLI command currently registered in datadog_checks_dev's legacy tooling tree is being moved into ddev as native code, with the legacy tooling/ directory deleted at the end of the multi-phase migration. This PR ports validate jmx-metrics and exposes a Integration.jmx_metrics_file property that downstream commands (notably meta jmx) will reuse, advancing the wave's goal of consolidating CLI logic in ddev.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

github-actions · 2026-05-10T20:20:38Z

⚠️ Major version bump
The changelog type changed or removed was used in this Pull Request, so the next release will bump major version. Please make sure this is a breaking change, or use the fixed or added type instead.

datadog-prod-us1-4 · 2026-05-10T20:23:20Z

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1 Test failed

test_e2e_scalar_oid_retry from test_e2e_core_vs_python.py

(Fix with Cursor)

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 01-check-apikey.sh: executing... 
[cont-init.d] 01-check-apikey.sh: exited 0.
[cont-init.d] 50-ci.sh: executing... 
[cont-init.d] 50-ci.sh: exited 0.
[cont-init.d] 50-ecs-managed.sh: executing... 
...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 93.57%
• Overall Coverage: 87.37% (+0.11%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: ef3cb1c | Docs | Datadog PR Page | Give us feedback!}

codecov · 2026-05-10T20:25:38Z

Codecov Report

❌ Patch coverage is 93.57143% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.86%. Comparing base (03b790e) to head (ef3cb1c).

Additional details and impacted files

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05fde98e62

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-10T21:47:54Z

+        selection: tuple[str, ...] = (check,) if check and check.lower() != 'all' else ()
+        candidates = app.repo.integrations.iter(selection)


Pass the all sentinel for full validation

When CHECK is omitted or set to all, this converts the selection to (), but IntegrationRegistry.iter(()) is not an all-integrations selection in ddev: __finalize_selection treats an empty selection as the changed root entries (or returns None when there are no changes), while ('all',) is the sentinel that iterates every integration. As a result ddev validate jmx-metrics all and ddev validate jmx-metrics validate zero or only changed integrations instead of all JMX integrations, so most metrics files are skipped.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-05-10T21:47:54Z

+    if check and check.lower() == 'changed':
+        candidates = app.repo.integrations.iter_changed_code()


Include spec-only changes in changed validation

For the changed target this uses iter_changed_code(), which yields only integrations whose changed files satisfy Integration.requires_changelog_entry (package directory files or pyproject.toml). This validator also checks assets/configuration/spec.yaml, which is outside the package directory, so a PR that only edits a JMX integration's spec can remove the init_config/jmx or instances/jmx template and ddev validate all changed will not run jmx-metrics for that integration.

Useful? React with 👍 / 👎.

Actually this is funny becasue if we change the spec it also requires a changelog entry... I prefer to keep it like this for now and open a follow up pr to change that logic.

PR: #23655

…ly validate every JMX integration An empty selection tuple makes IntegrationRegistry.__finalize_selection fall back to its 'changed roots' path, which iterates zero (or only changed) integrations instead of all of them. Use ('all',) so the finalizer returns set(), producing the intended full iteration. Add a parametrized regression test that creates three fake JMX checks and asserts both 'ddev validate jmx-metrics' and 'ddev validate jmx-metrics all' validate all three.

dd-octo-sts · 2026-05-10T22:26:44Z

Validation Report

All 20 validations passed.

Show details

Validation	Description	Status
`agent-reqs`	Verify check versions match the Agent requirements file	✅
`ci`	Validate CI configuration and Codecov settings	✅
`codeowners`	Validate every integration has a CODEOWNERS entry	✅
`config`	Validate default configuration files against spec.yaml	✅
`dep`	Verify dependency pins are consistent and Agent-compatible	✅
`http`	Validate integrations use the HTTP wrapper correctly	✅
`imports`	Validate check imports do not use deprecated modules	✅
`integration-style`	Validate check code style conventions	✅
`jmx-metrics`	Validate JMX metrics definition files and config	✅
`labeler`	Validate PR labeler config matches integration directories	✅
`legacy-signature`	Validate no integration uses the legacy Agent check signature	✅
`license-headers`	Validate Python files have proper license headers	✅
`licenses`	Validate third-party license attribution list	✅
`metadata`	Validate metadata.csv metric definitions	✅
`models`	Validate configuration data models match spec.yaml	✅
`openmetrics`	Validate OpenMetrics integrations disable the metric limit	✅
`package`	Validate Python package metadata and naming	✅
`readmes`	Validate README files have required sections	✅
`saved-views`	Validate saved view JSON file structure and fields	✅
`version`	Validate version consistency between package and changelog	✅

View full run

Migrate validate jmx-metrics from datadog_checks_dev to ddev

c5ac3f9

AAraKKe added the qa/skip-qa Automatically skip this PR for the next QA label May 10, 2026

dd-octo-sts Bot added dev/tooling dev_package ddev labels May 10, 2026

Add changelog

7ddddc1

AAraKKe mentioned this pull request May 10, 2026

Add GitHub Actions annotation primitive to ddev's Application #23654

Open

6 tasks

AAraKKe added 2 commits May 10, 2026 23:12

Apply formatter

cfd7a95

Fix mypy union-attr errors in _duplicate_bean_check

05fde98

AAraKKe marked this pull request as ready for review May 10, 2026 21:43

AAraKKe requested a review from a team as a code owner May 10, 2026 21:43

dd-octo-sts Bot added the team/agent-integrations label May 10, 2026

chatgpt-codex-connector Bot reviewed May 10, 2026

View reviewed changes

AAraKKe mentioned this pull request May 10, 2026

Require changelog entry when conf.yaml.example changes #23655

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate validate jmx-metrics from datadog_checks_dev to ddev#23652

Migrate validate jmx-metrics from datadog_checks_dev to ddev#23652
AAraKKe wants to merge 5 commits into
masterfrom
aarakke/migrate-validate-jmx-metrics

AAraKKe commented May 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

datadog-prod-us1-4 Bot commented May 10, 2026 •

edited by datadog-official Bot

Loading

Uh oh!

codecov Bot commented May 10, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 10, 2026

Uh oh!

chatgpt-codex-connector Bot May 10, 2026

Uh oh!

AAraKKe May 10, 2026 •

edited

Loading

Uh oh!

dd-octo-sts Bot commented May 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		selection: tuple[str, ...] = (check,) if check and check.lower() != 'all' else ()
		candidates = app.repo.integrations.iter(selection)

		if check and check.lower() == 'changed':
		candidates = app.repo.integrations.iter_changed_code()

Conversation

AAraKKe commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Helper gap notes

Test plan

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

github-actions Bot commented May 10, 2026

Uh oh!

datadog-prod-us1-4 Bot commented May 10, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

codecov Bot commented May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 10, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot May 10, 2026

Choose a reason for hiding this comment

Uh oh!

AAraKKe May 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dd-octo-sts Bot commented May 10, 2026

Validation Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

AAraKKe commented May 10, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented May 10, 2026 •

edited by datadog-official Bot

Loading

codecov Bot commented May 10, 2026 •

edited

Loading

AAraKKe May 10, 2026 •

edited

Loading