Detect stale and unreleased entries in Agent release requirements

[AAraKKe]

What does this PR do?

Makes ddev validate agent-reqs fail when requirements-agent-release.txt pins a datadog-* package whose integration folder is gone, and lets Agent release generation skip integrations that were pinned but never actually shipped, via a new [overrides.release.agent.unreleased-integrations] section in .ddev/config.toml.

Motivation

Snowflake remained in Agent release output after its folder was removed, and the historical cleanup in #23387 / #23412 resurrected integrations that had not shipped. The first failure mode now hard-fails; the second has an explicit per-Agent-release allowlist so generated files stay correct.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[datadog-prod-us1-4]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 2 Pipeline jobs failed

PR All | test / j06ca546 / SNMP     

🔧 Fix in code (Fix with Cursor).

AssertionError: Needed at least 1 candidates for 'datadog.snmp.check_duration', got 0 in test_e2e_snmp_listener at tests/test_e2e_snmp_listener.py:118

PR All | test / j8d5404b / MarkLogic     

🔧 Fix in code (Fix with Cursor).

AssertionError: Some metrics are collected but not asserted in tests/test_marklogic.py.

🧪 2 Tests failed in 1 job

PR All | run  

test_check_with_filters from test_marklogic.py   (Fix with Cursor)

Some metrics are collected but not asserted:
Asserted Metrics:
	- marklogic.databases.average-forest-size
	- marklogic.databases.backup-count
	- marklogic.databases.backup-read-load
	- marklogic.databases.backup-read-rate
	- marklogic.databases.backup-write-load
	- marklogic.databases.backup-write-rate
	- marklogic.databases.compressed-tree-cache-hit-rate
	- marklogic.databases.compressed-tree-cache-miss-rate
...

❄️ Known flaky: test_e2e_snmp_listener from test_e2e_snmp_listener.py   (Fix with Cursor)

Needed at least 1 candidates for 'datadog.snmp.check_duration', got 0
Expected:
        MetricStub(name='datadog.snmp.check_duration', type=0, value=None, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.1', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname=None, device=None, flush_first_value=None)
Difference to closest:
        Expected tag snmp_device:172.18.0.1
        Found snmp_device:172.18.0.2

Similar submitted:
Score   Most similar
1.00    MetricStub(name='datadog.snmp.check_duration', type=0, value=0.18474292755126953, tags=['autodiscovery_subnet:172.18.0.0/28', 'device_vendor:apc', 'firmware_version:2.0.3-test', 'loader:python', 'model:APC Smart-UPS 600', 'serial_num:test_serial', 'snmp_device:172.18.0.2', 'snmp_profile:apc_ups', 'ups_name:testIdentName'], hostname='runnervmyxb0g', device=None, flush_first_value=False)
...

Not introduced in this PR.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 94.21%
• Overall Coverage: 87.44% (+0.09%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: af286e9 | Docs | Datadog PR Page | Give us feedback!}

[codecov]

Codecov Report

❌ Patch coverage is 94.21488% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.91%. Comparing base (1a6870a) to head (af286e9).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and Codecov settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run