Deploy Kubernetes Agent E2E backend with Helm

[nubtron]

What does this PR do?

Stacked on #24639. Review this PR after commit 496b3c25e12f54ed09ab1ad3c99f869edf534ece; the stack will be retargeted to master after #24639 merges.

Replaces the Kubernetes Agent backend's hand-written Pod and RBAC resources with the official Datadog Helm chart (datadog 3.231.6) and its Node Agent DaemonSet.

• Generates reviewed chart values for the selected Agent image while disabling the Operator, Cluster Agent, APM, logs, process collection, events, and auxiliary workloads.
• Keeps KubernetesAgent as the public AgentInterface backend and preserves the existing package/configuration and dd_agent_check flows.
• Isolates Helm cache/config/data in the ddev environment state and keeps the Namespace as the atomic ownership lock.
• Pins the DaemonSet to the validated single schedulable node and dynamically resolves its current Agent pod.
• Restores editable packages and static/Autodiscovery configuration after controller-driven pod replacement without contaminating JSON command output.
• Persists Namespace UID ownership, revalidates before destructive operations, and preserves retry state when Helm cannot safely complete cleanup.
• Labels chart resources for diagnostics while leaving all chart-resource deletion to Helm.

Motivation

The backend introduced by #24639 proves real in-cluster Kubernetes Autodiscovery, but its custom Pod/RBAC deployment is only an approximation of production. Using the pinned official chart exercises the production-shaped Node Agent DaemonSet, chart RBAC, labels, volumes, image mapping, and lifecycle while retaining the intentionally single-node E2E command semantics.

Validation

• hatch run test -q tests/e2e/agent tests/cli/env — 164 passed, 2 expected skips
• Focused Helm/Kubernetes tests — 54 passed
• Full ddev test suite during development — 1,638 passed, 16 expected skips
• Repository-pinned Ruff format/check — passed
• Targeted mypy for both Kubernetes backend modules — passed
• ddev --no-interactive test -fs velero — passed
• ddev env test --base velero py3.13 — static and discovery E2Es passed against the Helm DaemonSet, including uninstall and Kind cleanup
• Manual DaemonSet pod deletion followed by both Velero E2Es — passed with package/configuration recovery
• Pinned chart render checks — one agent container, no duplicate env keys, no Operator/Cluster Agent/CRDs, node affinity, private DogStatsD socket, bounded labels, owner-labeled RBAC, and local probes
• Final independent code review — no remaining actionable defects

The aggregate local lint hook could not create its Python 3.14 environment because this machine's Cargo toolchain cannot build pydantic-core==2.33.2; the same pinned Ruff checks and targeted mypy were run directly.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run