Add opt-in incremental query metrics collector for MySQL

[eric-weaver]

What does this PR do?

Adds an alternative query metrics collector for MySQL behind a new hidden, default-off incremental_query_metrics flag.

The existing collector obfuscates every row returned from events_statements_summary_by_digest on every collection, including statements that have not executed since the previous one. The new collector snapshots the cumulative counters, diffs them against the previous snapshot to determine which statements actually executed, and resolves and obfuscates query text only for those. Obfuscation results are memoized in a digest-keyed cache sized from performance_schema_digests_size.

Three new modules are added — delta_detector.py, obfuscation_lookup.py, and statements_v2.py — mirroring the structure of the equivalent Postgres implementation. Collector selection happens once at check initialization in _initialize_statement_metrics. A performance_schema_digests_size accessor is added to GlobalVariables, and the existing integer-coercing property bodies there are folded into a shared _get_int_variable helper.

The emitted payload is unchanged. Existing users are unaffected until they explicitly opt in.

Motivation

Obfuscation is a cgo call and is the dominant cost of query metrics collection on instances with large digest tables, since it is repeated every collection for statements whose counters have not moved.

Benchmarked end to end against a live Agent container with both collectors running side by side over four workload shapes (cold, warm, hot-light, hot-churn), comparing intercepted payloads:

• Obfuscation calls fall by up to 3572x on steady-state workloads.
• Wall clock improves by 1.01x to 1.54x depending on workload shape, and never regresses.
• Emitted metric payloads are identical to the existing collector in every scenario.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Made with Cursor

[dd-octo-sts]

Validation Report

Validation	Description	Status
license-headers	Validate Python files have proper license headers	❌

Run ddev validate all changed --fix to attempt to auto-fix supported validations.

Passed validations (20)

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run

[datadog-prod-us1-5]

   

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

Validate repository | Run Validations / Validate     

🔧 Fix in code. This looks caused by changes in this PR.

3 files do not match expected license format in mysql/datadog_checks/mysql.

📋 Copy prompt for your agent

CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Branch: eric.weaver/mysql-incremental-query-metrics

Validate repository | Run Validations / Validate
Commit: 26ea0f14db5b1e65fac54ecf5d392263e532526f
Error (code / quality):
3 files do not match expected license format in mysql/datadog_checks/mysql.
CI job: https://github.com/DataDog/integrations-core/actions/runs/31019160882/job/92351074984

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 93.39%
• Overall Coverage: 91.04% (+2.71%)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 24187ee | Docs | Datadog PR Page | Give us feedback!}