[mysql] Name REFERENCES in the schema collection privileges warning - #24737
Closed
azhou-datadog wants to merge 12 commits into
Closed
[mysql] Name REFERENCES in the schema collection privileges warning#24737azhou-datadog wants to merge 12 commits into
azhou-datadog wants to merge 12 commits into
Conversation
…posite FK columns MySQL only exposes a table in INFORMATION_SCHEMA to users holding a privilege on it, which is why schema collection currently asks customers for SELECT on their tables. The REFERENCES privilege satisfies that same visibility check without granting any ability to read table data, so the insufficient-privileges warning now points at it instead of SELECT. Also make composite foreign keys deterministic: the referenced_column_names aggregate had no ORDER BY, unlike the column_names aggregate beside it, so a multi-column foreign key could report its referenced columns in either order from one collection to the next. The test previously sorted that field before comparing, which hid the non-determinism; it now asserts ordinal order directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
azhou-datadog
force-pushed
the
allen.zhou/mysql-schema-collection-improvements
branch
from
July 30, 2026 17:18
fd47c70 to
c35b9f9
Compare
Contributor
Validation Report
Run Passed validations (20)
|
|
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Points the schema-collection insufficient-privileges warning at the
REFERENCESprivilege instead ofSELECT.MySQL exposes a table in
INFORMATION_SCHEMAonly to users that hold some privilege on it — that is why schema collection currently asks customers toGRANT SELECT ON *.*, and why the setup docs have to explain in prose that the Agent doesn't actually read their data.REFERENCESsatisfies the same visibility check and grants no ability to read (or write) anything.No query changes: the collected payload is identical under either privilege.
Motivation
Requiring
SELECTon customer tables is the main setup hurdle for MySQL schema collection and blocks enabling it by default. A spike verified thatREFERENCESproduces a byte-identical schema payload while leaving the Agent unable to read table data:SELECTbaseline on MySQL 5.6.51, 5.7.44, 8.0.45, 8.4.3, 9.7.2, Percona 8.4.10 and MariaDB 10.11.18, 11.4.12.mysql_databasespayload the Agent puts on the wire is unchanged after swapping the grant (18 tables / 97 columns / 41 indexes / 4 FKs / 5 partitions, empty diff), whileSELECT * FROM <table>as the Agent user returnsERROR 1142.REFERENCESalone cannot create a foreign key: that needsCREATE/ALTERon the referencing table, which thedatadoguser does not have.rds_superuser_roleincludes itWITH GRANT OPTION), Cloud SQL, and Azure MySQL.Worth noting:
tests/conftest.py:606already grantsREFERENCESto the test user, with the comment "needed to query INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS in mariadb 10.5 and above". That requirement was worked around in the fixture but never surfaced to customers, so MariaDB users following the docs today collect foreign keys without theirON UPDATE/ON DELETEactions.One behavioural consequence for the docs rather than the code: with
REFERENCESinstead ofSELECT, theSTATEMENTexplain strategy (a directEXPLAINas thedatadoguser) no longer works, so explain plans depend on theexplain_statementdefiner procedures that setup already prescribes. Verified both halves in a sandbox.Additional Notes
Docs counterpart: DataDog/documentation#38606.
Split out of this PR, to be handled separately:
SQL_FOREIGN_KEYSaggregatesreferenced_column_nameswithgroup_concatand noORDER BY, unlike thecolumn_namesaggregate beside it, so composite foreign keys report their referenced columns in a non-deterministic order (reproduced asa,bvsb,aon the same constraint across MySQL 5.6 and MariaDB).test_collect_schemascurrently sorts that field before comparing, which hides it.Review checklist (to be filled by reviewers)
qa/skip-qalabel if the PR doesn't need to be tested during QA.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged