Add pg_class optimizer and autovacuum columns to postgres schema collection by dujuku · Pull Request #22617 · DataDog/integrations-core

dujuku · 2026-02-11T22:04:42Z

Summary

Adds 7 new columns from pg_class to the PostgreSQL integration's schema collection to support optimizer suggestions and autovacuum tuning insights.

New Columns

Optimizer Statistics:

reltuples (float) - Estimated row count used by query planner
relpages (int) - Table size in pages (8KB blocks)

Autovacuum Configuration & Tracking:

reloptions (text[]) - Table-level storage parameters including autovacuum settings
relallvisible (int) - Pages marked all-visible in visibility map
relfrozenxid (text) - Frozen XID threshold for wraparound prevention
relminmxid (text) - Frozen multixact ID threshold
relallfrozen (int) - Pages marked all-frozen (PostgreSQL 18+ only)

Motivation

These columns enable several key use cases:

Detecting which tables have custom autovacuum tuning
Verifying suggested tuning actions were applied
Correlating autovacuum behavior with table-level settings
Identifying tables that need tuning based on bloat/activity
Supporting query optimizer suggestions

The reloptions field in particular exposes table-level autovacuum parameters like autovacuum_vacuum_threshold, autovacuum_vacuum_scale_factor, autovacuum_freeze_min_age, and 10+ other parameters.

Implementation Details

Version handling: relallfrozen is only collected on PostgreSQL 18+ using conditional query logic
Data types: XIDs cast to text for safe JSON serialization, reloptions kept as PostgreSQL text array format
Backward compatibility: Works across PostgreSQL versions 9.6-18
TypedDict updated: Added all new fields with proper type hints using total=False
Payload impact: ~40-240 bytes per table (avg ~100 bytes), negligible for typical databases

🤖 Generated with Claude Code

datadog-datadog-prod-us1 · 2026-02-12T00:03:20Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f31a007 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

codecov · 2026-02-12T18:21:32Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.34%. Comparing base (e218424) to head (f31a007).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add 8 new columns from pg_class to provide comprehensive table metadata: - has_indexes (relhasindex) - whether table has indexes - relation_kind (relkind) - type of relation (r=table, p=partition, f=foreign) - num_columns (relnatts) - number of user columns in table - num_check_constraints (relchecks) - number of CHECK constraints - has_triggers (relhastriggers) - whether table has triggers - row_security_enabled (relrowsecurity) - whether row security is enabled - is_populated (relispopulated) - whether table is populated (for materialized views) - is_partition (relispartition) - whether this is a partition (PG 10+) These columns complement the existing optimizer and autovacuum columns to provide a complete view of table configuration and characteristics. Updated all 19 test snapshots and changelog to reflect the changes. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Add two new column-level attributes from pg_attribute: - is_dropped (attisdropped) - whether the column has been dropped - storage_type (attstorage) - storage mode (p=plain, e=external, m=main, x=extended) These attributes provide insight into column lifecycle and storage optimization strategies. Updated all 19 test snapshots and changelog. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Refactored schema collection query to inline column definitions instead of building them as separate string variables. This makes the code cleaner, reduces repetition, and improves maintainability. - Replaced three string-building variables with direct inline columns - Created single is_pg10_or_newer boolean to avoid repeated version checks - Used inline conditionals to add is_partition column for PG 10+ - All schema tests pass Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Break long lines in schemas.py to comply with 120 character limit and run auto-formatter on all files. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

…lumns

Fix formatting to use ruff format instead of black to match CI expectations. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

sethsamuel · 2026-02-13T21:53:14Z

postgres/datadog_checks/postgres/postgres.py

            # TODO: Use the submission debouncer to only send this every 6 hours
            self.health.submit_health_event(
                name=HealthEvent.INITIALIZATION,
-                status=HealthStatus.ERROR


Why was this changed?

sethsamuel · 2026-02-13T21:53:50Z

postgres/datadog_checks/postgres/schemas.py

-                    query += f" AND ({
-                        ' OR '.join(f"datname ~ '{include_regex}'" for include_regex in self._config.include_databases)
-                    })"
+                    include_or_conditions = ' OR '.join(


Why was this and the other include blocks changed?

sethsamuel · 2026-02-13T21:54:30Z

postgres/datadog_checks/postgres/schemas.py

+                tables.frozen_xid, tables.min_mxid, tables.table_options,
+                tables.has_indexes, tables.relation_kind, tables.num_columns,
+                tables.num_check_constraints, tables.has_triggers, tables.row_security_enabled,
+                tables.is_populated{', tables.is_partition' if is_pg10_or_newer else ''}


This feels fragile and hard to read, would it be better to pull all these columns out into an array?

sethsamuel · 2026-02-13T21:54:58Z

postgres/datadog_checks/postgres/schemas.py

                {partition_joins}
            GROUP BY schema_tables.schema_id, schema_tables.schema_name, schema_tables.schema_owner,
-                schema_tables.table_id, schema_tables.table_name, schema_tables.table_owner, schema_tables.toast_table
+                schema_tables.table_id, schema_tables.table_name, schema_tables.table_owner, schema_tables.toast_table,


Do we have any sense of if a group by this large will cause performance problems?

sethsamuel · 2026-02-13T21:55:13Z

postgres/datadog_checks/postgres/schemas.py

-                    ]
-                    if cursor_row.get("table_name")
-                    else [],
+                    "tables": (


Why did this whole block shift?

sethsamuel · 2026-02-13T21:56:06Z

postgres/tests/fixtures/schema_snapshot_v10_C.json

                    {
-                        "id": "16678",
-                        "name": "personsdup10",
+                        "id": "16697",


Why did all of these churn? Without being able to clearly map the changes to the intended changes we can't be sure there weren't unintended side effects

sethsamuel · 2026-02-13T22:20:40Z

postgres/datadog_checks/postgres/schemas.py

+       c.relallvisible       AS all_visible_pages,
+       c.relfrozenxid::text  AS frozen_xid,
+       c.relminmxid::text    AS min_mxid,
+       c.reloptions          AS table_options,


These columns don't match the ones in the description

temporal-github-worker-1 bot added agent/review-requested ecosystems/review-requested product/review-requested labels Feb 11, 2026

datadog-agent-integrations-bot bot added the integration/postgres label Feb 11, 2026

add attrnum col

baf0a3d

dujuku force-pushed the dujuku/pg-schema-columns branch from f00af7f to baf0a3d Compare February 11, 2026 23:18

changelog and snapshot tests updates

77419c5

dujuku and others added 2 commits February 12, 2026 14:57

dujuku force-pushed the dujuku/pg-schema-columns branch 2 times, most recently from d5fe936 to dc86a87 Compare February 13, 2026 04:13

dujuku marked this pull request as ready for review February 13, 2026 20:34

dujuku requested review from a team as code owners February 13, 2026 20:34

datadog-agent-integrations-bot bot added team/agent-integrations team/database-monitoring-agent labels Feb 13, 2026

dujuku and others added 3 commits February 13, 2026 15:46

Fix line length linting issues and format code

ae27bfc

Break long lines in schemas.py to comply with 120 character limit and run auto-formatter on all files. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/master' into dujuku/pg-schema-co…

0ab95f3

…lumns

Run ruff format to match CI formatter

9f5c125

Fix formatting to use ruff format instead of black to match CI expectations. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

sethsamuel requested changes Feb 13, 2026

View reviewed changes

Revert unnecessary formatting change in postgres.py

6343d80

sethsamuel reviewed Feb 13, 2026

View reviewed changes

dujuku added 2 commits February 13, 2026 17:41

Revert unnecessary formatting changes in schemas.py

966b11b

Reorder snapshot fixtures to match original master order

f31a007

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pg_class optimizer and autovacuum columns to postgres schema collection#22617

Add pg_class optimizer and autovacuum columns to postgres schema collection#22617
dujuku wants to merge 11 commits intomasterfrom
dujuku/pg-schema-columns

dujuku commented Feb 11, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Feb 12, 2026 •

edited by datadog-official bot

Loading

Uh oh!

codecov bot commented Feb 12, 2026 •

edited

Loading

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

sethsamuel Feb 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dujuku commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New Columns

Motivation

Implementation Details

Uh oh!

datadog-datadog-prod-us1 bot commented Feb 12, 2026 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

sethsamuel Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dujuku commented Feb 11, 2026 •

edited

Loading

datadog-datadog-prod-us1 bot commented Feb 12, 2026 •

edited by datadog-official bot

Loading

codecov bot commented Feb 12, 2026 •

edited

Loading