Skip to content

docs: record that ANALYZE collects no column statistics#130

Merged
jdatcmd merged 1 commit into
jdatcmd:mainfrom
ChronicallyJD:docs/analyze-no-statistics
Jul 25, 2026
Merged

docs: record that ANALYZE collects no column statistics#130
jdatcmd merged 1 commit into
jdatcmd:mainfrom
ChronicallyJD:docs/analyze-no-statistics

Conversation

@ChronicallyJD

Copy link
Copy Markdown
Contributor

Third audit finding. Docs-only, because the fix is either large (implement sampling) or harmful (make it loud), and neither belongs in the same change as recording the state.

What I found

static bool
columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS)
{
    return false;
}

static bool
columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS)
{
    return false;
}

Both are wired into the AM routine, so ANALYZE on a columnar table samples zero rows and writes nothing to pg_statistic. It reports success.

That makes it the one unsupported operation with no signal. TABLESAMPLE, whose callbacks sit twenty lines further down in the same file, raises through COLUMNAR_UNSUPPORTED. A user running ANALYZE on a columnar table gets the same output they would get on a heap table and no indication that nothing was collected — they would have to query pg_stats and notice it is empty.

Nothing in docs/ mentions it. Every "ANALYZE" in the documentation today is EXPLAIN ANALYZE, which is unrelated.

What is and is not affected

Not affected: the planner's row count. columnar_relation_estimate_size derives *tuples from row-group metadata rather than from pg_class.reltuples, and deliberately so, per its comment. Scan and join cardinality at the top level is sized correctly.

Affected: everything ANALYZE would say about the values — most-common values, histograms, n_distinct, null fraction, average width. Predicates fall back to the planner's defaults, so WHERE col = 'x' is estimated at DEFAULT_EQ_SEL regardless of the actual distribution, and join orders are chosen from default distinct counts. For an extension whose documentation recommends indexes and index-only scans for selective access, those are exactly the estimates that decide whether the index gets used.

One thing I could not check without running a server: whether ANALYZE also writes pg_class.reltuples = 0 as a side effect of sampling nothing. The planner would not care, since it uses the callback above, but autovacuum's thresholds read reltuples directly, and a zero there would make the analyze threshold trivially reachable on every table. Worth confirming before deciding how urgent the real fix is — if it does, autoanalyze may be running far more often than intended on every columnar table in a cluster.

Why docs rather than a behaviour change

  • Erroring like TABLESAMPLE would break autovacuum, which calls analyze on its own schedule.
  • A WARNING would be emitted on every autoanalyze cycle, into the server log, forever.
  • Implementing sampling is the real fix, and it is not small: the sample callbacks would need to map PostgreSQL's block sampler onto row groups, and it wants its own tests against a heap oracle.

So this PR only records the state. If you would rather have the implementation than the paragraph, say so and I will close it.

Verification

Read-only: the two callbacks and their registration in the AM routine, the TABLESAMPLE contrast, columnar_relation_estimate_size's independence from ANALYZE, and that no existing doc covers it. Zero em-dashes in the added prose, per the project convention.

Not built, not gated — no PostgreSQL server headers in my environment. Docs-only, so the risk is confined to whether the paragraph is accurate rather than whether it compiles.

🤖 Generated with Claude Code

columnar_scan_analyze_next_block() and columnar_scan_analyze_next_tuple() both
return false unconditionally, so ANALYZE on a columnar table samples zero rows
and writes nothing to pg_statistic. It reports success, which makes this the one
unsupported operation the user gets no signal about: TABLESAMPLE, whose callbacks
sit next to these, raises through COLUMNAR_UNSUPPORTED instead.

The planner's row count is unaffected because columnar_relation_estimate_size
derives it from row-group metadata rather than from ANALYZE. What is lost is the
per-column distribution, so predicates fall back to default selectivities.

Documenting rather than changing behaviour: erroring would break autovacuum's
periodic analyze, and a warning would be logged on every autoanalyze cycle.
Implementing sampling properly is the real fix; this records the state until then.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified, and the framing is right: this belongs in the docs rather than being fixed in the same change.

Checked each supporting claim rather than the headline:

  • Both columnar_scan_analyze_next_block and columnar_scan_analyze_next_tuple are return false, and both are wired into the AM routine. ANALYZE reports success and writes nothing.
  • The row count really is unaffected: columnar_relation_estimate_size computes *tuples by summing rowCount across row-group metadata, not from pg_class.reltuples. So the distinction the text draws between accurate cardinality and absent column statistics is exact, not approximate.
  • TABLESAMPLE does raise, through COLUMNAR_UNSUPPORTED, twenty lines below the analyze callbacks. The contrast you draw is real and is the sharpest part of the finding: the one unsupported operation that stays silent.

The advice at the end is the right shape too, since it tells a user what to do (check EXPLAIN, keep filtered columns in heap) rather than just what is missing.

Approving. Docs only, nothing to gate.

Worth a follow-up issue for the underlying gap, so it does not live only in a limitations entry: sampling for ANALYZE is implementable here (a row-group-aware sampler could feed pg_statistic without a full scan), and it is the sort of thing that quietly costs plan quality on exactly the wide filtered tables this storage is for. Say the word and I will open it.

@jdatcmd
jdatcmd merged commit dfd9b9d into jdatcmd:main Jul 25, 2026
jdatcmd pushed a commit that referenced this pull request Jul 25, 2026
Seven defects were found and fixed (#128, #129, #130, #131, #132, #134, #136,
#137). Those are in the commit log already. What is not recorded anywhere is
which areas were examined and came back sound, which techniques found the bugs,
and which traps cost time, so the next audit does not re-cover the same ground.

Includes the negative results in full (encoders, compression, storage, planner
integration, visibility map, setting invariance, transactions and DDL, vacuum and
projections, interoperability), the four techniques that produced findings, and
five traps: ColumnarCatalogSnapshot results being unsafe for index scans,
unique_conc running its own cluster, grep -q under pipefail producing a check
that cannot pass, a spinning backend surviving its client, and which suites to
run when touching metadata reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants