pgcolumnar.analyze() takes null_frac from the zone maps, so after a DELETE it is normalised against rows the table no longer holds

[ChronicallyJD]

pgcolumnar.analyze() reads null_frac from the zone maps, which count rows that were WRITTEN. A DELETE marks rows dead without rewriting those counts, so the fraction stays normalised against a population the table no longer holds.

Found while measuring the function for a follow-on to #414. It is on merged main (f6b09ce), independent of anything I have open, and VACUUM does not clear it.

The statistics disagree with each other, in the same pg_stats row

1,200 rows: 120 null, 300 holding 7, 300 holding 9, the rest unique. Delete the rows holding 9, so 900 rows remain and 7 is still a most-common value.

CREATE TABLE d2 (v int) USING pgcolumnar;
INSERT INTO d2
SELECT CASE WHEN i % 10 = 0 THEN NULL
            WHEN i %  4 = 0 THEN 7
            WHEN i %  4 = 1 THEN 9
            ELSE 100000 + i END
  FROM generate_series(1, 1200) i;
DELETE FROM d2 WHERE v = 9;
SELECT pgcolumnar.analyze('d2'::regclass, ARRAY['v']);

	value	implies a table of
null_frac written	0.100000	1,200 rows
most_common_freqs[1] written	0.2666667	900 rows
rows actually present		900

null_frac is 0.1 where the truth is 120/900 = 0.1333, a 25% understatement. The most-common frequency is right. They disagree because they come from different places:

• null_frac — sum(null_count) / sum(value_count + null_count) over pgcolumnar.zone_map
• most_common_freqs — count(*) / totalrows, where totalrows is count(*) over the table

So null_frac + sum(mcv_freqs) + rest = 1 stops holding. That identity is not decorative: eqsel subtracts null_frac and the most-common total to price everything else, so the residual it computes for non-MCV values is wrong by whatever the two populations differ by.

VACUUM does not heal it

A second fixture, 1,000 rows with 100 nulls, deleting the 301 rows holding one value:

	null_frac
truth (100/699)	0.143062
written	0.100000
written after VACUUM d1	0.100000

30% understated, and unchanged by VACUUM. I did not test whether a compaction rewrite clears it — if it does, the window is "until the next rewrite" rather than permanent, which is still unbounded in practice.

Why it survived review

The suite's fixtures insert and never delete, so the two denominators are equal in every existing check and the zone-map read looks exact. It IS exact — of the rows on disk. The comment in the function says "null_frac, exactly", and that is true of the wrong population.

This is the same failure shape as the attstattarget NULL bug and the swallowed-ERROR one: a plausible value, written successfully, with nothing raising.

Fix

The cheap one is to take the null count from the read the function is already doing:

SELECT count(DISTINCT v)::bigint, count(*)::bigint,
       count(*) FILTER (WHERE v IS NULL)::bigint FROM ...

count(*) is already in that query, so the null count is one more aggregate over a scan that happens either way — the zone-map read was saving nothing here, because the function never collects null_frac alone. The zone maps keep the job they can still do exactly: answering whether the column has any row groups at all.

That makes both statistics come from one population by construction, which is the property worth having rather than "each is exact against its own source".

The wider question — whether pgcolumnar.zone_map's counts should account for the delete vector — is a real one and touches pruning as well as statistics, but it is not needed to fix this.

I have this fixed and tested locally as part of the #414 follow-on and can send it as its own PR ahead of that if you would rather keep them separate. Reproduction above is self-contained either way.