fix: a bigint column against an integer literal now prunes chunk groups (#477)

[jdatcmd]

Closes #477. Found while building the fixture for #461, which is blocked on this.

The measurement

Two columns, identical values, same operator, same layout:

CREATE TABLE zc (idi int, seq bigint, tag int) USING pgcolumnar;
INSERT INTO zc SELECT g, g, g % 5 FROM generate_series(1, 20480) g;

predicate	chunk groups removed
idi > 16000	7 of 10
seq > 16000	0
seq > 16000::bigint	7 of 10

The only difference is that the bigint column met an int4 literal.

Cause, and why the old code was half right

columnar_reader.c dropped the scan key:

/* avoid cross-type comparisons that our column cmp proc cannot do */
if (OidIsValid(key->sk_subtype) && key->sk_subtype != att->atttypid)
    continue;

Right about the proc, wrong about the remedy. btint8cmp handed an int4 Datum
reads a value that is not there, so refusing to use it was correct. But refusing
meant no skip predicate was built at all, and every bigint column compared
against a bare integer literal read every chunk group. That is what ordinary SQL
looks like.

btree opfamilies already carry the answer: integer_ops supplies btint84cmp
for (int8, int4). The proc is now resolved for the pair of types; where a
family has none, the key is still skipped, which is the old behaviour kept as the
fallback rather than the rule.

Two details that are not obvious

The equality branch had to be rewritten column-first. It compared
cmp(const, min) and cmp(const, max). That is equivalent for a same-type proc
and wrong for a cross-type one, whose argument order is part of its signature.
Now skip if min > const or max < const.

The bloom probe stays disabled for cross-type equality. The filter is built
by hashing column-type values, so hashing an int4 constant with the int8 hash
proc probes a slot the writer never set and would skip a group that holds the
row. That is a wrong answer, where min/max pruning is only ever conservative.
It is the one place in this change whose failure mode is not a missed
optimisation, which is why equality parity and an absent-value check are both
asserted.

It was invisible from EXPLAIN, and that is how it survived our own gate

Columnar Pushed-Down Filters counts scan keys handed to the reader, not
predicates able to exclude anything, so it read 1 while zero groups were
skipped.

test/zonemap_cost.sh has been in that state since it was written. It
declares seq bigint and queries WHERE <col> > 160000, so its correlated
arm -- the one whose entire purpose is to prune -- removed zero chunk groups.
#460's discount was validated against it. The suite asserted a cost relation
between two priced plans, and that is true or false whether or not the pruning it
prices actually happens.

It now asserts the physical premise first, from EXPLAIN ANALYZE's own counter:

-- groups removed: seq = 16, scat = 0

#460 is not wrong to discount; the layout genuinely is prunable. Its evidence was.

Verification

	without the fix	with it
native_skip.sh	3 checks fail	19/19
zonemap_cost.sh	new premise fails, seq = 0	13/13, seq = 16

RED proven before implementing, and proven again by restoring the cross-type
refusal afterwards. Full 132-suite matrix green on PG18 and PG19 (130 ran / 2
skipped on 18, 132 / 0 on 19).

Scope

Deliberately only the executor's pruning. #461's cost model is the follow-on and
now has a base where pruning demonstrably occurs. I have not touched the
Columnar Pushed-Down Filters counter, though it is arguably the reporting half
of this defect -- a filter that cannot exclude anything still counts as pushed
down. Worth its own issue if you agree.

🤖 Generated with Claude Code

https://claude.ai/code/session_013sY1RQR5MjZNUixA2um31r