fix: non-text column reads from compressed partitions (text[], inet, numeric)

[wezell]

Incident

2026-07-14: a Superset chart (SELECT array_length(hostnames,1) ... FROM pingbacks) reading the hostnames text[] column across ~503 compressed partitions killed the postmaster → CNPG failover. Smaller reads returned silent NULLs; deltax_decompress_partition errored invalid UTF-8 in dictionary. Live cluster is mitigated (compression policy cleared, pingback_reader SELECT revoked); this PR is the durable fix (v0.2.0-dotcms.5).

Root causes (two distinct bugs)

1. Emit-path type confusion — the crash + silent NULLs. classify_column defaults every unrecognized type (text[], inet, numeric, uuid, …) to ColumnKind::Text; the write side losslessly stores the ::text rendering. But str_slices_to_text_datums_arena built a raw text varlena for every type_oid except bpchar/jsonb and handed it to a slot whose attribute type is e.g. text[] (1009). PG then reads text bytes as an ArrayType header — garbage ndim/dims → silent NULLs from array_length(); anything walking the array (unnest, array_out) computes nitems from garbage → wild reads / huge allocations → backend/postmaster death.

Fix: only TEXTOID/VARCHAROID take the raw-varlena fast path; every other type_oid is reconstructed via the type input function (getTypeInputInfo + OidInputFunctionCall) — the exact inverse of the write side's text serialization, so EXISTING compressed partitions become readable without recompression. Materialization is main-backend-thread only, so the input-function calls are safe.

2. decompress_partition panicked invalid UTF-8 in dictionary on jsonb. Since upstream xataio#27, jsonb blobs hold PG's binary jsonb varlena payload; the scan path decodes them byte-level, but decompress_column_values still routed them through the UTF-8-validating text decoders. Fix: byte-level decode (decode_to_byte_slices / range-based LZ4) + new jsonb_binary_to_text (varlena wrap + jsonb_out, exact inverse of jsonb_text_to_binary). dictionary::parse_header's .expect is now a descriptive error pointing at the byte-level API (pgrx already converted that unwind to a PG ERROR — the backend never died on this one, just cryptic).

Pruning audit

batch quals / minmax / valbitmap / bloom are all gated on numeric or text oids, so text-ordered metadata can never wrongly prune segments for these column types — no changes needed there.

Tests

tests/test_nontext_columns.py: pingbacks-shaped table (text[], inet, numeric, uuid, jsonb, NULL-mixed) through Dictionary/DictionaryLz4 and Lz4Blocked codecs; exact row equality before compression, after compression, and after deltax_decompress_partition; the literal incident query shape.

Rollout (after merge)

Tag v0.2.0-dotcms.5 → .deb build → ovh-k8s-cluster Dockerfile bump → roll postgres-ovh-east → decompress the 503 pingbacks partitions → re-GRANT pingback_reader. Pingbacks compression stays OFF until the write path is re-validated against arrays on a test table.

🤖 Generated with Claude Code