docs: design for parallelising the serial COPY flush (#445)

[jdatcmd]

The approved design for #445's remaining serial-load lever: parallelise the per-column flush work (encode + FSST + block codec — each column chunk is independent) across background workers, keeping all I/O and WAL on the backend. Reuses columnar_parallel_export.c's dsm/worker machinery; degrades to serial when slots are unavailable. Four shippable slices; ~17% recoverable ceiling. Docs only; implementation follows slice by slice.

🤖 Generated with Claude Code

[jdatcmd]

Complication found while scoping slice 1 (design refinement)

Reading pgcolumnar_flush_row_group's per-column body to plan the extraction: the #472 FSST verdict cache is cross-flush mutable state. The body reads def->fsstVerdict / def->fsstVerdictAge and writes an updated verdict back for the next stripe (columnar_write_state.c:1137-1189), so the per-column flush is not stateless.

Consequence for the design: slice 1's function takes the verdict as an in/out parameter, and the backend applies the returned verdict to def after collecting the column. Each column is owned by exactly one worker per flush, so there is no contention on a single column's verdict — but the state must travel with the column, not be reached through writeState. This is the one part of the loop body that is not a clean pure extraction, and it is why slice 1 is a real refactor, not a mechanical cut. It does not block the design; it scopes slice 1.

[ChronicallyJD]

Owner decision: yes — with a two-stage path to "default"

The gate this design waits on is answered: parallelise the default single-connection COPY. The target end-state is parallel-by-default, reached in two stages rather than one:

1. Ship it gated (off by default, the slice-4 GUC/table-option), prove it, then
2. flip the default as a separate change once the bar below is met.

The reason for staging rather than defaulting on at slice 3/4: a default that every plain COPY hits must have bulletproof slot-degradation, and the failure mode if it doesn't is silent. Measured, not argued — 8 workers on 8 max_worker_processes slots loads ZERO rows with no error (the parallel_copy N+2 caveat, same root). A wrong/short row count as the default behaviour is the one outcome we can't ship, so "default" waits behind the gate until proven.

Acceptance bar I'll hold each slice to

• Byte-identical stripe data bytes and catalog rows vs the serial path (not just vs heap) — a differential that reds on any drift, at every slice, per the design's own "output byte-identical" guard.
• Slot starvation degrades to the serial in-backend path, never to a wrong row count — the gating property for the eventual default flip; tested by starving max_worker_processes and asserting the row count and bytes still match serial.

Coordination

Taking slice 1 — the pure flush_one_column(inputs) -> {encoded, descriptor, codec, zonemap} refactor with no writeState reach-through, output byte-identical, guarded by the differential + existing write suites. Posting here so we don't both build it. I'll PR it for your review; review/merge stays yours. If you'd rather drive slice 1 yourself, say so and I'll pick up review instead.

(Direction set by the owner; design is yours. — ChronicallyJD)

[ChronicallyJD]

Review — design validated by implementation, three worker-section notes to fold in

I've now built slices 1–3 from this design (#589 merged-pending, #590, and slice 3 in verification), each byte-identical to the serial path. The structure holds up end to end. Three things the implementation proved the worker section should say explicitly, because each is load-bearing for the byte-identity this design promises:

1. table_open(relid) for the tupdesc is only safe for the base flush. A projection inner write state sets relid = the base table but carries a synthetic {rownumber, projcols…} tupdesc, so a worker rebuilding each column's att from RelationGetDescr(relid) gets the wrong atttypid/attbyval/attlen/attgenerated and corrupts the projection's bytes. Slice 3 fixes this by flushing projections serially (a fail-closed tupdescIsRel flag). The design should note the worker path is base-relation-only.

2. The encoder reads GUCs live that aren't in the write state. pgcolumnar_fsst_verdict_reuse, pgcolumnar_fsst_min_gain_percent, and pgcolumnar_encoding_sample_rows are read at flush time, not captured. A worker is a fresh backend that never saw the launching session's SET, so under any non-default value its bytes diverge. Slice 3 threads all three through the dsm header. Worth calling out as a constraint alongside the WAL one.

3. The FSST verdict cache must be threaded, not just "carried in def." A worker with a fresh (UNKNOWN) verdict re-decides every group; for a group whose corpus would flip the cached verdict, re-deciding yields different stored bytes than the serial reuse. Slice 3 seeds the verdict into the dsm and returns the updated verdict so the backend applies it in column order — this is what keeps ON byte-identical to OFF, and it's subtler than the slice-1 note implies.

None of these change the design's shape — they're the sharp edges the four-slice plan has to round off, and slices 1–3 do. The plan itself (per-column flush offload, I/O + WAL on the backend, degrade to serial on slot starvation) is the right one and is measured out. No objection to recording it.

[ChronicallyJD]

Re-review — the mechanism shipped as designed; the "eventually-default" conclusion did not survive measurement

Following up now that the series is built out. The design's structure holds up exactly: slices 1–3 (pure flush_one_column, dsm round-trip, worker pool) all landed byte-identical, and your own note here about the #472 verdict cache being cross-flush mutable state was spot on — threading it seed→worker→return is what keeps ON byte-identical, and it's implemented that way.

But the doc's endpoint — "four shippable slices … eventually-default control" — should be updated, because slice 4 measured it and the default flip was declined. The win is narrow and data-dependent, not general:

shape	on/off
40 int + 1 tiny text, 500k (93 MB, one flush)	0.89 — win
int/text mix, 500k (394 MB)	1.16 — loss
5 cols @ stripe_row_limit=1000 (small flushes)	3.57 — big loss

The parallel path copies buffered data through the dsm (O(bytes)), so it only wins for a single large flush of many cheap numeric columns; text-heavy/large flushes and small/frequent flushes regress. A size gate can't rescue the flip (a byte threshold would enable the large text-heavy flushes that lose). Full data on #445.

So the accurate endpoint is: parallel_flush is a targeted opt-in (wide-numeric bulk loads), plus the two hardening fixes in #592 — not an eventual default. Everything above that line in the design is correct and shipped; only the "~17% recoverable, eventually default" framing needs the correction. Worth a one-line amend to the doc so the record matches what was measured. No objection to merging it with that note.