parallel_export_parquet: six follow-ups from the #329 review, on merged code

[jdatcmd]

parallel_export_parquet landed via #329/#331. The blocker found in review (a
cross-backend snapshot handoff that duplicated the table and dropped the caller's
uncommitted rows) was fixed and I verified it against the original reproduction.

These six were raised in that review, judged non-blocking, and are now on main.
Filing so they are not lost with the closed PR. Roughly in order of how much I
would care.

1. Partitioned export re-derives the leaf list from the live catalog. Each
worker enumerates leaves itself rather than receiving the set the dispatcher
computed. A concurrent ATTACH or DETACH CONCURRENTLY is not blocked by the
dispatcher's AccessShareLock, so workers can disagree about the relation set.
Note the snapshot fix does not cover this: an MVCC snapshot governs row
visibility, not which relations are enumerated. The fix is to ship the actual leaf
OIDs in the DSM rather than a recipe for recomputing them, which also makes the
split self-describing instead of positional.

2. No cleanup of partial output on failure. The require-empty-directory guard
protects entry but not exit. A worker failing mid-export leaves part files that
pgcolumnar.read_parquet will union as if the export were complete, and the
directory then fails the empty check on retry. Silent wrong answers on read-back
is the bad shape here.

3. ColumnarParquetCheckExportable validates the parent's tupdesc, the writer
runs per leaf. A leaf whose column shape differs from the parent passes the
fail-fast pre-check and then fails or misbehaves inside a worker, which is the
worst place to discover it.

4. Workers register as BgWorkerStart_RecoveryFinished. A read-only export
cannot run on a hot standby, which is exactly where you would want to run one.
BgWorkerStart_ConsistentState is the plausible value.

5. No dsm-detach callback. A FATAL in the dispatcher orphans the workers and
drops the snapshot's horizon protection while they are still reading.

6. ColumnarWriteParquetFile frees none of its per-call buffers, and the
partitioned worker calls it in a loop, so memory grows with the partition count
rather than staying flat.

Testing note, since it is the reason the original blocker survived to review: every
fixture in test/parallel_export_parquet.sh ran the export as a standalone
autocommit statement against a quiet table, which is the one configuration where
the snapshot handoff is a no-op. The in-transaction case is covered now. Items 1
and 2 above have the same property: neither is reachable by a fixture that does not
have something else happening at the same time, or a failure injected mid-run.

[ChronicallyJD]

Five of these six are already on main — they went in with the blocker fix (0a3f87d, the commit you verified), so I think this crossed with my #329/#331 replies. Grepping src/columnar_parallel_export.c on main:

• 1. leaf list from the live catalog — fixed. The dispatcher computes the oid-sorted leaves and ships them in the DSM (PEXPORT_KEY_LEAVES); workers read leaves[i] and never re-derive. find_all_inheritors runs only in the dispatcher now.
• 3. parent vs leaf tupdesc — fixed. The dispatcher loops every leaf and calls ColumnarParquetCheckExportable(part) before spawning.
• 4. BgWorkerStart_RecoveryFinished — fixed. Workers register BgWorkerStart_ConsistentState.
• 5. dsm-detach / orphaned workers on FATAL — fixed. The spawn+wait is wrapped in PG_ENSURE_ERROR_CLEANUP(pexport_terminate_workers, ...), which terminates the workers on any exit path including FATAL.
• 6. per-call buffers in the partition loop — fixed. Each partition runs in its own AllocSetContext that is deleted after its file, so memory stays flat in the partition count.

So the substantive one still open is:

• 2. no cleanup of partial output on failure — real, and the "silent wrong answers on read-back" framing is right. I left it deliberately (the require-empty-directory guard was the only mitigation). Taking it now: on any failure or cancel the dispatcher removes the *.parquet it wrote, so a failed run leaves a clean directory and a retry is not blocked and cannot be half-read. PR shortly.

If you agree 1/3/4/5/6 are covered, this issue reduces to item 2 and I'll close it with that PR. If any of the five doesn't match what you're seeing on main, point me at it.

[ChronicallyJD]

Confirmed, all five. I checked each against main rather than take the summary:

item	what I looked for	result
1	leaf OIDs shipped, not re-derived	PEXPORT_KEY_LEAVES present, find_all_inheritors appears once and only in the dispatcher
3	per-leaf exportable check	ColumnarParquetCheckExportable called per leaf before spawn
4	hot standby	BgWorkerStart_ConsistentState, zero remaining BgWorkerStart_RecoveryFinished
5	FATAL orphans workers	PG_ENSURE_ERROR_CLEANUP wrapping spawn+wait with a terminate callback
6	per-call buffers	per-partition AllocSetContextCreate, deleted after each file

Item 1 is the one I most wanted to see, because the snapshot fix explicitly does
not cover it: a snapshot governs row visibility, not which relations get
enumerated. Shipping the OIDs rather than a recipe for recomputing them is the
right shape, and it makes the split self-describing at the same time.

Correcting my own record: these went in with 0a3f87d, the commit I verified for
the snapshot blocker. I checked the blocker there and did not re-check the rest of
my findings against the same commit, so I carried all six forward as open and
recommended item 1 as the next work an hour after it had been fixed. The filing was
still worth doing, since item 2 was genuinely outstanding and is now closed by #333,
but the "five still open" framing was wrong and it was mine.

Nothing further from me here.