Phase F reclaim: free-space splitting and coalescing - #90
Merged
Conversation
Adds partial-block reuse and adjacency coalescing to the physical-reclaim free list, behind pgcolumnar.reclaim_coalesce (default on; off reverts to whole-range reuse). record_free_space stores the page-aligned footprint so free ranges tile the file page-aligned; ColumnarAllocateFreeSpace splits an oversized chosen range and records the page-aligned remnant; record_free_space merges an immediately adjacent range before insertion. Coalescing is restricted to ranges freed by the SAME transaction: merging a fresh free with an older already-reusable neighbor would force the union to the newer freed_xid (required for safety), delaying reuse of space that was reusable a moment ago -- a regression the native_reclaim_cycles guard caught (the file grew every cycle). Same-transaction coalescing still merges the many groups one maintenance op retires together, which is the fragmentation case that matters. Adds an assert-only invariant (ColumnarCheckFreeSpaceNoOverlap) run at the end of each online maintenance op: live row-group footprints and free ranges must tile the file with no overlap. Adds native_reclaim_frag.sh: retiring 18 adjacent groups yields 1 coalesced free range with the GUC on vs 18 fragmented rows off, and a large reclustering then reuses in place (smaller final file) instead of extending; parity holds in both modes. Full 15-19 matrix green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds partial-block reuse (split-on-allocate) and adjacency coalescing (coalesce-on-free) to the physical-reclaim free list, behind
pgcolumnar.reclaim_coalesce(default on; off reverts to whole-range reuse).record_free_spacestores the page-aligned footprint, so free ranges tile the file page-aligned.ColumnarAllocateFreeSpacesplits an oversized chosen range and records the page-aligned remnant.record_free_spacemerges an immediately adjacent range before insertion — restricted to ranges freed by the same transaction, so it never forces an older already-reusable neighbor to a newerfreed_xid(which delays reuse;native_reclaim_cyclescaught that regression). Same-transaction coalescing still merges the many groups one maintenance op retires together — the fragmentation case that matters.Guards: an assert-only invariant (
ColumnarCheckFreeSpaceNoOverlap) runs at the end of each online maintenance op — live footprints and free ranges must tile with no overlap. Newnative_reclaim_frag.sh: retiring 18 adjacent groups → 1 coalesced free range (GUC on) vs 18 fragmented rows (off), and a large recluster then reuses in place (smaller final file) instead of extending; parity holds in both modes.Full PostgreSQL 15-19 matrix green.
🤖 Generated with Claude Code