perf(store-runtime): release publish-time memory when the build permit is released

[ScriptedAlchemy]

Problem

The #803 publication coordination slice serializes corpus-sized manifest builds per project publication shard (measured: peak overlapping builds = 1 at 1/2/4/8 scopes). Peak process RSS nevertheless grows linearly with the number of published worktree scopes, because memory acquired while a scope publishes is retained after that scope releases the build permit.

Measured with concurrent_worktree_scopes_publish_with_one_corpus_build_and_bounded_rss (600-file / 7,200-function corpus, one project, N distinct worktree generations, publish_verified_snapshot concurrently):

scopes	RSS before publish	sampled peak during publish	growth	growth / scope
1	0.27 GB	0.90 GB	0.64 GB	0.64 GB
2	0.39 GB	1.64 GB	1.25 GB	0.62 GB
4	0.59 GB	3.00 GB	2.41 GB	0.60 GB
8	1.03 GB	5.77 GB	4.74 GB	0.59 GB

Builds do not overlap (wall ≈ N × single publish), so this is retained published state per RetainedCodeGraphRuntimeV1/scope (staged relational projection, replay/materialization buffers, or the seated generation), not concurrent build memory. ~0.6 GB for a 7,200-function generation is also far above the sealed generation's on-disk size, so part of it is likely transient build memory that is never returned to the allocator or is held by an Arc that outlives publication.

Required design

• Attribute the per-scope retention with Hotpath allocation tracking across publish_verified_snapshot (manifest projection, relational staging, sealed copy, replay pool) and identify what outlives the permit.
• Release publish-time buffers when the permit is released; retain only what serving needs, and make retained-per-scope memory proportional to the served index, not to the build.
• Keep the perf(store-runtime): coordinate graph publication per project #803 lock cell and lock order unchanged.

Acceptance evidence

• Re-run the 1/2/4/8-scope harness: peak RSS growth after the first scope should be bounded by the served-state size per scope, with the measurement in the closing comment.
• Existing publication, journal-resume, cancellation, and rollback suites unchanged.

Refs #803 (measurement), PR #707 (9de84bc8a harness).

[ScriptedAlchemy]

Slice 2 of the memory work landed on the #707 branch as 7638233 (perf(graph-db): release sealed generation staging rows once installed), picked up from the Cursor memory lane and re-verified: graph-db unit and contract suites (including the new staging_footprint tests: rows held for at most the active and in-flight generations, remount after release serves from the sealed store, hibernated retirement and release sweeps never open the engine), store-runtime unit suite, daemon maintenance and graph-replay tests, clippy and rustfmt on the pinned toolchain.

What it changes: a generation whose sealed store is installed, whose head is verified and which nothing depends on has its staging rows released after seal-install and serves from the sealed store; the release sweep keeps rows for in-flight, dependency-bearing and not-yet-sealed generations; a hibernated staging engine returns RetentionPending / StagingEngineHibernated instead of being opened for retirement or release; a zero-row leftover staging commit is treated as absent on recovery.

Not yet done for this issue's acceptance: the re-run of the allocation / RSS journey with the numbers in a closing comment. The measurement scaffolding that lane used (temporary systemd drop-ins pointing the operator daemon at a lane binary and gating starts on a WINDOW_OPEN marker) was removed so the production daemon runs again; a measurement window needs to be re-established deliberately before that evidence can be captured. Leaving open for the measurement.

[ScriptedAlchemy]

Live pre-fix acceptance baseline from the preserved operator profile, daemon beta.37+150a104b5: after generation 27 sealed and its staging engine closed, the eager open left RSS at about 21.88 GB (start 4.90 GB; observed peak 21.97 GB). The generation graph reports ready, but rebuild remains in flight and stale master stays served because admission sees RSS above the 18.84 GB high watermark. This is the exact post-publication release boundary #830 must improve. Keep open until the current release/arena-trim WIP is deployed and the same journey demonstrates resident memory returning below the existing watermark before the permit is reusable.

[ScriptedAlchemy]

Post-fix live run on exact pushed 250c5e0 did not reach a reusable build permit: generation 29 remained pending and process RSS reached 23.46 GB by 169 seconds and 26.26 GB by 221 seconds, so I stopped it through the normal service lifecycle. The hibernation change is present, but it does not address the active projection/build working set that crosses the 18.84 GB admission watermark before publication can settle. This issue remains open; acceptance still requires the same preserved-profile journey to return below the existing watermark without a restart.

[ScriptedAlchemy]

Landed on the #707 branch: 298424812 perf(store-runtime): release publish memory at the build permit, with 250c5e01b (graph-db) underneath it.

Measured on concurrent_worktree_scopes_publish_with_one_corpus_build_and_bounded_rss (600 files / 7,200 functions, builds never overlap):

scopes	before growth	after growth	retained after publish
1	0.63 GB	0.66 GB	0.36 GB
2	1.24 GB	0.87 GB	0.43 GB
4	2.42 GB	1.29 GB	0.83 GB

Marginal peak growth per scope 0.59-0.61 GB → 0.21 GB. Attribution experiments: TRACEDECAY_GRAPH_SEALED_STORE=off made it worse (staging rows retained); MALLOC_ARENA_MAX=1 cut the 4-scope growth 2.42 → 1.47 GB, so ~0.24 GB/scope was glibc arena retention of dead memory, now trimmed at the permit boundary. The harness asserts a retention ceiling and a peak ceiling that the pre-fix numbers fail from two scopes up. Remaining: ~0.21 GB/scope marginal retention still above the served index size (allocator high-water the trim does not reach).