perf(snapshot): drop fsyncs on transient extract/copy paths; fold NoSync variants into SyncMode#119
perf(snapshot): drop fsyncs on transient extract/copy paths; fold NoSync variants into SyncMode#119CMGS wants to merge 1 commit into
Conversation
e21d748 to
7979759
Compare
7979759 to
dd17eed
Compare
Restore staging, stream-clone extraction, and direct clone/restore file clones are rebuilt from a durable source on retry, so their per-file fsyncs bought no durability while adding GiB-scale writeback to clone and restore latency. ExtractTar gains the SyncMode parameter and the clone/restore copyPairs flip to NoSync; the AtomicWrite NoSync variants fold into the same parameter, leaving one sync convention across utils. Store ingestion stays durable: Import keeps Sync, and Create extracts NoSync then syncs once via SyncTree instead of twice.
dd17eed to
5507725
Compare
|
Parked rather than rejected. Rebased onto master (post-#120) so the branch carries exactly one clean commit for revival. Why close now:
|
|
Two more findings from a whole-repo over-durability audit (master @ f8180ed), recorded here since they are the same class as this PR — optional fsync/durability scoping, not worth reopening on their own:
If clone/restore latency enters the product budget, both can ride the same change as this branch's ExtractTar/CloneSnapshotFiles scoping. |
Stacked on #120 (review/merge that first; after it lands, rebase this branch onto the updated master and confirm the diff shows only this one commit before merging).
What
The counterpart to #120: paths whose product is rebuilt from a durable source on retry stop paying per-file fsyncs.
PrepareStagingDir) and stream-clone extraction (CloneFromStream):ExtractTargains theSyncModeparameter from fix(snapshot): complete ingestion durability; move capture fsyncs off the pause window #120; these passNoSync.CloneSnapshotFiles→copyPairs): flip toNoSync.ImportkeepsSync;Createextracts withNoSyncand relies on its existingSyncTree(files + dir entries) — removing the double fsync fix(snapshot): complete ingestion durability; move capture fsyncs off the pause window #120 left (each file was synced by ExtractTar, then re-synced by SyncTree), and letting extraction overlap background writeback instead of blocking per file.AtomicWriteFileNoSync/AtomicWriteJSONNoSyncfold into the same parameter — one sync convention across utils, five exported write helpers instead of ten.Crash-semantics change (the part to review)
Interrupted operations converge exactly as before (restore tombstone, create placeholder + GC). What changes: after host power loss, an instance cloned/restored within the writeback window (~30s) may have holes in its COW base. That surfaces as loud guest fs errors on the next cold boot; recovery is recreate (
vm rm+ re-clone/re-restore) — sources stay durable in the snapshot store. Ops note: after a host power-loss reboot, recreate instances cloned/restored in the last minute instead of debugging them.Why
Same measurement as #120: fsync tax ≈ 0.155 s/GiB on the ext4/NVMe floor (host
ai), seconds/GiB on PD-class disks — paid on the clone/restore latency path for zero durability benefit (the extracted/copied working set is consumed immediately from page cache and is replayable).Verification
go build ./...,go vet ./...make test(full suite,-race): 28 packages okmake lint(GOOS=linux + darwin): 0 issues