fix(sync): give durable meta and data phases independent deadline budgets - #2118
Merged
Conversation
…gets On a live 10.0.12 Base-mainnet edge node (2026-08-06) a trivial 604-triple CG could not receive its curator's SWM updates for hours. One contributing defect: the durable requester gave meta and data ONE shared per-CG fetch deadline, so the system-CG meta streams (agents/ontology, ~16k+ triples) over circuit relays burned the whole window and every cycle ended with "Sync timeout for durable data phase (0 triples received)" — the data request was never even sent. Split the per-CG fetch budget into per-phase deadlines, requester-side scheduling only: - DURABLE_META_PHASE_BUDGET_FRACTION (0.4): meta gets at most 40% of the remaining CG budget, so data always keeps the majority share while meta retains enough room to finish or at least advance its resume checkpoint. - DURABLE_DATA_PHASE_MIN_BUDGET_MS (SYNC_MIN_GRAPH_BUDGET_MS / 2): the meta deadline additionally never intrudes on this floor when an operation deadline clamps the CG window below the normal minimum; a window too small to reserve the floor keeps the shared deadline and degrades exactly as before. - Deadlines stay absolute: the data phase runs to the unchanged per-CG fetchDeadline, so a meta phase that finishes early hands its unused time to data automatically and the combined phases can never exceed the prior single-deadline total — no regression in global pacing. - runDurableSync threads a per-phase fetch context (meta gets the bounded deadline, clamped to the CG deadline even for custom budgets), so page-fetch phase timeout logs report against the correct boundary. - DurableSyncContextGraphBudget.metaFetchDeadline is optional: legacy deep-import budgets and the compat adapter keep their single shared deadline, and meta-skipping system CGs keep the full window for their only (data) phase. Wire format untouched. Tests: meta exhausting its slice leaves data exactly the floor; early meta finish rolls unused time into data (95s of a 100s window after a 5s meta); phase deadlines never extend the prior total; skip-agents-meta CGs and legacy positional-ABI callers unaffected. Verified 7/8 new tests fail without the src change (the 8th guards the clamp invariant). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
otReviewAgent
left a comment
otReviewAgent
left a comment
There was a problem hiding this comment.
Operational Notice: Review Agent could not complete this review.
Business logic reviewer failed: WARNING: failed to clean up stale arg0 temp dirs: Permission denied (os error 13)
otReviewAgent
left a comment
otReviewAgent
left a comment
There was a problem hiding this comment.
Operational Notice: Review Agent could not complete this review.
Business logic reviewer failed: WARNING: failed to clean up stale arg0 temp dirs: Permission denied (os error 13)
otReviewAgent
left a comment
otReviewAgent
left a comment
There was a problem hiding this comment.
Operational Notice: Review Agent could not complete this review.
Business logic reviewer failed: WARNING: failed to clean up stale arg0 temp dirs: Permission denied (os error 13)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Durable sync's meta and data phases draw down one shared per-CG deadline. On the affected live 10.0.12 node, the large system-CG meta stream consumed the whole window; the data phase then reported a timeout with zero triples received without sending a request, every reconciler cycle.
Fix
Introduce an optional per-phase deadline inside the existing per-CG budget:
agentsandontology), meta gets at most 40% of the remaining CG window and never consumes the data floor (DURABLE_DATA_PHASE_MIN_BUDGET_MS).agentssync still gives its only data phase the full window.User/runtime impact
A large system directory can no longer consume the complete durable-sync window before the node even asks for its data, while verified user CGs retain their fail-closed cursor semantics. This is additive scheduling behavior; no protocol, storage, or configuration migration is required.
Validation
testnet-canarycomposition builds cleanly; network smoke follows after merge/promotion🤖 Generated with Claude Code and completed by Codex.