bridge-monitor: self-healing execution loop (dormant)#1140
Merged
Conversation
added 13 commits
July 13, 2026 15:05
…fix tracker reset
…cution mutex primitive DailySpentUSD only ever incremented and was zeroed by every restart, so a crash-loop minted a fresh budget each time. The tracker resets exactly at UTC midnight, persists (date, spent) to SPEND_STATE_PATH (default ./spend-state.json, not /tmp) and is mutex-guarded. Also declares the package-level single-flight lock and the FAILED_TX_FEE_ESTIMATE_USD (default 0.50) helper for gas bled by failed broadcasts.
…get, single-flight lock Fund-loss fixes from hostile review: - a corrective transfer whose deposit broadcast but whose bridge status never resolved is now TERMINAL for the scheduler slot (funds are likely still in flight, a retry double-sends); only pre-broadcast failures may consume another attempt. executeOnBridge keeps TxHash on error paths so callers can tell the two apart. - ONE corrective-transfer budget (max 2) per scheduler slot, shared across all downgrade-ladder rungs (was per rung: up to 6 transfers per slot). - package-level execution mutex: scheduler slots own the wallets end to end, the reaper TryLocks and skips its tick instead of queueing. - broadcasts that failed or timed out after getting a TxHash book a flat FAILED_TX_FEE_ESTIMATE_USD against the daily cap instead of bleeding unaccounted gas; all spend goes through the persisted UTC-day tracker. - gas top-up now honors MaxDailySpendUSD before broadcasting.
The reaper (and any other consumer) used to accept simulated balances even in production with broadcast-capable keys, so phantom numbers could green-light real transfers. SimulateBalances now takes the execution mode and returns nil outside dry-run; startup logs a warning when the flag is set in a broadcast-capable mode.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Self-healing for the paused execution loop so it never silently stalls once reactivated: non-interactive auto-rebalancer built from the pre-flight simulation's refill fields (cheapest of 3 bridges, per-slot budget of 2 corrective transfers), tier downgrade ladder 300 to 50 to 5 instead of skipping, on-chain balance fallback with a degraded flag (the zeros-on-API-outage bug made every tier look broke), hourly stuck-fund reaper with stranded-hours gauge, gas top-up plumbing gated off by default, UTC-date spend tracker persisted to disk, single-flight mutex across every broadcast path. Slack-only notifications per owner. Two hostile review passes: all first-pass fund-loss findings fixed (double-send on in-flight fills, per-slot budgets, cap reset, nonce races), second-pass residuals fixed (send-ambiguity hash preservation, dry-run state isolation). go build/vet/test -race clean, keyless dry-run smoke verified. Everything stays inert while BENCHMARK_PAUSED=true.