Skip to content

v1.3.10 — WaitForSync settle-window fix

Choose a tag to compare

@twobitapps twobitapps released this 10 Apr 01:30
· 28 commits to main since this release

v1.3.10 — WaitForSync settle-window fix

Hotfix for v1.3.9. The WaitForSync gate I introduced in v1.3.9 had
a bug: it returned "caught up" the instant the first peer Status
handshake arrived, before the peer had time to report its real head.
On a fresh restart against a running network, this let val-2 boot its
consensus engine before catching up, and it produced its own private
fork.

The bug (in v1.3.9)

if peerHead == 0 || localHead+tolerance >= peerHead {
    return true, nil  // "caught up"
}

When val-2 restarted, peers handshaked instantly (14 connections).
But their initial Status messages reported Number=0 because they
were themselves still in their bootstrap phase. The peerHead == 0
shortcut fired and consensus started before the peers had time to
report their real heads.

Observed in v1.3.9 live testnet:

WaitForSync: caught up: local_head=0 peer_head=0 tolerance=10 waited_ms=0 peers=14
... 30 seconds later:
Failed to insert synced block: error="invalid block: invalid parent block"

The fix

WaitForSync now has a three-phase implementation:

  1. Peer wait (≤30s) — block until at least one peer has completed
    the StatusMsg handshake.
  2. Settle window (60s) — SAMPLE the max peer head over time, with
    early-exit if we catch up. The key insight: distinguish "fresh
    bootstrap" (peers stay at head=0) from "late joiner" (at least one
    peer reaches head>0 within the window).
  3. Catch-up — if peers were producing during the settle window,
    sync until local + tolerance >= peer_head with the rest of
    maxWait as the deadline. If all peers stayed at 0 throughout the
    settle window, we're in fresh-bootstrap mode → proceed.

The settle window is the key insight: we observe peer behaviour
over time before deciding what state we're in, instead of trusting
the first reading.

Trade-off

  • Fresh-bootstrap restart adds 60s+30s = 90s to startup (settle window
    • peer wait).
  • Late-joiner restart catches up properly without producing local forks.

The 90s overhead on fresh-bootstrap is acceptable because the alternative
is losing 1+ hours to a fork cascade. Operators who don't want any
sync gate can set --no-sync-wait (TODO — not exposed yet).

v1.3.9 status

chain-v1.3.9 is marked as prerelease so the auto-updater
no longer treats it as latest. v1.3.10 supersedes it. Validators
running v1.3.9 will pick up v1.3.10 on the next 5-minute auto-update
poll.

Upgrade notes

Drop-in upgrade from any prior 1.3.x. No chain data wipe required.
The first restart on v1.3.10 takes ~90s longer than usual due to the
new settle window — this is the intended cost of the fix.

Co-Authored-By: Claude Opus 4.6 (1M context)