v1.3.1 — Consensus Determinism Fix
Critical Consensus Determinism Fix
Fixes a four-way chain fork discovered in production on 2026-04-09 where each of
the 4 infrastructure validators was producing its own independent chain since
genesis, despite all starting from the same genesis block.
Verified in production: deployed to the 4-validator + 3-bootnode testnet;
all 4 validators now produce identical block hashes at every height.
Root causes
Three concurrent non-determinism bugs in NarwhalTusk consensus caused each
validator to derive a different block header from the same commit decision:
-
Randomness beacon (
randomness_beacon.go) — When BLS was disabled, the
beacon fell back to hashing ECDSA vote signatures from the committed anchor.
Different validators commit different anchors on different forks, so each
derived a different leader-election seed and picked a different round
leader. Fix: beacon now only publishes seeds when the anchor carries a
BLS aggregate (which is deterministic regardless of which validator
aggregates). In ECDSA-only mode, the beacon is empty and
computeLeaderWithBeaconfalls back todeterministicSeed(round), which
depends only on the round number and is identical everywhere. -
Block timestamp (
consensus.go) —Block.Timewas set from
time.Now(), stamping each validator's wall-clock onto the block header,
so every validator produced a distinct hash from the same commit decision.
Fix:blockTime = parent.Time + 1— monotonic and deterministic. -
Block coinbase (
consensus.go) — Every validator set the coinbase to
its OWN address instead of the round leader, so block headers were unique
by construction. Fix: coinbase is now the deterministic leader of the
committed round, computed from the same round number and shared validator
set on every node.
Additional fixes
-
Optimistic wave skipping (
bullshark.go) — When quorum couldn't be
reached on a leader, each validator unilaterally skipped the wave based on
LOCAL wall-clock elapsed time, producing divergent committed-round
sequences. Disabled: waves only advance via the standard view-change
path (2f+1 votes). If a validator can't reach quorum, consensus stalls —
which is the correct BFT behaviour; producing different blocks on each node
is actively worse than stalling. -
Bootstrap delay (
consensus.go) — 30s settle delay before starting
consensus so all validators can discover each other via bootnodes/DHT
before producing certs. The previous 5s was insufficient — the slowest
validator started consensus before the DAG was fully gossipped, producing
its own empty blocks from genesis and creating a permanent fork.
Regression tests
TestPublishCommittedSeed_SkipsNonBLS— beacon refuses to publish seeds
from ECDSA-only anchors.TestLeaderElection_DeterministicWithoutBLS— 4 independent beacons
committing divergent anchors still elect the same leader for every round.
Upgrade notes
Chain data MUST be wiped on upgrade — the pre-v1.3.1 chain was forked and has
no single canonical history. Operators running the auto-updater will receive
v1.3.1 automatically within 5 minutes; chain data wipe needs to be coordinated
with the infrastructure team.
Files changed
consensus/narwhaltusk/randomness_beacon.goconsensus/narwhaltusk/consensus.goconsensus/narwhaltusk/bullshark.goconsensus/narwhaltusk/randomness_beacon_test.go(new)main.go,cmd/version.go(version bump 1.3.0 → 1.3.1)