Skip to content

v0.13.15

Choose a tag to compare

@sym-bot sym-bot released this 01 Oct 22:17
· 181 commits to main since this release

A hotfix for one fault: the witness storm. Every node should take it: a node keeps relaying the
storm until it runs this release.

Fixed

  • Two signed copies of one witness were relayed by every node without end. A node that
    witnessed a checkpoint again after a restart signed a second copy of the same statement. The store
    kept only the latest copy per witness, so each copy was new whenever the other arrived, and every
    node stored and relayed it again. On one host this grew one node's witnesses.jsonl to 625,079
    lines (257 MB) of two witnesses, and kept every node in the room busy relaying them. A checkpoint
    is now held once per (attester, position) and a witness once per (attester, position, witness): a
    later copy is a duplicate and is not relayed. A node witnesses a checkpoint once, across restarts.
  • A second root for a held position is a conflict. It is not stored or relayed; the first copy
    stays held. The first time a checkpoint position conflicts, the conflicting copy is appended once
    (so a restart re-derives the conflict), an attestation-conflict metric names both roots, and
    reconcileChain reports conflicted: true, so an attester that signed two roots (by
    equivocating, or by restarting its chain after losing its log) is told apart from tampering. A
    witness that signs a second root raises the same metric (kind: 'witness').
  • A position is an integer. The signed payload spells a position as text, so 8 and "8"
    verified alike and a replayed checkpoint could be stored under many positions; a position that is
    not a non-negative integer is refused, before any signature check.
  • Repeats are cheap. An attestation, checkpoint or witness already held is dropped before its
    signature is checked.
  • A witness is kept only for a checkpoint this node holds. One that arrives first waits in memory
    (at most 2,048) until its checkpoint does, so no peer can grow the witness log with positions
    nobody committed. A node also remembers the checkpoints it witnessed itself, so it signs a witness
    once even after its witness index has dropped it.
  • Checkpoints and witnesses are bounded. At most 32 checkpoints per attester (dropping the oldest
    with its witnesses), for at most 1,024 attesters, and 256 witnesses per checkpoint (50,000 in all,
    the oldest checkpoints' witnesses dropped first).
  • An oversized attestation log is read from its newest part only. Each log is read at start up
    to a budget that covers what its cap holds (attestations 64 MiB, witnesses 32 MiB, checkpoints
    8 MiB); reading the whole of one grown by the storm held a node's thread for minutes. The logs are
    append-only and are not rewritten: one grown by the storm keeps its size on disk, and stops growing.

Not in this release

The logs are still not rotated, and gossiped attestations still have no per-peer budget beyond the
existing per-(CMB, attester) limit (whose table now drops closed windows); both are planned for
0.14.0.