v1.34

Catching up without falling over

Faster bootstrap that adapts to the machine doing the work.

Bootstrap is one of the hardest workloads an Atto node faces. A new node has to find missing ranges across thousands of account chains, download them from historical peers, accept transactions that arrive out of order, write a large unchecked backlog, and repeatedly resolve every transaction whose dependencies have become available. Meanwhile, MySQL must maintain account state, transactions, receivables, indexes, and its own redo and undo data. If discovery runs too slowly, bootstrap takes too long. If it runs too quickly, the queue and database can become the bottleneck.

Version 1.34 rebuilds that path around bounded discovery, exact persistence retries, and measured database throughput. In our observed runs, total bootstrap time fell from roughly 15 hours on 1.33 to 6–7 hours on 1.34. That is more than twice as fast, or roughly a 53–60% reduction. Exact times still depend on hardware, disk performance, database configuration, network conditions, and the peers serving history.

Highlights

• Reduced observed end-to-end bootstrap time from about 15 hours to 6–7 hours.
• Replaced unlimited discovery buffering with a bounded queue that pauses new work without discarding admitted transactions during saturation.
• Added adaptive bootstrap pacing based on measured unchecked-transaction insert throughput rather than the location of MySQL or host disk-pressure signals.
• Reworked gap discovery and unchecked queries for large, populated databases.
• Changed unchecked persistence to true multi-row inserts, with batches of up to 1,000 transactions and exact-batch retry after database failures.
• Removed the artificial delay between historical transaction-stream responses.
• Added detailed metrics for discovery backlog, persistence latency, capacity, failures, resolution, cleanup, and controller decisions.
• Upgraded commons from 6.8.1 to 7.0.2 and reduced the native-image minimum heap from 400 MiB to 200 MiB.
• Added signed provenance attestations to published container images.

Bootstrap and database performance

• Gap selection was rewritten as a deterministic indexed query. On a MySQL 8.4 database with roughly 577,000 unchecked transactions, the no-gap path dropped from about 8 seconds to under 200 ms, a roughly 50–60× improvement.
• Migration V11 adds the (timestamp, public_key, height) index used to select the oldest unchecked work efficiently.
• Unchecked inserts now use one multi-row SQL statement per persistence batch instead of driver-level repeated executions.
• Cleanup is bounded, keeping individual delete transactions smaller and reducing lock, undo, and I/O spikes.
• Resolution, gap discovery, and cleanup are centrally scheduled as bounded bootstrap actions instead of independent loops competing for MySQL.
• The controller drains all queued persistence batches before starting more bootstrap discovery or maintenance work.

The adaptive load monitor starts with one batch of admission capacity. It compares measured insert time with the configured throughput target, grows capacity when MySQL keeps up, and reduces it proportionally when saves become slow. Persistence itself is always drained and does not spend bootstrap work credits; the credits limit the work that can discover or process more data.

Lossless backpressure and recovery

• Gap, head, send, and dependency discovery now share one bounded persistence queue.
• New discovery pauses at the adaptive target capacity, while configured headroom allows replies already in flight to finish entering the queue.
• Within a running node process, every admitted transaction remains queued until it is saved, including during saturation.
• A failed save retains the exact batch for retry rather than silently dropping or reshuffling it.
• Gap sessions reserve capacity for expected responses, accept valid out-of-order replies, release reservations on failure or timeout, and recover automatically.
• Disconnected historical peers release their active gap sessions so another peer can continue the work.

Faster historical streams

Version 1.33 intentionally waited 10 ms between transaction-stream responses. A full 1,000-transaction range therefore carried roughly 10 seconds of pacing before network and database time were considered. Version 1.34 removes that delay, rejects stale stream requests after one minute, and coordinates concurrent gap sessions across available historical peers.

This improvement has a network effect. During the first part of the rollout, new 1.34 nodes will still download some history from 1.33 peers. After most historical nodes migrate to 1.34, new nodes may bootstrap faster because more of their transaction streams will be served without the old per-transaction delay.

Observability and configuration

The new bootstrap path exposes metrics for:

• Discovery admission and persistence by source.
• Backlog depth, adaptive capacity, overshoot, queue wait, and at-capacity state.
• Unchecked insert duration, persistence failures, affected rows, and elapsed-to-target ratio.
• Gap rows found, unchecked transactions resolved, cleanup deletions, controller decisions, and available work credit.

The default persistence target is 1,000 unchecked transactions per second. It can be changed with:

ATTO_DISCOVERY_PERSISTENCE_TARGET_PER_SECOND=1000

Queue capacity, headroom, and persistence batch size are also configurable through ATTO_DISCOVERY_CAPACITY, ATTO_DISCOVERY_HEADROOM, and ATTO_DISCOVERY_BATCH_SIZE.

The transaction prioritizer metric transactions.prioritizer.active.elections was renamed to transactions.prioritizer.pending.dependencies to describe what it measures more accurately.

Build and runtime

• Upgraded commons to 7.0.2.
• Lowered the native-image minimum heap from 400 MiB to 200 MiB.
• Refreshed committed GraalVM reachability metadata.
• Published container images now include signed build-provenance attestations.
• Release image aliases are created from the original image manifest, preserving the built image rather than pulling and retagging it locally.

Full changelog: v1.33...v1.34