v5.2.0

Nuthatch 5.2.0 makes address history (RFC-0063) meet rotki's sync on Ethereum mainnet: history up to
the chain head on demand, reorgs taken back, and answers that are exact or refused. Nothing outside an
[address_history] nest changes. An address-history store written by 5.1 is moved aside intact and
the history read again (see Compatibility).

Two cold tests of 5.1.1, by testers with no build history, said not ready. Every item below answers one
of their findings. RFC-0063 §12 has the reasoning.

Exact or refused

• An explicit endblock is answered whole or refused. 5.1.1 shortened an endblock above its
  head to the head and answered status 1, so a client recorded history it never received: 17 of
  Etherscan's 65 rows came back as complete. Now the answer is the whole range or
  NUTHATCH_INCOMPLETE. Without endblock the answer runs to the end of the covered run, and a page
  past the first needs an endblock.
• getblocknobytime without partitions. The 5.1.1 notes said timestamp to block was answered "from
  the RPC or from headers-only partitions". It was not: 5.1.1 answered from verified partitions only,
  and every nest-only rotki sync without them failed. Now a timestamp the partitions do not bracket is
  searched on the nest's own RPC, and the answer is proven against its neighbour block by parent
  hash; above the finalized head the answer must also link, parent hash by parent hash, to the
  chain head the search began from. A timestamp newer than the chain head is refused, and so is any lookup on a chain whose blocks
  can share a second (Arbitrum, BNB Chain).

The tip on demand

rotki ends a sync 15 seconds behind now. The cursor still backfills to the finalized head every poll;
a request above what is covered, and at or below the chain head, is now handed to the cursor, which
covers that address up to the block asked for and answers within request_deadline (new in
[address_history], default 20 s). The cursor stays the only writer: requests wait on a bounded queue
and are served between its windows, so none waits behind another address's backfill. A request past
the chain head is refused. A read reaching above the finalized head also goes through the cursor,
which checks for a reorg first. txhash lookups are traced by the cursor too, up to the chain head,
and refused if the trace names a block the main endpoint does not.

Reorgs

Anything read above the finalized head is read against its window's last block hash, checked before
and after the fetch, together with the newest kept hash under it, and kept with the record. Each poll
and each request above the finalized head compares the newest kept hash with the RPC's, and on a
mismatch takes back everything above the fork: rows, coverage, cached transactions, timestamps,
traces and relay answers. A fetch that began before a reorg keeps nothing above the finalized head,
and a transaction re-included in another block is read again. A fetch is checked against the newest
kept hash under it, so a window read on a fork cannot hide a stale block beneath its own matching
anchor; every block a trace names must be the main endpoint's, finalized or not; and the trace
endpoint must report the main endpoint's hash at each window's last block, so a trace source on
another fork or behind cannot answer an empty range the main chain does not have. Each check is made
twice, so a node that changes fork between comparisons cannot pass. A fetch that fails a check keeps
nothing. A head that fell under a kept block counts as a reorg. Kept hashes survive a restart, and a reorg the chain finalized
while the nest was down is still taken back. Only a block under the
finalized head the store committed is a conflict, which stops the cursor with the reason on /ready.

Finalized history is checked too. The store keeps the hashes of the last 32 finalized heads it
committed, and every poll, the first after a restart included, compares the newest and one older one
in turn with the RPC. A mismatch, which is a reorg under finality or an RPC that came back on another
chain under the same chain id, stops the cursor with the reason on /ready, and every answer is
refused, an answer already being prepared included. The conflict is kept in the store, so a restart
does not clear it; removing the address-history store does. Every fetch is also held against the
newest finalized hash, so a fork switch after a request's own check cannot join two forks. Each
check asks every endpoint in the pool, and the finalized head advances only where they all agree. A new finalized head's
hash is read before and after the checks and committed with the head in one transaction.

Two property tests over random reorg depths gate it, one through the cursor's own window against a
node whose traces follow the chain, with reorgs that cross finality.

rotki's ordinary Etherscan sync can import unfinalized history and does not automatically reconcile
already imported history after a reorg. The nest integration retains this shared limitation.

Produced blocks and MEV

• Payout contracts. Builders that pay the proposer through a payout contract pay as an internal
  transfer inside the block's last transaction. Those are now found: the cold test's 16 such payments
  over 265,000 blocks had come back as no MEV.
• unrecorded, not none. The rows come from the chain; MEV is a label: relay, conflict, or
  unrecorded (no delivery at the queried relays when last asked). A relay saves its delivery record
  after the block is published and can fail to, so unrecorded blocks and builder payments without a
  row are asked again daily for 30 days, and a delivery found then upgrades or inserts the row. The
  schedule commits with the coverage, so a crash cannot drop it.
• Finalized only. Produced blocks are served through the finalized head; a request past it is
  refused with the block it is served through.
• First in each pass. Produced blocks and the re-asks are covered before the backfill, so a long
  backlog cannot hold them back.
• Every leg. A builder payment split across several transactions reports every hash in
  paymentTx and their sum in paymentValue.

Smaller fixes

• logs/getLogs honours page and offset, starts with 16-block spans that grow only while they
  come back sparse, stops once it holds the page, and narrows any span whose answer passes 4 MiB, so
  a dense range no longer holds every log in memory; two run at a time. A range that needs more than 64 calls answers NUTHATCH_INCOMPLETE, as
  does a failed range, rather than an error.
• The block-body scan keeps each block's miner, hash, timestamp and withdrawals for the newest
  100,000 blocks, so an address watched later reads those rather than the chain.
• /ready reports stalled: true while the last poll failed; rpc_requests_total is current; a
  refusal always names its reason; addresses = [] is valid; a bind other than loopback warns; and
  unwatching one address no longer throws away another's window in flight.
• /ready answers 503 until the first finalized head is read, still with chain_id.
• A span retried after a timeout or a failure logs at warn, and a failed reconcile counts on
  poll_errors_total: a nest whose RPC was failing went sixteen minutes without a log line.
• The address-history store's page cache is 32 MiB rather than redb's 1 GiB default. It did not
  change the mirrored nest's idle figure on macOS, where the cold test saw 240 MB: see Measured.

Measured

All on Ethereum mainnet over Alchemy.

• Moving head against Etherscan. One address (0x9522…bafe5) over a 600-block window ending at the
  head. rotki's lookup of now minus 15 seconds returned the same block as Etherscan's, and txlist
  (21 rows), txlistinternal (3) and tokentx (0) up to it were identical to Etherscan's, every
  shared field equal (17 per transaction row, 14 per internal row). Three minutes later the head had
  moved 18 blocks; the same queries were covered on demand and identical again. 687 RPC calls in
  all, 572 of them the block-body scan of the window.
• Start. 206 ms from a covered store to serving, 36 MB RSS after.
• Memory with a mirror (26 partitions, 25.9M to 26.16M, one address). On Linux (jemalloc): 5.1.1
  peaked at 108 MB and idled at 36 MB, 5.2.0 peaked at 112 MB and idled at 48 MB. On macOS RSS
  idled at 129 MB (5.1.1) and 140 MB (5.2.0), but 5.2.0's physical footprint was 68 MB (peak 84.5):
  the rest is freed allocator regions macOS keeps resident, and a restart idles at 32 MB. The cold
  test's 240 MB was this macOS figure.
• At rest a poll costs one more call than before (the RPC's finalized tag) and one more while
  hashes above the finalized head are kept.

Gated on Lodestar's data

Every change is in the address-history path, which no production nest declares, so this release changes
no query path the release gate measures.

Compatibility and upgrade

Replace the binary and restart. An address-history store written by 5.1 has no finalized block hashes,
so it cannot show that its finalized history is the chain's. At the first start it is renamed, intact,
to nuthatch.redb.pre-5.2-<unix time> beside where it was, one warning line says so and where it went,
and the nest reads its history again from start_block, keeping every account the old store watched.
Nothing is deleted: the old file can be kept, inspected or removed by hand. The rebuild costs what the
first sync did; getminedblocks, withdrawals and produced blocks on chains other than Ethereum mainnet
are unchanged.