test: Refactor flaky eip1559 tests

[0xNeshi]

Seems that all fee-related tests are subject to flakiness.

Some failures we've caught in CI:

• test_estimate_eip1559_fees_with_reward_percentile_based

    thread 'eth_namespace::fee::test_estimate_eip1559_fees_with_reward_percentile_based' panicked at tests/eth_namespace/fee.rs:204:5:
    assertion `left == right` failed
      left: Eip1559Estimation { max_fee_per_gas: 4156, max_priority_fee_per_gas: 0 }
     right: Eip1559Estimation { max_fee_per_gas: 3638, max_priority_fee_per_gas: 0 }

• test_estimate_eip1559_fees_with_default_estimator

    thread 'eth_namespace::fee::test_estimate_eip1559_fees_with_default_estimator' panicked at tests/eth_namespace/fee.rs:104:5:
    assertion `left == right` failed
      left: Eip1559Estimation { max_fee_per_gas: 4157, max_priority_fee_per_gas: 1 }
     right: Eip1559Estimation { max_fee_per_gas: 3639, max_priority_fee_per_gas: 1 }

• test_estimate_eip1559_fees_with_zero_priority_fee
• test_get_fee_history_succeeds

The problem seems to be that the fee changes between two GET calls.

[LeoPatOZ]

eth_feeHistory returns incomplete/inconsistent results due to async cache population race

Summary

eth_feeHistory can return incomplete or empty results when called shortly after blocks are mined. The fee history cache is populated asynchronously by a background service, creating a race window where the cache is stale relative to the actual chain head. This causes non-deterministic behavior in any consumer that queries fee history immediately after block production — including estimate_eip1559_fees().

Root Cause

The fee history cache is populated through a pipeline with two decoupled stages:

Stage 1: Block Mining (Backend::mine_block)

When a block is mined, Backend::mine_block() stores the block and receipts in blockchain.storage, then sends a NewBlockNotification through an unbounded channel:

// crates/anvil/src/eth/backend/mem/mod.rs:1443-1444
// notify all listeners
self.notify_on_new_block(header, block_hash);

At this point the block is fully committed to storage, the chain head is updated, and mine_block() returns — but the fee history cache has not been updated.

Stage 2: Async Cache Population (FeeHistoryService)

The FeeHistoryService runs as a separate future inside NodeService::poll. It receives NewBlockNotification messages and inserts cache entries:

// crates/anvil/src/service.rs:89-90
// poll the fee history task
let _ = pin.fee_history.poll_unpin(cx);

// crates/anvil/src/eth/fees.rs:349-355
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
 let pin = self.get_mut();
 while let Poll::Ready(Some(notification)) = pin.new_blocks.poll_next_unpin(cx) {
 // add the imported block.
 pin.insert_cache_entry_for_block(notification.hash, notification.header.as_ref());
 }

This only runs when the tokio runtime polls the NodeService task — which may not happen before an incoming RPC request reads the cache.

Stage 3: Cache Read (EthApi::fee_history)

The eth_feeHistory RPC handler reads directly from the cache. If a block's entry hasn't been inserted yet, it is silently skipped:

// crates/anvil/src/eth/api.rs:1731-1740
for n in lowest..=highest {
 if let Some(block) = fee_history.get(&n) {
 // ... populate response
 }
}

This means the response can have fewer entries than requested, with no error or indication that data is missing.

The Race

Timeline:
─────────────────────────────────────────────────────────────────────────────────────────────────────────

mine_block()          mine_block()               ...              mine_block()
block 1               block 2                                     block 100
notification sent     notification sent                           notification sent
      |                     |                                           |
      |                     |                                           |
      |         FeeHistoryService polled                                |
      |         (processes blocks 1..42)                                |
      |                     |                                           |
      |                     |       eth_feeHistory(91..=100)            |
      |                     |       cache has [0..42] ← MISS!          |
      |                     |                                           |
      |                     |                    FeeHistoryService polled
      |                     |                    (processes blocks 43..100)
      |                     |                              |
      |                     |       eth_feeHistory(91..=100)
      |                     |       cache has [0..100] ← OK

Reproduction

Confirmed with instrumentation (replacing silent skip with panic in EthApi::fee_history):

Message: fee history cache missing entry for block 91
 (requested range 91..=100, cache has blocks:
 [0, 1, 2, ..., 42])
Location: crates/anvil/src/eth/api.rs:1735

The service had only processed up to block 42 when the RPC call arrived, despite the chain being at block 100.

Impact

• eth_feeHistory returns fewer entries than requested, or empty results
• estimate_eip1559_fees() (which calls eth_feeHistory internally) can return incorrect fee estimates or fail
• Non-deterministic: two identical calls can return different results depending on tokio task scheduling
• Worse under load: more concurrent blocks = larger gap between chain head and cache

Proposed Fix

Insert the fee history cache entry synchronously inside Backend::mine_block(), immediately after the block and receipts are stored in blockchain.storage. This guarantees the cache is always consistent with the chain head before any RPC call can observe the new block.