Skip to content

tests@v21.0.2

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 26 Oct 16:29
tests@v21.0.2
e27f097
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This is a patch follow-up to tests@v21.0.1 containing no consensus changes for Frontier through Amsterdam.

Key Updates

  • Ready-to-run hive simulator images (#3572, #3883, ethereum/hive#1609): releases are now also published as images (from v21.0.1 on), making first-time EELS simulator builds 8x faster. Use --sim.buildarg tag=v21.0.2, or latest with --docker.pull; see the docs. @danceratopz @spencer-tb
  • Synthetic BPO test forks follow Amsterdam (#3597): renamed to AmsterdamBPOIncrease and AmsterdamBPODecrease, and blob gas tests now also run on BPO1 and BPO2; see BPO Test Forks. @danceratopz
  • New Merkle Patricia Trie structure suite (#3600): 116 cases on Osaka and Amsterdam that force specific trie shapes across the storage, account, withdrawals, transactions and receipts tries. @CPerezz
  • Amsterdam invalid-transaction blocks carry a complete BAL (#3848): the rejected transaction's addresses are now in the block access list, so clients should report the transaction exception rather than a BAL mismatch. @marioevz @fselmo
  • Engine x fixtures carry syncPayloads (#3364): an extra block above the test chain that lets a consumer run the whole test through devp2p sync. It's additive; older framework versions and consume enginex ignore it. See the format docs. @danceratopz @marioevz
  • 32% fewer engine x pre-allocation groups (#3758, #3876): 3,337 to 2,255 on the forks both releases fill, from reworking the ported static tests; see Engine X Pre-allocation Groups. @marioevz

Tests

Test cases, compared to the previous release, tests@v21.0.1, change in count as follows:

Test type v21.0.1 v21.0.2 Added Removed Changed Unchanged
State test 83,367 83,555 7,389 7,201 15,159 61,007
Blockchain test 20,791 21,343 1,812 1,260 306 19,225
Transaction test 720 750 30 0 0 720
  • The large number of test additions and removals is due to renames and reparametrizations: the intrinsic gas relabel (#3837, 3,780 state tests out and back in under new ids), the dropped blob base fee parametrization (#3794), the call-chain collapse (#3876) and the removed @bigmem id suffix (#3364).
  • The new cases are mainly the blob tests on the new BPO forks (#3597, 810 blockchain tests) and the MPT suite (#3600).
Expand to see the changes by fixture format.

State tests are also filled as blockchain fixtures; engine fixtures exist only from Paris on, so pre-merge forks have blockchain_test fixtures without an engine counterpart (21,288 in this release):

Fixture format v21.0.1 v21.0.2 Added Removed Changed Unchanged
state_test 83,367 83,555 7,389 7,201 15,159 61,007
blockchain_test 104,063 104,803 9,201 8,461 15,955 79,647
blockchain_test_engine 83,258 83,598 8,027 7,687 15,937 59,634
blockchain_test_engine_x 83,258 83,598 8,027 7,687 72,258 3,313
blockchain_test_sync 18 18 18 18 0 0
transaction_test 720 750 30 0 0 720
Total 354,684 356,322 32,692 31,054 119,309 204,321

New Tests

Changed Tests

  • Engine x: 72,258 of 83,598 fixtures change, from the new syncPayloads field (#3364) and the repacked pre-allocation groups (#3758, #3876, #3752). Old pre_alloc folders cannot be mixed with this release's engine x fixtures. @danceratopz @marioevz
  • Ported static: 43,684 fixtures change, from the pre-allocation group reduction (#3758). test_eoa_empty_paris is rewritten as test_external_account_empty_paris with explicit account kinds. @marioevz
  • Amsterdam invalid-transaction blocks: the block access list now includes the rejected transaction's sender, recipient and authorization signers, so the rejected transaction is the only reason the block is invalid. Clients that read state from the BAL before validating the transaction previously rejected these blocks for the BAL instead. 2,334 blockchain and engine fixtures (#3848) @marioevz @fselmo
  • Intrinsic gas: from Prague a transaction must cover both the standard intrinsic cost and the calldata floor, and the tests now expect the exception for the threshold the gas limit actually misses, or either exception when it misses both. The EIP-7623 suite also sizes its calldata against the full Amsterdam floor (#3837) @fselmo @LouisTsai-Csie
  • test_tx_intrinsic_gas now sends the calldata it names: its data was written as Bytes(b"0x00"), which sent the ASCII text 0x00 instead of the byte. Its below_intrinsic_True cases are now below_standard_cost, below_calldata_floor or below_standard_cost_and_calldata_floor (#3837) @fselmo @LouisTsai-Csie
  • Invalid blocks: the receipts root in the header of a block with a rejected transaction is computed from the accepted transactions only (#3849, #3361) @alok-108 @skbaek @fselmo
  • EIP-4844 fork transition (test_fork_transition_excess_blob_gas_post_blob_genesis): base fees are now derived from each fork's blob parameters so the EIP-7918 reserve price cases keep exercising the reserve branch; parametrized as reserve_inactive / reserve_boundary / reserve_active (#3794, #3721, #3597) @shubham5080 @spencer-tb @danceratopz
  • Amsterdam and Cancun tests converted to relative balance and nonce post-state expectations, 733 fixtures: the funded sender accounts and gas prices change, the checked behaviour does not (#3699) @marioevz @fselmo
  • EIP-2537 pairing: gas values derived from fork parameters, 18 fixtures (#3772, #3641) @LouisTsai-Csie @rorupuntou
  • EIP-7002 / EIP-7251 out-of-gas system contract tests no longer hard-code gas limits and are reparametrized: 36 fixtures added, 24 removed (#3457) @zexoverz @fselmo

Removed and Renamed Tests

  • The 183 ported call-chain files in stCallCodes, stCallDelegateCodesHomestead and stCallDelegateCodesCallCodeHomestead are removed in favour of test_call_chain.py (#3876): 2,274 fixtures. Every chain they covered is still tested; duplicate DELEGATECALL-only chains had identical fixtures. @marioevz
  • The Osaka-line BPO transitions BPO2ToBPO3AtTime15k and BPO3ToBPO4AtTime15k are removed; their scenarios now follow Amsterdam (#3597, see BPO Test Forks). @danceratopz
  • test_blob_txs.py drops the block_base_fee_per_gas [7, 100] parametrization (v21.0.1) from test_valid_blob_tx_combinations and test_sufficient_balance_blob_tx (#3794): test_reserve_price_various_base_fee_scenarios already covers the reserve price. This removes 1,046 unique cases and adds 523 under the new ids. @shubham5080
  • test_tx_intrinsic_gas ids change from below_intrinsic_False / below_intrinsic_True to at_intrinsic and the threshold missed (#3837, see Changed Tests). @fselmo @LouisTsai-Csie
  • 1,772 test ids lose an @bigmem suffix that leaked from the xdist group into the id (#3364). The fixtures are identical; only the id changes. @danceratopz

BPO Test Forks

The test-only BPO scenarios moved from Osaka's BPO2 to Amsterdam and were renamed (#3597). Clients now place BPO3 and later after Amsterdam, and these scenarios check that a client handles a BPO fork after a named fork.

Scenario Blob target / max / update fraction Cases
AmsterdamBPOIncrease 21 / 32 / 17,527,006 190
AmsterdamBPODecrease 9 / 14 / 7,511,574 190
AmsterdamToBPOIncreaseAtTime15k Amsterdam to Increase 26
AmsterdamToBPODecreaseAtTime15k Amsterdam to Decrease 24
BPO1, BPO2 mainnet schedule 190 each
  • Both scenarios use the next free client BPO slot, HIVE_BPO3_*, which hive already maps.
  • Every excess blob gas and blob base fee test that is not marked slow now also runs on BPO1, BPO2 and the two scenarios, with the blob counts at the edges of each fork's schedule. Other forks keep their full ranges.
  • New test: a block above a lowered blob max is rejected at the activation block.
  • Removed: the old Osaka-line transitions BPO2ToBPO3AtTime15k (25 cases) and BPO3ToBPO4AtTime15k (23 cases).

Engine X Pre-allocation Groups

On the forks both releases fill, engine x needs 32% fewer pre-allocation groups (3,337 to 2,255), and so 1,082 fewer genesis initializations in consume enginex. Most of the drop is on Cancun to Amsterdam, where the ported static tests live (#3758, #3876). The new BPO forks add 308 groups. Counts come from each release's blockchain_tests_engine_x/pre_alloc/ files.

Forks Groups v21.0.1 Groups v21.0.2 Change Pre-alloc v21.0.1 (MB) Pre-alloc v21.0.2 (MB)
Paris and Shanghai 162 162 +0% 9.6 9.7
Cancun 654 357 -45% 10.3 9.5
Prague 718 427 -41% 26.6 20.6
Osaka 765 512 -33% 26.6 21.4
Amsterdam 997 754 -24% 42.0 36.3
Other transitions 41 43 +5% 5.9 6.0
Old BPO3/BPO4 transitions 11 – – 0.2 –
New BPO forks – 308 – – 7.0
Total 3,348 2,563 -23% 121.3 110.3

New Contributors

Full Changelog: https://github.com/ethereum/execution-specs/compare/tests@v21.0.1...tests@v21.0.2

Contributors

fselmo, marioevz, and 17 other contributors
Assets 3
Loading