diff --git a/CHANGELOG.md b/CHANGELOG.md index b7735803ca2..38eee8df53c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,164 @@ -## Unreleased - -No changes yet :) +# Release Versioning + +Releases for the tests library follow [semantic versioning](https://semver.org/): + +- Major number updates every time there is a backwards incompatibility for test runners, e.g. + - New fork rules + - Backwards-incompatible format changes +- Minor number updates when new tests are added that are backwards compatible +- Patch number updates if an existing test is bugfixed + +Here is an example how a follow-up release line could look like: + +- v6-alpha - starting to implement EIPs, but not all EIPs are finalized +- v6-beta - all EIPs finalized, but some tests still in progress +- v6.0.0 - all desired coverage for Constantinople tests to be considered "done enough" +- v6.1.0 - more coverage added +- v6.2.0 - more coverage added +- v6.2.1 - bugfix on one of the tests +- v7.0.0 - change in test format output (if backwards incompatible) + +# Release Notes + +## v7.0.0-beta.1 + +This is the first `Ethereum` tests release with broader `Istanbul` support, +see the "Istanbul support" section for a list with relevant PRs. + +This release also comes with some structural changes to the test folder layout, +the most important ones being the introduction of a new separate `LegacyTests` +test suite for state tests up to `Constantinople`, a new sub folder structure +for `BlockchainTests` and a new separate suite `GeneralStateTests/stTimeConsuming/` +for time consuming tests. For further details please have a look at the +"Test Format Changes" section. + +### Istanbul Support + +#### EIP Test Support + +- [EIP-152](https://github.com/ethereum/EIPs/pull/2129): Blake2b `F` precompile, + `CALL` and `CALLCODE` tests added for the standard unit test vectors for + Blake2b `F`, + PR [#619](https://github.com/ethereum/tests/pull/619) +- [EIP-1344](https://eips.ethereum.org/EIPS/eip-1344): ChainID opcode, state + tests added in PR [#627](https://github.com/ethereum/tests/pull/627) +- [EIP-1884](https://eips.ethereum.org/EIPS/eip-1884): Repricing for + trie-size-dependent opcodes, `SELFBALANCE` and `SLOAD` gas cost tests added + in PR [#627](https://github.com/ethereum/tests/pull/627) + +#### Test Regeneration + +- Regeneration of `BlockchainTests/GeneralStateTests` (`hive` tests), + PR [#632](https://github.com/ethereum/tests/pull/632) +- Updated `TransactionTests/` to `Istanbul`, + PR [#633](https://github.com/ethereum/tests/pull/633) +- Updated state tests with latest `lllc`, + PR [#635](https://github.com/ethereum/tests/pull/635) +- Updated `GeneralStateTests` to `Istanbul`, + PR [#639](https://github.com/ethereum/tests/pull/639) + +### Constantinople/Petersburg Updates + +The following tests touching Constantinople/Petersburg behavior have been added +or updated since the last release: + +- Added `ConstantinopleFix` (aka `Petersburg`) tests, + PR [#582](https://github.com/ethereum/tests/pull/582) +- Updated `ByzantiumToConstantinople` transition test to + `ByzantiumToConstantinopleFix` + (see `BlockchainTests/TransitionTests`), + PR [#583](https://github.com/ethereum/tests/pull/583), + PR [#588](https://github.com/ethereum/tests/pull/588) +- New `SAR`, `SHL`, `SHR` combinations, + PR [#574](https://github.com/ethereum/tests/pull/574) + +### Test Format Changes + +- New `LegacyTests` suite for `BlockchainTests/GeneralStateTests` and + `GeneralStateTests` for HFs up to `Constantinople` (so not: + `ConstantinopleFix` aka `Petersburg`), + PR [#623](https://github.com/ethereum/tests/pull/623) +- New subfolder structure for `BlockchainTests` with added folders + for `InvalidBlocks` and `ValidBlocks`, + PR [#605](https://github.com/ethereum/tests/pull/605) +- New separate suite `GeneralStateTests/stTimeConsuming/` for time + consuming tests, + PR [#595](https://github.com/ethereum/tests/pull/595) +- Moved blockchain specific tests from `GeneralStateTests` to + `BlockchainTests`, + PR [#590](https://github.com/ethereum/tests/pull/590) +- Old unmaintained RPC test scripts in `RPCTests` have been removed, + PR [#573](https://github.com/ethereum/tests/pull/573) + +### Retesteth / RPC + +- Added `Istanbul` to `Retesteth` configuration, + PR [#638](https://github.com/ethereum/tests/pull/638) +- Updated `Retesteth` configurations, + PR [#634](https://github.com/ethereum/tests/pull/634) +- Default `Retesteth` genesis configurations, + PR [#625](https://github.com/ethereum/tests/pull/625) +- Added `Retesteth` configuration for `Pantheon` client, + PR [#622](https://github.com/ethereum/tests/pull/622) +- Autokill `Geth` threads in `startGethThreads.sh`, + PR [#613](https://github.com/ethereum/tests/pull/613) +- Added fork configurations to `Geth` config, + PR [#602](https://github.com/ethereum/tests/pull/602) +- Fixed `BlockchainTests` `RPC` issues, + PR [#594](https://github.com/ethereum/tests/pull/594) + +### General Test Additions + +- More `RLP` invalid tests for non-optimal lengths + (see `RLPTests/`), + PR [#612](https://github.com/ethereum/tests/pull/612) +- More tests on touching precompiles along `REVERT` usage + (see `GeneralStateTests/stRevertTest/*Touch*.json`), + PR [#580](https://github.com/ethereum/tests/pull/580), + PR [#610](https://github.com/ethereum/tests/pull/610) +- New `sStore` test with non-zero nonce on collision + (see `GeneralStateTests/stSStoreTest/InitCollisionNonZeroNonce.json`), + PR [#578](https://github.com/ethereum/tests/pull/578) + +### Bug Fixes/Optimizations + +- Corrected tests with a missing `expect` section, + PR [#624](https://github.com/ethereum/tests/pull/624) +- Refill of all `BlockchainTests/GeneralStateTests/`, + PR [#621](https://github.com/ethereum/tests/pull/621) +- Removed post sections with no post conditions, + PR [#618](https://github.com/ethereum/tests/pull/618) +- Corrected huge expect sections in state tests + (see `stAttackTest/`, 'stQuadraticComplexity/'), + PR [#617](https://github.com/ethereum/tests/pull/617) +- Compressed huge state data in `bcExploitTest` tests, + PR [#616](https://github.com/ethereum/tests/pull/616) +- Removed ambiguous test in + `BlockchainTests/ValidBlocks/bcMultiChainTest/ChainAtoChainB_blockorder2.json`, + PR [#615](https://github.com/ethereum/tests/pull/615) +- Fixed `RevertPrecompiledTouch` test, + PR [#609](https://github.com/ethereum/tests/pull/609) +- Converted blockchain specific state tests into `BlockchainTests`, + PR [#607](https://github.com/ethereum/tests/pull/607) +- Various test fixes, + PR [#603](https://github.com/ethereum/tests/pull/603) +- Fixes and updates to various state tests, + PR [#599](https://github.com/ethereum/tests/pull/599) +- Fixed `0x` missing in `transaction` -> `data` in `GeneralStateTests`, + PR [#598](https://github.com/ethereum/tests/pull/598) +- Removed a test case from `badOpcodes` state tests + (see `GeneralStateTests/stBadOpcode/`), + PR [#592](https://github.com/ethereum/tests/pull/592) +- Fixed `gasLimit` issue in `GeneralStateTests`, + PR [#590](https://github.com/ethereum/tests/pull/590) +- Consistently use `0x` prefixes in `RLPTests`, + PR [#587](https://github.com/ethereum/tests/pull/587) +- Removed underspecified `lotsOfBranches` test + (see `BlockchainTests/bcTotalDifficultyTest/lotsOfBranches.json`), + PR [#579](https://github.com/ethereum/tests/pull/579) +- Changed `gasUsed` to 0 in `genesisBlockHeader` for `dataTx` tests + (see `BlockchainTests/bcValidBlockTest/dataTx.json` and `dataTx2.json`), + PR [#577](https://github.com/ethereum/tests/pull/577) ## v6.0.0-beta.3