Blockchain Tests Format Changes #464

winsvega · 2018-05-30T13:11:26Z

A small change to the blockchain test format is needed.
This format is used by have and many clients so I want to keep the changes at minimum.
However we do not have any agreed format scheme so far. I think some fields could be removed as it is obsolite.

The current blockchain format looks like this:

{
    "add11_d0g0v0_Byzantium" : {
        "_info" : {
            "comment" : "",
            "filledwith" : "cpp-1.3.0+commit.8fb57c56.Linux.g++",
            "lllcversion" : "Version: 0.4.20-develop.2018.1.8+commit.2548228b.Linux.g++",
            "source" : "src/BlockchainTestsFiller/GeneralStateTests/stExample/add11_d0g0v0Filler.json",
            "sourceHash" : "9808cacd8f347355858cefd1d92ef14147e16909768505207c42e0340d4fb1b6"
        },
        "blocks" : [
            {
                "blockHeader" : {
                    "bloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
                    "coinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
                    "difficulty" : "0x020000",
                    "extraData" : "",
                    "gasLimit" : "0x0f4240",
                    "gasUsed" : "0xa034",
                    "hash" : "0x758938e476972612fd1756c62b54ffb18f8c30b51c37d7ab25b94869e0f76e66",
                    "mixHash" : "0x80fec5d1292bfe4fd831b54a7559389e5b98f13c2fce165cd3e246b228adc0c4",
                    "nonce" : "0xf44b34eca2bf9916",
                    "number" : "0x01",
                    "parentHash" : "0x26f4cb1cb9685bad697c0c16ca0d27fe0a4b09364f83ed8418cef0d523783625",
                    "receiptTrie" : "0x5a0b89fb1e5f44d02b5900c39683a1796c591b2c3b08e3b1bfc94f5aa4f3c44d",
                    "stateRoot" : "0xb15e96a28489e2cf9e6458dd731ba124fe6f472c11417227e934a3f40af1bbfd",
                    "timestamp" : "0x03e8",
                    "transactionsTrie" : "0x93ca2a18d52e7c1846f7b104e2fc1e5fdc71ebe38187248f9437d39e74f43aab",
                    "uncleHash" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
                },
                "rlp" : "0xf90262f901f7a026f4cb1cb9685bad697c0c16ca0d27fe0a4b09364f83ed8418cef0d523783625a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942adc25665018aa1fe0e6bc666dac8fc2697ff9baa0b15e96a28489e2cf9e6458dd731ba124fe6f472c11417227e934a3f40af1bbfda093ca2a18d52e7c1846f7b104e2fc1e5fdc71ebe38187248f9437d39e74f43aaba05a0b89fb1e5f44d02b5900c39683a1796c591b2c3b08e3b1bfc94f5aa4f3c44db90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008302000001830f424082a0348203e880a080fec5d1292bfe4fd831b54a7559389e5b98f13c2fce165cd3e246b228adc0c488f44b34eca2bf9916f865f863800183061a8094095e7baea6a6c7c4c2dfeb977efac326af552d87830186a0801ca0e94818d1f3b0c69eb37720145a5ead7fbf6f8d80139dd53953b4a782301050a3a01fcf46908c01576715411be0857e30027d6be3250a3653f049b3ff8d74d2540cc0",
                "transactions" : [
                    {
                        "data" : "0x",
                        "gasLimit" : "0x061a80",
                        "gasPrice" : "0x01",
                        "nonce" : "0x00",
                        "r" : "0xe94818d1f3b0c69eb37720145a5ead7fbf6f8d80139dd53953b4a782301050a3",
                        "s" : "0x1fcf46908c01576715411be0857e30027d6be3250a3653f049b3ff8d74d2540c",
                        "to" : "0x095e7baea6a6c7c4c2dfeb977efac326af552d87",
                        "v" : "0x1c",
                        "value" : "0x0186a0"
                    }
                ],
                "uncleHeaders" : [
                ]
            }
        ],
        "genesisBlockHeader" : {
            "bloom" : "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
            "coinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
            "difficulty" : "0x020000",
            "extraData" : "0x42",
            "gasLimit" : "0x0f4240",
            "gasUsed" : "0x00",
            "hash" : "0x26f4cb1cb9685bad697c0c16ca0d27fe0a4b09364f83ed8418cef0d523783625",
            "mixHash" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "nonce" : "0x0102030405060708",
            "number" : "0x00",
            "parentHash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
            "receiptTrie" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "stateRoot" : "0x4b4b7a0d58a2388c0e6b3b048c3c27edd6febc6f04171167ed15a77ab2e60b16",
            "timestamp" : "0x03b6",
            "transactionsTrie" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "uncleHash" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
        },
        "genesisRLP" : "0xf901faf901f5a00000000000000000000000000000000000000000000000000000000000000000a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942adc25665018aa1fe0e6bc666dac8fc2697ff9baa04b4b7a0d58a2388c0e6b3b048c3c27edd6febc6f04171167ed15a77ab2e60b16a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008302000080830f4240808203b642a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421880102030405060708c0c0",
        "lastblockhash" : "0x758938e476972612fd1756c62b54ffb18f8c30b51c37d7ab25b94869e0f76e66",
        "network" : "Byzantium",
        "postState" : {
            "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
                "balance" : "0x0de0b6b3a76586a0",
                "code" : "0x6001600101600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0x02"
                }
            },
            "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba" : {
                "balance" : "0x29a2241af62ca034",
                "code" : "",
                "nonce" : "0x00",
                "storage" : {
                }
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
                "balance" : "0x0de0b6b3a761d92c",
                "code" : "",
                "nonce" : "0x01",
                "storage" : {
                }
            }
        },
        "pre" : {
            "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x6001600101600055",
                "nonce" : "0x00",
                "storage" : {
                }
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "",
                "nonce" : "0x00",
                "storage" : {
                }
            }
        }
    }
}

I have found out that this genesis fields are the minimum of information that is required for chain initialization:

    data["author"] = m_genesisHeader.getData().at("coinbase");    // Miner reward address
    data["difficulty"] = m_genesisHeader.getData().at("difficulty");   // Block difficulty
    data["gasLimit"] = m_genesisHeader.getData().at("gasLimit");  // Block gasLimit
    data["nonce"] = m_genesisHeader.getData().at("nonce");         // for current tests compatibility.  could be dropped.
    data["extraData"] = m_genesisHeader.getData().at("extraData");    // for current tests compatibility. could be dropped.
    data["timestamp"] = m_genesisHeader.getData().at("timestamp");  // Block timestamp for difficulty calc
    data["mixHash"] = m_genesisHeader.getData().at("mixHash");       // for current tests compatibility. could be dropped.

We don't really need rlp decoded fields;
The changed format could be like this:

{
    "add11_d0g0v0_Byzantium" : {
        "_info" : {
            "comment" : "",
            "filledwith" : "cpp-1.3.0+commit.8fb57c56.Linux.g++",
            "lllcversion" : "Version: 0.4.20-develop.2018.1.8+commit.2548228b.Linux.g++",
            "source" : "src/BlockchainTestsFiller/GeneralStateTests/stExample/add11_d0g0v0Filler.json",
            "sourceHash" : "9808cacd8f347355858cefd1d92ef14147e16909768505207c42e0340d4fb1b6"
        },
        "blocks" : [
            {
                "rlp" : "0xf90262f901f7a026f4cb1cb9685bad697c0c16ca0d27fe0a4b09364f83ed8418cef0d523783625a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347942adc25665018aa1fe0e6bc666dac8fc2697ff9baa0b15e96a28489e2cf9e6458dd731ba124fe6f472c11417227e934a3f40af1bbfda093ca2a18d52e7c1846f7b104e2fc1e5fdc71ebe38187248f9437d39e74f43aaba05a0b89fb1e5f44d02b5900c39683a1796c591b2c3b08e3b1bfc94f5aa4f3c44db90100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008302000001830f424082a0348203e880a080fec5d1292bfe4fd831b54a7559389e5b98f13c2fce165cd3e246b228adc0c488f44b34eca2bf9916f865f863800183061a8094095e7baea6a6c7c4c2dfeb977efac326af552d87830186a0801ca0e94818d1f3b0c69eb37720145a5ead7fbf6f8d80139dd53953b4a782301050a3a01fcf46908c01576715411be0857e30027d6be3250a3653f049b3ff8d74d2540cc0",
                 "exceptions" : "GasLimitReached"
            }
        ],
        "genesisBlockHeader" : {
            "coinbase" : "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba",
            "difficulty" : "0x020000",
            "extraData" : "0x42",
            "gasLimit" : "0x0f4240",
            "mixHash" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
            "nonce" : "0x0102030405060708",
            "timestamp" : "0x03b6",
        },
        "lastblockhash" : "0x758938e476972612fd1756c62b54ffb18f8c30b51c37d7ab25b94869e0f76e66",
        "network" : "Byzantium",
        "engine" : "NoProof", 
        "postState" : {
            "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
                "balance" : "0x0de0b6b3a76586a0",
                "code" : "0x6001600101600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0x02"
                }
            },
            "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba" : {
                "balance" : "0x29a2241af62ca034",
                "code" : "",
                "nonce" : "0x00",
                "storage" : {
                }
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
                "balance" : "0x0de0b6b3a761d92c",
                "code" : "",
                "nonce" : "0x01",
                "storage" : {
                }
            }
        },
        "pre" : {
            "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "0x6001600101600055",
                "nonce" : "0x00",
                "storage" : {
                }
            },
            "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b" : {
                "balance" : "0x0de0b6b3a7640000",
                "code" : "",
                "nonce" : "0x00",
                "storage" : {
                }
            }
        }
    }
}

This is a small change.

If a block is invalid then there will be field "exceptions" : "exceptionName". Clients have different exceptions so the exception name is just a comment. If there is no such field the block must be valid and imported without issues.

Sections Transactions and Uncles are removed. The information about it is in RLP.
New field "engine" : "NoProof", - meaning that you dont have to check the block's difficulty and nonce in this test. (should optimize the test generation time and test execution time)

This is a small change that should not be hard to change on other clients.
However I would like to rise the question of developing ethereum tests to an even more general set of tests which could be used by other ethereum implementations.

The fields "genesisBlockHeader" , "engine", "pre" and "network" basically represent genesis.json configuration. If we come to a standart in such configuration we could encode in into blockchain tests as well.

Fathermore if in such configuration you would be able to configure EVM opcode gasPrices - the blockchain tests could be used for other client implementations.

The text was updated successfully, but these errors were encountered:

winsvega · 2018-05-30T13:14:45Z

ethereum/EIPs#1085

Since tests has been regenerated those one were failing on block difficulty check. Update ethereum/tests, waiting for cost fix (block test are still commented).

…9744) (#9814) * Update test reference. Block test are really not working so I disabled a few by commenting directly in source. * Move ethtest commit cursor. * Implements 'NoProof' engine from ethereum/tests#464 . Since tests has been regenerated those one were failing on block difficulty check. Update ethereum/tests, waiting for cost fix (block test are still commented). * Update tests submodule reference to latest (all test passing except an identified case). Fix block reward of constantinople json. * Restore broken test by using old json tests files. * Use CanonNoSeal instead of a custom engine, still have to include some additional tests code. * Gas upper limit check in json_chain test was bad, moving the test to verification, the test is running in `verify_header_param`. Note that test was previously only for ethash, and now for any engine. * Restore old behavior (gas uper limit only for ethash engine), at the cost of an additional trait method. * Proper rpc test fix. * Update tests submodule, add SStore bug tests. * Fix json issue tabulation. Update tests submodule to latest master (lot of new sstore tests passing) * Switch ethereum/tests to tag 6.0.0-beta.1 (no tests changes from latest synch). * Display hex with separator, use indirection instead of clone for copy types.

* version: mark 2.2.0 beta * ci: remove failing tests for android, windows, and macos (#9788) * ci: remove failing tests for android, windows, and macos * ci: restore android build jobs * Move state root verification before gas used (#9841) * Classic.json Bootnode Update (#9828) * fix: Update bootnodes list to only responsive nodes * feat: Add more bootnodes to classic.json list * feat: Add retested bootnodes * Implement NoProof for json tests and update tests reference (replaces #9744) (#9814) * Update test reference. Block test are really not working so I disabled a few by commenting directly in source. * Move ethtest commit cursor. * Implements 'NoProof' engine from ethereum/tests#464 . Since tests has been regenerated those one were failing on block difficulty check. Update ethereum/tests, waiting for cost fix (block test are still commented). * Update tests submodule reference to latest (all test passing except an identified case). Fix block reward of constantinople json. * Restore broken test by using old json tests files. * Use CanonNoSeal instead of a custom engine, still have to include some additional tests code. * Gas upper limit check in json_chain test was bad, moving the test to verification, the test is running in `verify_header_param`. Note that test was previously only for ethash, and now for any engine. * Restore old behavior (gas uper limit only for ethash engine), at the cost of an additional trait method. * Proper rpc test fix. * Update tests submodule, add SStore bug tests. * Fix json issue tabulation. Update tests submodule to latest master (lot of new sstore tests passing) * Switch ethereum/tests to tag 6.0.0-beta.1 (no tests changes from latest synch). * Display hex with separator, use indirection instead of clone for copy types.

* version: mark 2.2.0 beta * ci: remove failing tests for android, windows, and macos (openethereum#9788) * ci: remove failing tests for android, windows, and macos * ci: restore android build jobs * Move state root verification before gas used (openethereum#9841) * Classic.json Bootnode Update (openethereum#9828) * fix: Update bootnodes list to only responsive nodes * feat: Add more bootnodes to classic.json list * feat: Add retested bootnodes * Implement NoProof for json tests and update tests reference (replaces openethereum#9744) (openethereum#9814) * Update test reference. Block test are really not working so I disabled a few by commenting directly in source. * Move ethtest commit cursor. * Implements 'NoProof' engine from ethereum/tests#464 . Since tests has been regenerated those one were failing on block difficulty check. Update ethereum/tests, waiting for cost fix (block test are still commented). * Update tests submodule reference to latest (all test passing except an identified case). Fix block reward of constantinople json. * Restore broken test by using old json tests files. * Use CanonNoSeal instead of a custom engine, still have to include some additional tests code. * Gas upper limit check in json_chain test was bad, moving the test to verification, the test is running in `verify_header_param`. Note that test was previously only for ethash, and now for any engine. * Restore old behavior (gas uper limit only for ethash engine), at the cost of an additional trait method. * Proper rpc test fix. * Update tests submodule, add SStore bug tests. * Fix json issue tabulation. Update tests submodule to latest master (lot of new sstore tests passing) * Switch ethereum/tests to tag 6.0.0-beta.1 (no tests changes from latest synch). * Display hex with separator, use indirection instead of clone for copy types.

winsvega · 2019-11-08T22:58:37Z

of this changes implemented is "NoProof" seal engine

cheme mentioned this issue Oct 12, 2018

Implement NoProof for json tests and update tests reference openethereum/parity-ethereum#9744

Closed

holgerd77 mentioned this issue Oct 20, 2018

Empty mixHash values resulting in failing blockchain tests #530

Closed

cheme mentioned this issue Oct 26, 2018

Implement NoProof for json tests and update tests reference (replaces #9744) openethereum/parity-ethereum#9814

Merged

lithp added the breaking-change Items with this tag should be called out in release notes label Dec 18, 2018

winsvega closed this as completed Nov 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Blockchain Tests Format Changes #464

Blockchain Tests Format Changes #464

winsvega commented May 30, 2018 •

edited

winsvega commented May 30, 2018

winsvega commented Nov 8, 2019

Blockchain Tests Format Changes #464

Blockchain Tests Format Changes #464

Comments

winsvega commented May 30, 2018 • edited

winsvega commented May 30, 2018

winsvega commented Nov 8, 2019

winsvega commented May 30, 2018 •

edited