diff --git a/specs/test_formats/epoch_processing/README.md b/specs/test_formats/epoch_processing/README.md new file mode 100644 index 0000000000..117be89d13 --- /dev/null +++ b/specs/test_formats/epoch_processing/README.md @@ -0,0 +1,34 @@ +# Epoch processing tests + +The different epoch sub-transitions are tested individually with test handlers. +The format is similar to block-processing state-transition tests. +There is no "change" factor however, the transitions are pure functions with just the pre-state as input. +Hence, the format is shared between each test-handler. (See test condition documentation on how to run the tests.) + +## Test case format + +```yaml +description: string -- description of test case, purely for debugging purposes +bls_required: bool -- optional, true if the test validity is strictly dependent on BLS being ON. False otherwise. +bls_ignored: bool -- optional, true if the test validity is strictly dependent on BLS being OFF. False otherwise. +pre: BeaconState -- state before running the sub-transition +post: BeaconState -- state after applying the epoch sub-transition. +``` + +Note: if both `bls_required` and `bls_ignored` are false (or simply not included), + then the test consumer can freely choose to run with BLS ON or OFF. +One may choose for OFF for performance reasons during repeated testing. Otherwise it is recommended to run with BLS ON. + +## Condition + +A handler of the `epoch_processing` test-runner should process these cases, + calling the corresponding processing implementation. + +Sub-transitions: + +| *`sub-transition-name`* | *`processing call`* | +|-------------------------|-----------------------------------| +| `crosslinks` | `process_crosslinks(state)` | +| `registry_updates` | `process_registry_updates(state)` | + +The resulting state should match the expected `post` state. diff --git a/specs/test_formats/operations/README.md b/specs/test_formats/operations/README.md index 842dc3615f..039e01a2f9 100644 --- a/specs/test_formats/operations/README.md +++ b/specs/test_formats/operations/README.md @@ -2,9 +2,39 @@ The different kinds of operations ("transactions") are tested individually with test handlers. -The tested operation kinds are: -- [`deposits`](./deposits.md) -- More tests are work-in-progress. +## Test case format +```yaml +description: string -- description of test case, purely for debugging purposes +bls_required: bool -- optional, true if the test validity is strictly dependent on BLS being ON. False otherwise. +bls_ignored: bool -- optional, true if the test validity is strictly dependent on BLS being OFF. False otherwise. +pre: BeaconState -- state before applying the operation +: -- the YAML encoded operation, e.g. a "ProposerSlashing", or "Deposit". +post: BeaconState -- state after applying the operation. No value if operation processing is aborted. +``` +Note: if both `bls_required` and `bls_ignored` are false (or simply not included), + then the test consumer can freely choose to run with BLS ON or OFF. +One may choose for OFF for performance reasons during repeated testing. Otherwise it is recommended to run with BLS ON. +## Condition + +A handler of the `operations` test-runner should process these cases, + calling the corresponding processing implementation. + +Operations: + +| *`operation-name`* | *`operation-object`* | *`input name`* | *`processing call`* | +|-------------------------|----------------------|----------------------|--------------------------------------------------------| +| `attestation` | `Attestation` | `attestation` | `process_attestation(state, attestation)` | +| `attester_slashing` | `AttesterSlashing` | `attester_slashing` | `process_attester_slashing(state, attester_slashing)` | +| `block_header` | `Block` | `block` | `process_block_header(state, block)` | +| `deposit` | `Deposit` | `deposit` | `process_deposit(state, deposit)` | +| `proposer_slashing` | `ProposerSlashing` | `proposer_slashing` | `process_proposer_slashing(state, proposer_slashing)` | +| `transfer` | `Transfer` | `transfer` | `process_transfer(state, transfer)` | +| `voluntary_exit` | `VoluntaryExit` | `voluntary_exit` | `process_voluntary_exit(state, voluntary_exit)` | + +Note that `block_header` is not strictly an operation (and is a full `Block`), but processed in the same manner, and hence included here. + +The resulting state should match the expected `post` state, or if the `post` state is left blank, + the handler should reject the input operation as invalid. diff --git a/specs/test_formats/operations/deposits.md b/specs/test_formats/operations/deposits.md deleted file mode 100644 index 8f44ebb228..0000000000 --- a/specs/test_formats/operations/deposits.md +++ /dev/null @@ -1,18 +0,0 @@ -# Test format: Deposit operations - -A deposit is a form of an operation (or "transaction"), modifying the state. - -## Test case format - -```yaml -description: string -- description of test case, purely for debugging purposes -pre: BeaconState -- state before applying the deposit -deposit: Deposit -- the deposit -post: BeaconState -- state after applying the deposit. No value if deposit processing is aborted. -``` - -## Condition - -A `deposits` handler of the `operations` should process these cases, - calling the implementation of the `process_deposit(state, deposit)` functionality described in the spec. -The resulting state should match the expected `post` state, or if the `post` state is left blank, the handler should reject the inputs as invalid. diff --git a/specs/test_formats/sanity/README.md b/specs/test_formats/sanity/README.md new file mode 100644 index 0000000000..20b36208a4 --- /dev/null +++ b/specs/test_formats/sanity/README.md @@ -0,0 +1,7 @@ +# Sanity tests + +The aim of the sanity tests is to set a base-line on what really needs to pass, i.e. the essentials. + +There are two handlers, documented individually: +- [`slots`](./slots.md): transitions of one or more slots (and epoch transitions within) +- [`blocks`](./blocks.md): transitions triggered by one or more blocks diff --git a/specs/test_formats/sanity/blocks.md b/specs/test_formats/sanity/blocks.md new file mode 100644 index 0000000000..6968137404 --- /dev/null +++ b/specs/test_formats/sanity/blocks.md @@ -0,0 +1,19 @@ +# Sanity blocks testing + +Sanity tests to cover a series of one or more blocks being processed, aiming to cover common changes. + +## Test case format + +```yaml +description: string -- description of test case, purely for debugging purposes +bls_required: bool -- optional, true if the test validity is strictly dependent on BLS being ON. False otherwise. +bls_ignored: bool -- optional, true if the test validity is strictly dependent on BLS being OFF. False otherwise. +pre: BeaconState -- state before running through the transitions triggered by the blocks. +blocks: [BeaconBlock] -- blocks to process, in given order, following the main transition function (i.e. process slot and epoch transitions in between blocks as normal) +post: BeaconState -- state after applying all the transitions triggered by the blocks. +``` + +## Condition + +The resulting state should match the expected `post` state, or if the `post` state is left blank, + the handler should reject the series of blocks as invalid. diff --git a/specs/test_formats/sanity/slots.md b/specs/test_formats/sanity/slots.md new file mode 100644 index 0000000000..e8961608d0 --- /dev/null +++ b/specs/test_formats/sanity/slots.md @@ -0,0 +1,24 @@ +# Sanity slots testing + +Sanity tests to cover a series of one or more empty-slot transitions being processed, aiming to cover common changes. + +## Test case format + +```yaml +description: string -- description of test case, purely for debugging purposes +bls_required: bool -- optional, true if the test validity is strictly dependent on BLS being ON. False otherwise. +bls_ignored: bool -- optional, true if the test validity is strictly dependent on BLS being OFF. False otherwise. +pre: BeaconState -- state before running through the transitions. +slots: N -- amount of slots to process, N being a positive numer. +post: BeaconState -- state after applying all the transitions. +``` + +The transition with pure time, no blocks, is known as `state_transition_to(state, slot)` in the spec. +This runs state-caching (pure slot transition) and epoch processing (every E slots). + +To process the data, call `state_transition_to(pre, pre.slot + N)`. And see if `pre` mutated into the equivalent of `post`. + + +## Condition + +The resulting state should match the expected `post` state. diff --git a/test_generators/epoch_processing/README.md b/test_generators/epoch_processing/README.md new file mode 100644 index 0000000000..9b57875e2a --- /dev/null +++ b/test_generators/epoch_processing/README.md @@ -0,0 +1,11 @@ +# Epoch processing + +Epoch processing covers the sub-transitions during an epoch change. + +An epoch-processing test-runner can consume these sub-transition test-suites, + and handle different kinds of epoch sub-transitions by processing the cases using the specified test handler. + +Information on the format of the tests can be found in the [epoch-processing test formats documentation](../../specs/test_formats/epoch_processing/README.md). + + + diff --git a/test_generators/epoch_processing/main.py b/test_generators/epoch_processing/main.py new file mode 100644 index 0000000000..8f067e4a35 --- /dev/null +++ b/test_generators/epoch_processing/main.py @@ -0,0 +1,38 @@ +from typing import Callable, Iterable + +from eth2spec.phase0 import spec +from eth2spec.test.epoch_processing import ( + test_process_crosslinks, + test_process_registry_updates +) +from gen_base import gen_runner, gen_suite, gen_typing +from gen_from_tests.gen import generate_from_tests +from preset_loader import loader + + +def create_suite(transition_name: str, config_name: str, get_cases: Callable[[], Iterable[gen_typing.TestCase]]) \ + -> Callable[[str], gen_typing.TestSuiteOutput]: + def suite_definition(configs_path: str) -> gen_typing.TestSuiteOutput: + presets = loader.load_presets(configs_path, config_name) + spec.apply_constants_preset(presets) + + return ("%s_%s" % (transition_name, config_name), transition_name, gen_suite.render_suite( + title="%s epoch processing" % transition_name, + summary="Test suite for %s type epoch processing" % transition_name, + forks_timeline="testing", + forks=["phase0"], + config=config_name, + runner="epoch_processing", + handler=transition_name, + test_cases=get_cases())) + + return suite_definition + + +if __name__ == "__main__": + gen_runner.run_generator("epoch_processing", [ + create_suite('crosslinks', 'minimal', lambda: generate_from_tests(test_process_crosslinks)), + create_suite('crosslinks', 'mainnet', lambda: generate_from_tests(test_process_crosslinks)), + create_suite('registry_updates', 'minimal', lambda: generate_from_tests(test_process_registry_updates)), + create_suite('registry_updates', 'mainnet', lambda: generate_from_tests(test_process_registry_updates)), + ]) diff --git a/test_generators/epoch_processing/requirements.txt b/test_generators/epoch_processing/requirements.txt new file mode 100644 index 0000000000..595cee69cd --- /dev/null +++ b/test_generators/epoch_processing/requirements.txt @@ -0,0 +1,4 @@ +eth-utils==1.6.0 +../../test_libs/gen_helpers +../../test_libs/config_helpers +../../test_libs/pyspec \ No newline at end of file diff --git a/test_generators/operations/README.md b/test_generators/operations/README.md index e0b9d0e187..5cb3afc989 100644 --- a/test_generators/operations/README.md +++ b/test_generators/operations/README.md @@ -3,7 +3,6 @@ Operations (or "transactions" in previous spec iterations), are atomic changes to the state, introduced by embedding in blocks. -This generator provides a series of test suites, divided into handler, for each operation type. An operation test-runner can consume these operation test-suites, and handle different kinds of operations by processing the cases using the specified test handler. diff --git a/test_generators/operations/main.py b/test_generators/operations/main.py index 41a6ff806e..96c639d12d 100644 --- a/test_generators/operations/main.py +++ b/test_generators/operations/main.py @@ -1,3 +1,5 @@ +from typing import Callable, Iterable + from eth2spec.test.block_processing import ( test_process_attestation, test_process_attester_slashing, @@ -8,9 +10,29 @@ test_process_voluntary_exit ) -from gen_base import gen_runner +from gen_base import gen_runner, gen_suite, gen_typing +from gen_from_tests.gen import generate_from_tests +from preset_loader import loader +from eth2spec.phase0 import spec + + +def create_suite(operation_name: str, config_name: str, get_cases: Callable[[], Iterable[gen_typing.TestCase]]) \ + -> Callable[[str], gen_typing.TestSuiteOutput]: + def suite_definition(configs_path: str) -> gen_typing.TestSuiteOutput: + presets = loader.load_presets(configs_path, config_name) + spec.apply_constants_preset(presets) + + return ("%s_%s" % (operation_name, config_name), operation_name, gen_suite.render_suite( + title="%s operation" % operation_name, + summary="Test suite for %s type operation processing" % operation_name, + forks_timeline="testing", + forks=["phase0"], + config=config_name, + runner="operations", + handler=operation_name, + test_cases=get_cases())) + return suite_definition -from suite_creator import generate_from_tests, create_suite if __name__ == "__main__": gen_runner.run_generator("operations", [ diff --git a/test_generators/operations/suite_creator.py b/test_generators/operations/suite_creator.py deleted file mode 100644 index caff0c7db9..0000000000 --- a/test_generators/operations/suite_creator.py +++ /dev/null @@ -1,39 +0,0 @@ -from typing import Callable, Iterable -from inspect import getmembers, isfunction -from gen_base import gen_suite, gen_typing -from preset_loader import loader -from eth2spec.phase0 import spec - - -def generate_from_tests(pkg): - fn_names = [ - name for (name, _) in getmembers(pkg, isfunction) - if name.startswith('test_') - ] - out = [] - print("generating test vectors from tests package: %s" % pkg.__name__) - for name in fn_names: - tfn = getattr(pkg, name) - try: - out.append(tfn(generator_mode=True, bls_active=True)) - except AssertionError: - print("ERROR: failed to generate vector from test: %s (pkg: %s)" % (name, pkg.__name__)) - return out - - -def create_suite(operation_name: str, config_name: str, get_cases: Callable[[], Iterable[gen_typing.TestCase]])\ - -> Callable[[str], gen_typing.TestSuiteOutput]: - def suite_definition(configs_path: str) -> gen_typing.TestSuiteOutput: - presets = loader.load_presets(configs_path, config_name) - spec.apply_constants_preset(presets) - - return ("%s_%s" % (operation_name, config_name), operation_name, gen_suite.render_suite( - title="%s operation" % operation_name, - summary="Test suite for %s type operation processing" % operation_name, - forks_timeline="testing", - forks=["phase0"], - config=config_name, - runner="operations", - handler=operation_name, - test_cases=get_cases())) - return suite_definition diff --git a/test_generators/sanity/README.md b/test_generators/sanity/README.md new file mode 100644 index 0000000000..6d2e2f30dd --- /dev/null +++ b/test_generators/sanity/README.md @@ -0,0 +1,8 @@ +# Sanity tests + +Sanity tests cover regular state-transitions in a common block-list format, to ensure the basics work. + +Information on the format of the tests can be found in the [sanity test formats documentation](../../specs/test_formats/sanity/README.md). + + + diff --git a/test_generators/sanity/main.py b/test_generators/sanity/main.py new file mode 100644 index 0000000000..bba6ed03df --- /dev/null +++ b/test_generators/sanity/main.py @@ -0,0 +1,35 @@ +from typing import Callable, Iterable + +from eth2spec.test.sanity import test_blocks, test_slots + +from gen_base import gen_runner, gen_suite, gen_typing +from gen_from_tests.gen import generate_from_tests +from preset_loader import loader +from eth2spec.phase0 import spec + + +def create_suite(handler_name: str, config_name: str, get_cases: Callable[[], Iterable[gen_typing.TestCase]]) \ + -> Callable[[str], gen_typing.TestSuiteOutput]: + def suite_definition(configs_path: str) -> gen_typing.TestSuiteOutput: + presets = loader.load_presets(configs_path, config_name) + spec.apply_constants_preset(presets) + + return ("%sanity_s_%s" % (handler_name, config_name), handler_name, gen_suite.render_suite( + title="sanity testing", + summary="Sanity test suite, %s type, generated from pytests" % handler_name, + forks_timeline="testing", + forks=["phase0"], + config=config_name, + runner="sanity", + handler=handler_name, + test_cases=get_cases())) + return suite_definition + + +if __name__ == "__main__": + gen_runner.run_generator("sanity", [ + create_suite('blocks', 'minimal', lambda: generate_from_tests(test_blocks)), + create_suite('blocks', 'mainnet', lambda: generate_from_tests(test_blocks)), + create_suite('slots', 'minimal', lambda: generate_from_tests(test_slots)), + create_suite('slots', 'mainnet', lambda: generate_from_tests(test_slots)), + ]) diff --git a/test_generators/sanity/requirements.txt b/test_generators/sanity/requirements.txt new file mode 100644 index 0000000000..595cee69cd --- /dev/null +++ b/test_generators/sanity/requirements.txt @@ -0,0 +1,4 @@ +eth-utils==1.6.0 +../../test_libs/gen_helpers +../../test_libs/config_helpers +../../test_libs/pyspec \ No newline at end of file diff --git a/test_libs/gen_helpers/gen_from_tests/__init__.py b/test_libs/gen_helpers/gen_from_tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/test_libs/gen_helpers/gen_from_tests/gen.py b/test_libs/gen_helpers/gen_from_tests/gen.py new file mode 100644 index 0000000000..e7d8011310 --- /dev/null +++ b/test_libs/gen_helpers/gen_from_tests/gen.py @@ -0,0 +1,25 @@ +from inspect import getmembers, isfunction + +def generate_from_tests(src, bls_active=True): + """ + Generate a list of test cases by running tests from the given src in generator-mode. + :param src: to retrieve tests from (discovered using inspect.getmembers) + :param bls_active: optional, to override BLS switch preference. Defaults to True. + :return: the list of test cases. + """ + fn_names = [ + name for (name, _) in getmembers(src, isfunction) + if name.startswith('test_') + ] + out = [] + print("generating test vectors from tests source: %s" % src.__name__) + for name in fn_names: + tfn = getattr(src, name) + try: + test_case = tfn(generator_mode=True, bls_active=bls_active) + # If no test case data is returned, the test is ignored. + if test_case is not None: + out.append(test_case) + except AssertionError: + print("ERROR: failed to generate vector from test: %s (src: %s)" % (name, src.__name__)) + return out diff --git a/test_libs/gen_helpers/setup.py b/test_libs/gen_helpers/setup.py index 29cf04fd19..ee2c815c76 100644 --- a/test_libs/gen_helpers/setup.py +++ b/test_libs/gen_helpers/setup.py @@ -2,7 +2,7 @@ setup( name='gen_helpers', - packages=['gen_base'], + packages=['gen_base', 'gen_from_tests'], install_requires=[ "ruamel.yaml==0.15.96", "eth-utils==1.6.0" diff --git a/test_libs/pyspec/eth2spec/test/block_processing/test_process_attestation.py b/test_libs/pyspec/eth2spec/test/block_processing/test_process_attestation.py index 0dae852f0b..af6b39ef6e 100644 --- a/test_libs/pyspec/eth2spec/test/block_processing/test_process_attestation.py +++ b/test_libs/pyspec/eth2spec/test/block_processing/test_process_attestation.py @@ -75,7 +75,7 @@ def test_success_previous_epoch(state): @always_bls @spec_state_test def test_invalid_attestation_signature(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY yield from run_attestation_processing(state, attestation, False) @@ -105,7 +105,7 @@ def test_old_source_epoch(state): state.finalized_epoch = 2 state.previous_justified_epoch = 3 state.current_justified_epoch = 4 - attestation = get_valid_attestation(state, slot=(spec.SLOTS_PER_EPOCH * 3) + 1, signed=False) + attestation = get_valid_attestation(state, slot=(spec.SLOTS_PER_EPOCH * 3) + 1) # test logic sanity check: make sure the attestation is pointing to oldest known source epoch assert attestation.data.source_epoch == state.previous_justified_epoch @@ -120,7 +120,7 @@ def test_old_source_epoch(state): @spec_state_test def test_wrong_shard(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.data.shard += 1 @@ -132,7 +132,7 @@ def test_wrong_shard(state): @spec_state_test def test_new_source_epoch(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.data.source_epoch += 1 @@ -144,7 +144,7 @@ def test_new_source_epoch(state): @spec_state_test def test_source_root_is_target_root(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.data.source_root = attestation.data.target_root @@ -165,7 +165,7 @@ def test_invalid_current_source_root(state): state.current_justified_epoch = 4 state.current_justified_root = b'\xff' * 32 - attestation = get_valid_attestation(state, slot=(spec.SLOTS_PER_EPOCH * 3) + 1, signed=False) + attestation = get_valid_attestation(state, slot=(spec.SLOTS_PER_EPOCH * 3) + 1) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY # Test logic sanity checks: @@ -182,7 +182,7 @@ def test_invalid_current_source_root(state): @spec_state_test def test_bad_source_root(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.data.source_root = b'\x42' * 32 @@ -194,7 +194,7 @@ def test_bad_source_root(state): @spec_state_test def test_non_zero_crosslink_data_root(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.data.crosslink_data_root = b'\x42' * 32 @@ -221,7 +221,7 @@ def test_bad_previous_crosslink(state): @spec_state_test def test_inconsistent_bitfields(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.custody_bitfield = deepcopy(attestation.aggregation_bitfield) + b'\x00' @@ -233,7 +233,7 @@ def test_inconsistent_bitfields(state): @spec_state_test def test_non_empty_custody_bitfield(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.custody_bitfield = deepcopy(attestation.aggregation_bitfield) @@ -245,7 +245,7 @@ def test_non_empty_custody_bitfield(state): @spec_state_test def test_empty_aggregation_bitfield(state): - attestation = get_valid_attestation(state, signed=False) + attestation = get_valid_attestation(state) state.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation.aggregation_bitfield = b'\x00' * len(attestation.aggregation_bitfield) diff --git a/test_libs/pyspec/eth2spec/test/block_processing/test_process_block_header.py b/test_libs/pyspec/eth2spec/test/block_processing/test_process_block_header.py index 28e215a3a0..454f557c5c 100644 --- a/test_libs/pyspec/eth2spec/test/block_processing/test_process_block_header.py +++ b/test_libs/pyspec/eth2spec/test/block_processing/test_process_block_header.py @@ -50,13 +50,13 @@ def test_success_block_header(state): @always_bls @spec_state_test def test_invalid_sig_block_header(state): - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) yield from run_block_header_processing(state, block, valid=False) @spec_state_test def test_invalid_slot_block_header(state): - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot = state.slot + 2 # invalid slot sign_block(state, block) @@ -65,7 +65,7 @@ def test_invalid_slot_block_header(state): @spec_state_test def test_invalid_previous_block_root(state): - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.previous_block_root = b'\12' * 32 # invalid prev root sign_block(state, block) diff --git a/test_libs/pyspec/eth2spec/test/block_processing/test_process_deposit.py b/test_libs/pyspec/eth2spec/test/block_processing/test_process_deposit.py index b520c809f8..336af3bf73 100644 --- a/test_libs/pyspec/eth2spec/test/block_processing/test_process_deposit.py +++ b/test_libs/pyspec/eth2spec/test/block_processing/test_process_deposit.py @@ -69,7 +69,7 @@ def test_invalid_sig_new_deposit(state): # fresh deposit = next validator index = validator appended to registry validator_index = len(state.validator_registry) amount = spec.MAX_EFFECTIVE_BALANCE - deposit = prepare_state_and_deposit(state, validator_index, amount, signed=False) + deposit = prepare_state_and_deposit(state, validator_index, amount) yield from run_deposit_processing(state, deposit, validator_index, valid=True, effective=False) @@ -87,7 +87,7 @@ def test_success_top_up(state): def test_invalid_sig_top_up(state): validator_index = 0 amount = spec.MAX_EFFECTIVE_BALANCE // 4 - deposit = prepare_state_and_deposit(state, validator_index, amount, signed=False) + deposit = prepare_state_and_deposit(state, validator_index, amount) # invalid signatures, in top-ups, are allowed! yield from run_deposit_processing(state, deposit, validator_index, valid=True, effective=True) @@ -97,7 +97,7 @@ def test_invalid_sig_top_up(state): def test_wrong_index(state): validator_index = len(state.validator_registry) amount = spec.MAX_EFFECTIVE_BALANCE - deposit = prepare_state_and_deposit(state, validator_index, amount, signed=False) + deposit = prepare_state_and_deposit(state, validator_index, amount) # mess up deposit_index deposit.index = state.deposit_index + 1 @@ -114,7 +114,7 @@ def test_wrong_index(state): def test_bad_merkle_proof(state): validator_index = len(state.validator_registry) amount = spec.MAX_EFFECTIVE_BALANCE - deposit = prepare_state_and_deposit(state, validator_index, amount, signed=False) + deposit = prepare_state_and_deposit(state, validator_index, amount) # mess up merkle branch deposit.proof[-1] = spec.ZERO_HASH diff --git a/test_libs/pyspec/eth2spec/test/block_processing/test_process_transfer.py b/test_libs/pyspec/eth2spec/test/block_processing/test_process_transfer.py index e5f52e209f..83af755743 100644 --- a/test_libs/pyspec/eth2spec/test/block_processing/test_process_transfer.py +++ b/test_libs/pyspec/eth2spec/test/block_processing/test_process_transfer.py @@ -86,7 +86,7 @@ def test_success_active_above_max_effective_fee(state): @always_bls @spec_state_test def test_invalid_signature(state): - transfer = get_valid_transfer(state, signed=False) + transfer = get_valid_transfer(state) # un-activate so validator can transfer state.validator_registry[transfer.sender].activation_eligibility_epoch = spec.FAR_FUTURE_EPOCH diff --git a/test_libs/pyspec/eth2spec/test/block_processing/test_process_voluntary_exit.py b/test_libs/pyspec/eth2spec/test/block_processing/test_process_voluntary_exit.py index fe33fb6318..53fb4e3f7c 100644 --- a/test_libs/pyspec/eth2spec/test/block_processing/test_process_voluntary_exit.py +++ b/test_libs/pyspec/eth2spec/test/block_processing/test_process_voluntary_exit.py @@ -62,7 +62,7 @@ def test_invalid_signature(state): validator_index = get_active_validator_indices(state, current_epoch)[0] privkey = pubkey_to_privkey[state.validator_registry[validator_index].pubkey] - voluntary_exit = build_voluntary_exit(state, current_epoch, validator_index, privkey, signed=False) + voluntary_exit = build_voluntary_exit(state, current_epoch, validator_index, privkey) yield from run_voluntary_exit_processing(state, voluntary_exit, False) diff --git a/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_crosslinks.py b/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_crosslinks.py index 688bb54ac8..cfbcd18834 100644 --- a/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_crosslinks.py +++ b/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_crosslinks.py @@ -21,6 +21,7 @@ fill_aggregate_attestation, get_crosslink_committee, get_valid_attestation, + sign_attestation, ) @@ -33,7 +34,7 @@ def run_process_crosslinks(state, valid=True): """ # transition state to slot before state transition slot = state.slot + (spec.SLOTS_PER_EPOCH - state.slot % spec.SLOTS_PER_EPOCH) - 1 - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot = slot sign_block(state, block) state_transition(state, block) @@ -103,19 +104,24 @@ def test_single_crosslink_update_from_previous_epoch(state): @spec_state_test def test_double_late_crosslink(state): + if spec.get_epoch_committee_count(state, spec.get_current_epoch(state)) < spec.SHARD_COUNT: + print("warning: ignoring test, test-assumptions are incompatible with configuration") + return + next_epoch(state) state.slot += 4 attestation_1 = get_valid_attestation(state, signed=True) fill_aggregate_attestation(state, attestation_1) - # add attestation_1 in the next epoch + # add attestation_1 to next epoch next_epoch(state) add_attestation_to_state(state, attestation_1, state.slot + 1) for slot in range(spec.SLOTS_PER_EPOCH): - attestation_2 = get_valid_attestation(state, signed=True) + attestation_2 = get_valid_attestation(state) if attestation_2.data.shard == attestation_1.data.shard: + sign_attestation(state, attestation_2) break next_slot(state) apply_empty_block(state) diff --git a/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_registry_updates.py b/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_registry_updates.py index 2086f4ef2d..71bf89c702 100644 --- a/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_registry_updates.py +++ b/test_libs/pyspec/eth2spec/test/epoch_processing/test_process_registry_updates.py @@ -3,11 +3,41 @@ from eth2spec.phase0.spec import ( get_current_epoch, is_active_validator, + process_registry_updates ) +from eth2spec.phase0.state_transition import state_transition +from eth2spec.test.helpers.block import build_empty_block_for_next_slot, sign_block from eth2spec.test.helpers.state import next_epoch from eth2spec.test.context import spec_state_test +def run_process_registry_updates(state, valid=True): + """ + Run ``process_crosslinks``, yielding: + - pre-state ('pre') + - post-state ('post'). + If ``valid == False``, run expecting ``AssertionError`` + """ + # transition state to slot before state transition + slot = state.slot + (spec.SLOTS_PER_EPOCH - state.slot % spec.SLOTS_PER_EPOCH) - 1 + block = build_empty_block_for_next_slot(state) + block.slot = slot + sign_block(state, block) + state_transition(state, block) + + # cache state before epoch transition + spec.cache_state(state) + + # process components of epoch transition before registry update + spec.process_justification_and_finalization(state) + spec.process_crosslinks(state) + spec.process_rewards_and_penalties(state) + + yield 'pre', state + process_registry_updates(state) + yield 'post', state + + @spec_state_test def test_activation(state): index = 0 @@ -19,17 +49,10 @@ def test_activation(state): state.validator_registry[index].effective_balance = spec.MAX_EFFECTIVE_BALANCE assert not is_active_validator(state.validator_registry[index], get_current_epoch(state)) - yield 'pre', state - - blocks = [] for _ in range(spec.ACTIVATION_EXIT_DELAY + 1): - block = next_epoch(state) - blocks.append(block) + next_epoch(state) - # provide extra type hinting here, since it is wrapped in a list. - yield 'blocks', blocks, [spec.BeaconBlock] - - yield 'post', state + yield from run_process_registry_updates(state) assert state.validator_registry[index].activation_eligibility_epoch != spec.FAR_FUTURE_EPOCH assert state.validator_registry[index].activation_epoch != spec.FAR_FUTURE_EPOCH @@ -48,17 +71,10 @@ def test_ejection(state): # Mock an ejection state.validator_registry[index].effective_balance = spec.EJECTION_BALANCE - yield 'pre', state - - blocks = [] for _ in range(spec.ACTIVATION_EXIT_DELAY + 1): - block = next_epoch(state) - blocks.append(block) - - # provide extra type hinting here, since it is wrapped in a list. - yield 'blocks', blocks, [spec.BeaconBlock] + next_epoch(state) - yield 'post', state + yield from run_process_registry_updates(state) assert state.validator_registry[index].exit_epoch != spec.FAR_FUTURE_EPOCH assert not is_active_validator( diff --git a/test_libs/pyspec/eth2spec/test/helpers/attestations.py b/test_libs/pyspec/eth2spec/test/helpers/attestations.py index e9b863463c..b541e610f4 100644 --- a/test_libs/pyspec/eth2spec/test/helpers/attestations.py +++ b/test_libs/pyspec/eth2spec/test/helpers/attestations.py @@ -138,7 +138,7 @@ def fill_aggregate_attestation(state, attestation): def add_attestation_to_state(state, attestation, slot): - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot = slot block.body.attestations.append(attestation) state_transition_to(state, block.slot) diff --git a/test_libs/pyspec/eth2spec/test/sanity/__init__.py b/test_libs/pyspec/eth2spec/test/sanity/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/test_libs/pyspec/eth2spec/test/test_sanity.py b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py similarity index 86% rename from test_libs/pyspec/eth2spec/test/test_sanity.py rename to test_libs/pyspec/eth2spec/test/sanity/test_blocks.py index 6d65cc7f4b..c9aadbf2ac 100644 --- a/test_libs/pyspec/eth2spec/test/test_sanity.py +++ b/test_libs/pyspec/eth2spec/test/sanity/test_blocks.py @@ -13,42 +13,20 @@ get_block_root_at_slot, get_current_epoch, get_domain, - advance_slot, - cache_state, ) from eth2spec.phase0.state_transition import ( state_transition, ) -from .helpers.state import ( - get_balance, - get_state_root -) -from .helpers.transfers import get_valid_transfer -from .helpers.block import build_empty_block_for_next_slot, sign_block -from .helpers.keys import ( - privkeys, - pubkeys, -) -from .helpers.attester_slashings import get_valid_attester_slashing -from .helpers.proposer_slashings import get_valid_proposer_slashing -from .helpers.attestations import get_valid_attestation -from .helpers.deposits import prepare_state_and_deposit - -from .context import spec_state_test, never_bls - - -@spec_state_test -def test_slot_transition(state): - pre_slot = state.slot - pre_root = state.hash_tree_root() - yield 'pre', state - - cache_state(state) - advance_slot(state) - yield 'post', state +from eth2spec.test.helpers.state import get_balance +from eth2spec.test.helpers.transfers import get_valid_transfer +from eth2spec.test.helpers.block import build_empty_block_for_next_slot, sign_block +from eth2spec.test.helpers.keys import privkeys, pubkeys +from eth2spec.test.helpers.attester_slashings import get_valid_attester_slashing +from eth2spec.test.helpers.proposer_slashings import get_valid_proposer_slashing +from eth2spec.test.helpers.attestations import get_valid_attestation +from eth2spec.test.helpers.deposits import prepare_state_and_deposit - assert state.slot == pre_slot + 1 - assert get_state_root(state, pre_slot) == pre_root +from eth2spec.test.context import spec_state_test, never_bls @never_bls @@ -75,7 +53,7 @@ def test_skipped_slots(state): pre_slot = state.slot yield 'pre', state - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot += 3 sign_block(state, block) yield 'blocks', [block], [spec.BeaconBlock] @@ -93,7 +71,7 @@ def test_empty_epoch_transition(state): pre_slot = state.slot yield 'pre', state - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot += spec.SLOTS_PER_EPOCH sign_block(state, block) yield 'blocks', [block], [spec.BeaconBlock] @@ -112,7 +90,7 @@ def test_empty_epoch_transition(state): # pre_state = deepcopy(state) # yield 'pre', state # -# block = build_empty_block_for_next_slot(state, signed=False) +# block = build_empty_block_for_next_slot(state) # block.slot += spec.SLOTS_PER_EPOCH * 5 # sign_block(state, block, proposer_index=0) # yield 'blocks', [block], [spec.BeaconBlock] @@ -140,7 +118,7 @@ def test_proposer_slashing(state): # # Add to state via block transition # - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.body.proposer_slashings.append(proposer_slashing) sign_block(state, block) yield 'blocks', [block], [spec.BeaconBlock] @@ -173,7 +151,7 @@ def test_attester_slashing(state): # # Add to state via block transition # - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.body.attester_slashings.append(attester_slashing) sign_block(state, block) yield 'blocks', [block], [spec.BeaconBlock] @@ -209,7 +187,7 @@ def test_deposit_in_block(state): yield 'pre', state - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.body.deposits.append(deposit) sign_block(state, block) @@ -236,7 +214,7 @@ def test_deposit_top_up(state): yield 'pre', state - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.body.deposits.append(deposit) sign_block(state, block) @@ -260,7 +238,7 @@ def test_attestation(state): # Add to state via block transition pre_current_attestations_len = len(state.current_epoch_attestations) - attestation_block = build_empty_block_for_next_slot(state, signed=False) + attestation_block = build_empty_block_for_next_slot(state) attestation_block.slot += spec.MIN_ATTESTATION_INCLUSION_DELAY attestation_block.body.attestations.append(attestation) sign_block(state, attestation_block) @@ -271,7 +249,7 @@ def test_attestation(state): # Epoch transition should move to previous_epoch_attestations pre_current_attestations_root = spec.hash_tree_root(state.current_epoch_attestations) - epoch_block = build_empty_block_for_next_slot(state, signed=False) + epoch_block = build_empty_block_for_next_slot(state) epoch_block.slot += spec.SLOTS_PER_EPOCH sign_block(state, epoch_block) state_transition(state, epoch_block) @@ -309,7 +287,7 @@ def test_voluntary_exit(state): ) # Add to state via block transition - initiate_exit_block = build_empty_block_for_next_slot(state, signed=False) + initiate_exit_block = build_empty_block_for_next_slot(state) initiate_exit_block.body.voluntary_exits.append(voluntary_exit) sign_block(state, initiate_exit_block) state_transition(state, initiate_exit_block) @@ -317,7 +295,7 @@ def test_voluntary_exit(state): assert state.validator_registry[validator_index].exit_epoch < spec.FAR_FUTURE_EPOCH # Process within epoch transition - exit_block = build_empty_block_for_next_slot(state, signed=False) + exit_block = build_empty_block_for_next_slot(state) exit_block.slot += spec.SLOTS_PER_EPOCH sign_block(state, exit_block) state_transition(state, exit_block) @@ -346,7 +324,7 @@ def test_transfer(state): yield 'pre', state # Add to state via block transition - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.body.transfers.append(transfer) sign_block(state, block) @@ -374,7 +352,7 @@ def test_balance_driven_status_transitions(state): yield 'pre', state # trigger epoch transition - block = build_empty_block_for_next_slot(state, signed=False) + block = build_empty_block_for_next_slot(state) block.slot += spec.SLOTS_PER_EPOCH sign_block(state, block) state_transition(state, block) @@ -412,13 +390,13 @@ def test_historical_batch(state): # # blocks = [] # for _ in range(spec.SLOTS_PER_ETH1_VOTING_PERIOD - 1): -# block = build_empty_block_for_next_slot(state, signed=False) +# block = build_empty_block_for_next_slot(state) # state_transition(state, block) # expected_votes += 1 # assert len(state.eth1_data_votes) == expected_votes # blocks.append(block) # -# block = build_empty_block_for_next_slot(state, signed=False) +# block = build_empty_block_for_next_slot(state) # blocks.append(block) # # state_transition(state, block) diff --git a/test_libs/pyspec/eth2spec/test/sanity/test_slots.py b/test_libs/pyspec/eth2spec/test/sanity/test_slots.py new file mode 100644 index 0000000000..2e5f3a5df6 --- /dev/null +++ b/test_libs/pyspec/eth2spec/test/sanity/test_slots.py @@ -0,0 +1,58 @@ +import eth2spec.phase0.spec as spec + +from eth2spec.phase0.state_transition import state_transition_to +from eth2spec.test.helpers.state import get_state_root +from eth2spec.test.context import spec_state_test + + +@spec_state_test +def test_slots_1(state): + pre_slot = state.slot + pre_root = state.hash_tree_root() + yield 'pre', state + + slots = 1 + yield 'slots', slots + state_transition_to(state, state.slot + slots) + + yield 'post', state + assert state.slot == pre_slot + 1 + assert get_state_root(state, pre_slot) == pre_root + + +@spec_state_test +def test_slots_2(state): + yield 'pre', state + slots = 2 + yield 'slots', slots + state_transition_to(state, state.slot + slots) + yield 'post', state + + +@spec_state_test +def test_empty_epoch(state): + yield 'pre', state + slots = spec.SLOTS_PER_EPOCH + yield 'slots', slots + state_transition_to(state, state.slot + slots) + yield 'post', state + + +@spec_state_test +def test_double_empty_epoch(state): + yield 'pre', state + slots = spec.SLOTS_PER_EPOCH * 2 + yield 'slots', slots + state_transition_to(state, state.slot + slots) + yield 'post', state + + +@spec_state_test +def test_over_epoch_boundary(state): + state_transition_to(state, state.slot + (spec.SLOTS_PER_EPOCH // 2)) + yield 'pre', state + slots = spec.SLOTS_PER_EPOCH + yield 'slots', slots + state_transition_to(state, state.slot + slots) + yield 'post', state + diff --git a/test_libs/pyspec/eth2spec/test/utils.py b/test_libs/pyspec/eth2spec/test/utils.py index c1d4241099..b61801c3dd 100644 --- a/test_libs/pyspec/eth2spec/test/utils.py +++ b/test_libs/pyspec/eth2spec/test/utils.py @@ -19,8 +19,10 @@ def entry(*args, **kw): else: # description can be explicit out['description'] = description + has_contents = False # put all generated data into a dict. for data in fn(*args, **kw): + has_contents = True # If there is a type argument, encode it as that type. if len(data) == 3: (key, value, typ) = data @@ -32,11 +34,15 @@ def entry(*args, **kw): out[key] = encode(value, value.__class__) else: out[key] = value - return out + if has_contents: + return out + else: + return None else: # just complete the function, ignore all yielded data, we are not using it for _ in fn(*args, **kw): continue + return None return entry return runner @@ -54,7 +60,7 @@ def entry(*args, **kw): fn_out = fn(*args, **kw) # do not add tags if the function is not returning a dict at all (i.e. not in generator mode) if fn_out is None: - return fn_out + return None return {**tags, **fn_out} return entry return runner