Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use better state assertions in spec tests #2932

Merged
merged 1 commit into from
Aug 10, 2021

Conversation

dapplion
Copy link
Contributor

Motivation

Current spec test assertions show

expected false to be true

This is very hard to work with

Description

Use json data deep equality for much nicer diffs, for example:

1) epoch registry updates mainnet
       ejection_past_churn_limit:

      AssertionError: expected { Object (genesisTime, genesisValidatorsRoot, ...) } to deeply equal { Object (genesisTime, genesisValidatorsRoot, ...) }
      + expected - actual

           {
             "activationEligibilityEpoch": "0"
             "activationEpoch": "0"
             "effectiveBalance": "16000000000"
      -      "exitEpoch": "6"
      +      "exitEpoch": "5"
             "pubkey": "0xac9b60d5afcbd5663a8a44b7c5a02f19e9a77ab0a35bd65809bb5c67ec582c897feb04decc694b13e08587f3ff9b5b60"
             "slashed": false
      -      "withdrawableEpoch": "262"
      +      "withdrawableEpoch": "261"
             "withdrawalCredentials": "0x00842cef589da28d85b18f7655b70bd2e6d49dd682fb50f75c44293e9de9897a"
           }

@codeclimate
Copy link

codeclimate bot commented Aug 10, 2021

Code Climate has analyzed commit 68153f3 and detected 0 issues on this pull request.

View more on Code Climate.

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: a4e489f Previous: c9e92a7 Ratio
Process block - 250000 vs - 7PWei - with 16 validator exits 29.172 ms/op 116.93 ms/op 0.25
Full benchmark results
Benchmark suite Current: a4e489f Previous: c9e92a7 Ratio
getCommitteeAssignments - req 1000 vs - 250000 vc 9.7028 ms/op 8.9687 ms/op 1.08
epoch altair - 250000 vs - 7PWei - processInactivityUpdates 2.1995 s/op 2.8662 s/op 0.77
epoch altair - 250000 vs - 7PWei - processRewardsAndPenalties 887.00 ms/op 959.59 ms/op 0.92
epoch altair - 250000 vs - 7PWei - processParticipationFlagUpdates 388.48 ms/op 409.46 ms/op 0.95
Process block - 250000 vs - 7PWei - with 0 validator exit 476.52 us/op 567.50 us/op 0.84
Process block - 250000 vs - 7PWei - with 1 validator exit 27.788 ms/op 27.617 ms/op 1.01
Process block - 250000 vs - 7PWei - with 16 validator exits 29.172 ms/op 116.93 ms/op 0.25
epoch phase0 - 250000 vs - 7PWei - beforeProcessEpoch 732.55 ms/op 912.62 ms/op 0.80
epoch phase0 - 250000 vs - 7PWei - processRewardsAndPenalties 549.93 ms/op 466.89 ms/op 1.18
epoch phase0 - 250000 vs - 7PWei - processEffectiveBalanceUpdates 120.42 ms/op 58.193 ms/op 2.07
getAttestationDeltas - 250000 vs - 7PWei 116.85 ms/op 122.22 ms/op 0.96
processSlots - 250000 vs - 7PWei - 32 empty slots 5.4267 s/op 5.6142 s/op 0.97
shuffle list - 16384 els 3.1176 ms/op 1.9289 ms/op 1.62
shuffle list - 250000 els 44.750 ms/op 47.783 ms/op 0.94
getPubkeys - persistent - req 1000 vs - 250000 vc 23.828 us/op 20.476 us/op 1.16
BLS verify - blst-native 2.3270 ms/op 2.3403 ms/op 0.99
BLS verifyMultipleSignatures 3 - blst-native 4.7737 ms/op 4.9642 ms/op 0.96
BLS verifyMultipleSignatures 8 - blst-native 10.287 ms/op 10.488 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 37.298 ms/op 38.836 ms/op 0.96
BLS aggregatePubkeys 32 - blst-native 87.548 us/op 52.405 us/op 1.67
BLS aggregatePubkeys 128 - blst-native 194.35 us/op 197.65 us/op 0.98
getAttestationsForBlock 89.318 ms/op 95.602 ms/op 0.93
validate gossip signedAggregateAndProof - struct 8.1410 ms/op 5.7709 ms/op 1.41
validate gossip signedAggregateAndProof - treeBacked 5.6813 ms/op 5.7765 ms/op 0.98
validate gossip attestation - struct 2.6399 ms/op 2.6103 ms/op 1.01
validate gossip attestation - treeBacked 2.7031 ms/op 2.7389 ms/op 0.99

by benchmarkbot/action

@dapplion dapplion merged commit 05e8265 into master Aug 10, 2021
@dapplion dapplion deleted the dapplion/spec-test-assertions branch August 10, 2021 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants