Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: move withdrawal tests to sim tests #6507

Merged
merged 8 commits into from Mar 20, 2024
Merged

Conversation

nazarhussain
Copy link
Contributor

Motivation

Remove redundant tests

Description

  • Move withdrawal tests to sim tests.

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner March 5, 2024 11:17
@nazarhussain nazarhussain self-assigned this Mar 5, 2024
if (!payloadId) throw Error("InvalidPayloadId");

// 2. Get the payload
const payloadWithValue = await executionEngine.getPayload(ForkName.capella, payloadId);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's testing the engine API from the execution engine. Which should not be in our test suite.

The behavior passively is tested in sim tests anyway.

Copy link

codecov bot commented Mar 5, 2024

Codecov Report

Merging #6507 (c69d5d5) into unstable (40b8924) will decrease coverage by 0.27%.
Report is 35 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #6507      +/-   ##
============================================
- Coverage     61.72%   61.46%   -0.27%     
============================================
  Files           555      556       +1     
  Lines         58204    58850     +646     
  Branches       1844     1850       +6     
============================================
+ Hits          35927    36171     +244     
- Misses        22238    22638     +400     
- Partials         39       41       +2     

Copy link
Contributor

github-actions bot commented Mar 5, 2024

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: 4fce2eb Previous: f9676f1 Ratio
altair processEth1Data - 250000 vs - 7PWei normalcase 635.19 us/op 4.3122 ms/op 0.15
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.420 us/op 34.859 us/op 0.30
Full benchmark results
Benchmark suite Current: 4fce2eb Previous: f9676f1 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0045 ms/op 1.4899 ms/op 0.67
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 79.318 us/op 140.63 us/op 0.56
BLS verify - blst-native 1.2870 ms/op 1.7309 ms/op 0.74
BLS verifyMultipleSignatures 3 - blst-native 2.6992 ms/op 3.6474 ms/op 0.74
BLS verifyMultipleSignatures 8 - blst-native 5.9288 ms/op 8.0317 ms/op 0.74
BLS verifyMultipleSignatures 32 - blst-native 21.686 ms/op 29.456 ms/op 0.74
BLS verifyMultipleSignatures 64 - blst-native 42.687 ms/op 59.252 ms/op 0.72
BLS verifyMultipleSignatures 128 - blst-native 85.314 ms/op 115.16 ms/op 0.74
BLS deserializing 10000 signatures 910.37 ms/op 1.2015 s/op 0.76
BLS deserializing 100000 signatures 9.1712 s/op 11.709 s/op 0.78
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.2778 ms/op 1.7681 ms/op 0.72
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6058 ms/op 2.0264 ms/op 0.79
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.9360 ms/op 3.0875 ms/op 0.95
BLS verifyMultipleSignatures - same message - 64 - blst-native 4.3019 ms/op 4.5769 ms/op 0.94
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.6649 ms/op 7.5243 ms/op 0.75
BLS aggregatePubkeys 32 - blst-native 26.989 us/op 34.165 us/op 0.79
BLS aggregatePubkeys 128 - blst-native 99.019 us/op 131.94 us/op 0.75
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.124 ms/op 76.899 ms/op 0.82
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 64.185 ms/op 79.018 ms/op 0.81
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 37.381 ms/op 56.336 ms/op 0.66
getSlashingsAndExits - default max 198.91 us/op 299.60 us/op 0.66
getSlashingsAndExits - 2k 374.68 us/op 654.09 us/op 0.57
proposeBlockBody type=full, size=empty 6.6946 ms/op 7.5623 ms/op 0.89
isKnown best case - 1 super set check 308.00 ns/op 739.00 ns/op 0.42
isKnown normal case - 2 super set checks 290.00 ns/op 733.00 ns/op 0.40
isKnown worse case - 16 super set checks 277.00 ns/op 733.00 ns/op 0.38
CheckpointStateCache - add get delete 5.5940 us/op 9.7150 us/op 0.58
validate api signedAggregateAndProof - struct 2.6550 ms/op 3.7074 ms/op 0.72
validate gossip signedAggregateAndProof - struct 2.6582 ms/op 3.7124 ms/op 0.72
validate gossip attestation - vc 640000 1.2991 ms/op 1.8160 ms/op 0.72
batch validate gossip attestation - vc 640000 - chunk 32 153.35 us/op 233.50 us/op 0.66
batch validate gossip attestation - vc 640000 - chunk 64 138.05 us/op 204.04 us/op 0.68
batch validate gossip attestation - vc 640000 - chunk 128 136.25 us/op 206.64 us/op 0.66
batch validate gossip attestation - vc 640000 - chunk 256 126.39 us/op 182.84 us/op 0.69
pickEth1Vote - no votes 1.1133 ms/op 2.0266 ms/op 0.55
pickEth1Vote - max votes 10.749 ms/op 17.502 ms/op 0.61
pickEth1Vote - Eth1Data hashTreeRoot value x2048 19.274 ms/op 32.731 ms/op 0.59
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 38.291 ms/op 48.459 ms/op 0.79
pickEth1Vote - Eth1Data fastSerialize value x2048 587.61 us/op 1.2011 ms/op 0.49
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.4289 ms/op 11.866 ms/op 0.71
bytes32 toHexString 472.00 ns/op 985.00 ns/op 0.48
bytes32 Buffer.toString(hex) 280.00 ns/op 525.00 ns/op 0.53
bytes32 Buffer.toString(hex) from Uint8Array 413.00 ns/op 820.00 ns/op 0.50
bytes32 Buffer.toString(hex) + 0x 276.00 ns/op 535.00 ns/op 0.52
Object access 1 prop 0.15300 ns/op 0.29100 ns/op 0.53
Map access 1 prop 0.14500 ns/op 0.22800 ns/op 0.64
Object get x1000 6.9230 ns/op 11.448 ns/op 0.60
Map get x1000 0.72700 ns/op 1.1510 ns/op 0.63
Object set x1000 48.941 ns/op 91.412 ns/op 0.54
Map set x1000 37.897 ns/op 67.706 ns/op 0.56
Return object 10000 times 0.23160 ns/op 0.33380 ns/op 0.69
Throw Error 10000 times 3.8280 us/op 6.3300 us/op 0.60
fastMsgIdFn sha256 / 200 bytes 3.1930 us/op 4.7840 us/op 0.67
fastMsgIdFn h32 xxhash / 200 bytes 263.00 ns/op 483.00 ns/op 0.54
fastMsgIdFn h64 xxhash / 200 bytes 325.00 ns/op 579.00 ns/op 0.56
fastMsgIdFn sha256 / 1000 bytes 11.189 us/op 15.816 us/op 0.71
fastMsgIdFn h32 xxhash / 1000 bytes 391.00 ns/op 609.00 ns/op 0.64
fastMsgIdFn h64 xxhash / 1000 bytes 399.00 ns/op 649.00 ns/op 0.61
fastMsgIdFn sha256 / 10000 bytes 101.89 us/op 140.14 us/op 0.73
fastMsgIdFn h32 xxhash / 10000 bytes 1.8820 us/op 2.1380 us/op 0.88
fastMsgIdFn h64 xxhash / 10000 bytes 1.2970 us/op 1.4830 us/op 0.87
send data - 1000 256B messages 18.461 ms/op 29.949 ms/op 0.62
send data - 1000 512B messages 25.572 ms/op 38.473 ms/op 0.66
send data - 1000 1024B messages 41.221 ms/op 58.660 ms/op 0.70
send data - 1000 1200B messages 39.125 ms/op 65.198 ms/op 0.60
send data - 1000 2048B messages 49.153 ms/op 77.593 ms/op 0.63
send data - 1000 4096B messages 45.423 ms/op 48.421 ms/op 0.94
send data - 1000 16384B messages 122.58 ms/op 185.13 ms/op 0.66
send data - 1000 65536B messages 487.74 ms/op 755.93 ms/op 0.65
enrSubnets - fastDeserialize 64 bits 1.3290 us/op 2.5110 us/op 0.53
enrSubnets - ssz BitVector 64 bits 419.00 ns/op 837.00 ns/op 0.50
enrSubnets - fastDeserialize 4 bits 159.00 ns/op 326.00 ns/op 0.49
enrSubnets - ssz BitVector 4 bits 416.00 ns/op 799.00 ns/op 0.52
prioritizePeers score -10:0 att 32-0.1 sync 2-0 99.769 us/op 164.00 us/op 0.61
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 123.01 us/op 201.24 us/op 0.61
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 159.60 us/op 283.53 us/op 0.56
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 286.30 us/op 469.28 us/op 0.61
prioritizePeers score 0:0 att 64-1 sync 4-1 329.16 us/op 504.66 us/op 0.65
array of 16000 items push then shift 1.6074 us/op 2.3257 us/op 0.69
LinkedList of 16000 items push then shift 8.7750 ns/op 10.947 ns/op 0.80
array of 16000 items push then pop 84.658 ns/op 153.47 ns/op 0.55
LinkedList of 16000 items push then pop 8.5900 ns/op 11.258 ns/op 0.76
array of 24000 items push then shift 2.3915 us/op 3.4082 us/op 0.70
LinkedList of 24000 items push then shift 8.7050 ns/op 10.944 ns/op 0.80
array of 24000 items push then pop 107.48 ns/op 202.40 ns/op 0.53
LinkedList of 24000 items push then pop 8.5190 ns/op 10.785 ns/op 0.79
intersect bitArray bitLen 8 5.7000 ns/op 8.6490 ns/op 0.66
intersect array and set length 8 62.041 ns/op 124.99 ns/op 0.50
intersect bitArray bitLen 128 34.986 ns/op 49.338 ns/op 0.71
intersect array and set length 128 850.23 ns/op 1.4815 us/op 0.57
bitArray.getTrueBitIndexes() bitLen 128 1.4770 us/op 2.3830 us/op 0.62
bitArray.getTrueBitIndexes() bitLen 248 2.4680 us/op 4.0020 us/op 0.62
bitArray.getTrueBitIndexes() bitLen 512 4.7270 us/op 8.0910 us/op 0.58
Buffer.concat 32 items 1.0090 us/op 1.7910 us/op 0.56
Uint8Array.set 32 items 2.7130 us/op 2.7150 us/op 1.00
Set add up to 64 items then delete first 4.2329 us/op 6.0469 us/op 0.70
OrderedSet add up to 64 items then delete first 5.3465 us/op 8.1440 us/op 0.66
Set add up to 64 items then delete last 4.5381 us/op 6.6224 us/op 0.69
OrderedSet add up to 64 items then delete last 5.6861 us/op 8.9036 us/op 0.64
Set add up to 64 items then delete middle 4.4987 us/op 6.4994 us/op 0.69
OrderedSet add up to 64 items then delete middle 6.8906 us/op 10.467 us/op 0.66
Set add up to 128 items then delete first 9.1881 us/op 13.318 us/op 0.69
OrderedSet add up to 128 items then delete first 12.344 us/op 18.604 us/op 0.66
Set add up to 128 items then delete last 9.0584 us/op 13.107 us/op 0.69
OrderedSet add up to 128 items then delete last 11.458 us/op 17.720 us/op 0.65
Set add up to 128 items then delete middle 8.9588 us/op 13.262 us/op 0.68
OrderedSet add up to 128 items then delete middle 16.639 us/op 25.476 us/op 0.65
Set add up to 256 items then delete first 18.605 us/op 27.214 us/op 0.68
OrderedSet add up to 256 items then delete first 25.217 us/op 38.380 us/op 0.66
Set add up to 256 items then delete last 18.063 us/op 26.543 us/op 0.68
OrderedSet add up to 256 items then delete last 23.119 us/op 36.517 us/op 0.63
Set add up to 256 items then delete middle 17.908 us/op 26.717 us/op 0.67
OrderedSet add up to 256 items then delete middle 44.411 us/op 68.477 us/op 0.65
transfer serialized Status (84 B) 1.7150 us/op 3.0150 us/op 0.57
copy serialized Status (84 B) 1.3550 us/op 2.2860 us/op 0.59
transfer serialized SignedVoluntaryExit (112 B) 1.8830 us/op 3.2090 us/op 0.59
copy serialized SignedVoluntaryExit (112 B) 1.4580 us/op 2.3450 us/op 0.62
transfer serialized ProposerSlashing (416 B) 2.3220 us/op 3.6730 us/op 0.63
copy serialized ProposerSlashing (416 B) 1.9350 us/op 3.7790 us/op 0.51
transfer serialized Attestation (485 B) 2.1210 us/op 4.8540 us/op 0.44
copy serialized Attestation (485 B) 1.9210 us/op 4.5810 us/op 0.42
transfer serialized AttesterSlashing (33232 B) 2.6670 us/op 4.7060 us/op 0.57
copy serialized AttesterSlashing (33232 B) 4.9430 us/op 12.905 us/op 0.38
transfer serialized Small SignedBeaconBlock (128000 B) 2.7720 us/op 4.6140 us/op 0.60
copy serialized Small SignedBeaconBlock (128000 B) 14.210 us/op 34.541 us/op 0.41
transfer serialized Avg SignedBeaconBlock (200000 B) 3.3670 us/op 5.2430 us/op 0.64
copy serialized Avg SignedBeaconBlock (200000 B) 21.447 us/op 49.499 us/op 0.43
transfer serialized BlobsSidecar (524380 B) 3.0470 us/op 5.6120 us/op 0.54
copy serialized BlobsSidecar (524380 B) 84.447 us/op 125.09 us/op 0.68
transfer serialized Big SignedBeaconBlock (1000000 B) 3.0770 us/op 6.6100 us/op 0.47
copy serialized Big SignedBeaconBlock (1000000 B) 156.20 us/op 229.40 us/op 0.68
pass gossip attestations to forkchoice per slot 3.9351 ms/op 6.1082 ms/op 0.64
forkChoice updateHead vc 100000 bc 64 eq 0 665.20 us/op 904.09 us/op 0.74
forkChoice updateHead vc 600000 bc 64 eq 0 4.7382 ms/op 7.3537 ms/op 0.64
forkChoice updateHead vc 1000000 bc 64 eq 0 6.9095 ms/op 9.3324 ms/op 0.74
forkChoice updateHead vc 600000 bc 320 eq 0 4.1240 ms/op 5.6187 ms/op 0.73
forkChoice updateHead vc 600000 bc 1200 eq 0 4.2845 ms/op 5.8381 ms/op 0.73
forkChoice updateHead vc 600000 bc 7200 eq 0 5.1743 ms/op 7.6788 ms/op 0.67
forkChoice updateHead vc 600000 bc 64 eq 1000 10.782 ms/op 17.230 ms/op 0.63
forkChoice updateHead vc 600000 bc 64 eq 10000 11.712 ms/op 17.437 ms/op 0.67
forkChoice updateHead vc 600000 bc 64 eq 300000 16.099 ms/op 24.728 ms/op 0.65
computeDeltas 500000 validators 300 proto nodes 6.5620 ms/op 8.9553 ms/op 0.73
computeDeltas 500000 validators 1200 proto nodes 6.5290 ms/op 9.2949 ms/op 0.70
computeDeltas 500000 validators 7200 proto nodes 6.4467 ms/op 9.0264 ms/op 0.71
computeDeltas 750000 validators 300 proto nodes 9.8827 ms/op 13.505 ms/op 0.73
computeDeltas 750000 validators 1200 proto nodes 9.5312 ms/op 13.353 ms/op 0.71
computeDeltas 750000 validators 7200 proto nodes 9.8028 ms/op 13.400 ms/op 0.73
computeDeltas 1400000 validators 300 proto nodes 18.598 ms/op 24.522 ms/op 0.76
computeDeltas 1400000 validators 1200 proto nodes 19.024 ms/op 25.644 ms/op 0.74
computeDeltas 1400000 validators 7200 proto nodes 18.534 ms/op 24.456 ms/op 0.76
computeDeltas 2100000 validators 300 proto nodes 27.191 ms/op 38.595 ms/op 0.70
computeDeltas 2100000 validators 1200 proto nodes 27.268 ms/op 39.240 ms/op 0.69
computeDeltas 2100000 validators 7200 proto nodes 27.184 ms/op 38.272 ms/op 0.71
altair processAttestation - 250000 vs - 7PWei normalcase 2.1355 ms/op 3.7609 ms/op 0.57
altair processAttestation - 250000 vs - 7PWei worstcase 3.1319 ms/op 5.4933 ms/op 0.57
altair processAttestation - setStatus - 1/6 committees join 181.15 us/op 270.62 us/op 0.67
altair processAttestation - setStatus - 1/3 committees join 347.00 us/op 537.42 us/op 0.65
altair processAttestation - setStatus - 1/2 committees join 518.97 us/op 724.28 us/op 0.72
altair processAttestation - setStatus - 2/3 committees join 595.80 us/op 927.99 us/op 0.64
altair processAttestation - setStatus - 4/5 committees join 825.51 us/op 1.1855 ms/op 0.70
altair processAttestation - setStatus - 100% committees join 933.80 us/op 1.3576 ms/op 0.69
altair processBlock - 250000 vs - 7PWei normalcase 9.9520 ms/op 14.156 ms/op 0.70
altair processBlock - 250000 vs - 7PWei normalcase hashState 38.159 ms/op 46.842 ms/op 0.81
altair processBlock - 250000 vs - 7PWei worstcase 39.690 ms/op 56.581 ms/op 0.70
altair processBlock - 250000 vs - 7PWei worstcase hashState 93.708 ms/op 138.70 ms/op 0.68
phase0 processBlock - 250000 vs - 7PWei normalcase 2.7114 ms/op 4.2721 ms/op 0.63
phase0 processBlock - 250000 vs - 7PWei worstcase 28.863 ms/op 42.012 ms/op 0.69
altair processEth1Data - 250000 vs - 7PWei normalcase 635.19 us/op 4.3122 ms/op 0.15
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 12.631 us/op 28.223 us/op 0.45
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 57.097 us/op 102.61 us/op 0.56
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.420 us/op 34.859 us/op 0.30
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 13.732 us/op 22.472 us/op 0.61
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 208.69 us/op 338.50 us/op 0.62
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.2656 ms/op 2.8814 ms/op 0.44
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.9865 ms/op 3.2737 ms/op 0.61
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.9206 ms/op 3.3222 ms/op 0.88
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.5879 ms/op 6.2993 ms/op 0.57
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.6819 ms/op 4.9465 ms/op 0.54
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.7693 ms/op 8.2016 ms/op 0.58
Tree 40 250000 create 356.14 ms/op 857.31 ms/op 0.42
Tree 40 250000 get(125000) 198.22 ns/op 318.97 ns/op 0.62
Tree 40 250000 set(125000) 1.0355 us/op 3.0290 us/op 0.34
Tree 40 250000 toArray() 20.110 ms/op 28.597 ms/op 0.70
Tree 40 250000 iterate all - toArray() + loop 18.911 ms/op 28.648 ms/op 0.66
Tree 40 250000 iterate all - get(i) 65.890 ms/op 110.66 ms/op 0.60
MutableVector 250000 create 20.635 ms/op 25.758 ms/op 0.80
MutableVector 250000 get(125000) 6.5820 ns/op 9.7130 ns/op 0.68
MutableVector 250000 set(125000) 272.76 ns/op 682.58 ns/op 0.40
MutableVector 250000 toArray() 3.2984 ms/op 4.8961 ms/op 0.67
MutableVector 250000 iterate all - toArray() + loop 3.5241 ms/op 5.6930 ms/op 0.62
MutableVector 250000 iterate all - get(i) 1.5165 ms/op 2.4039 ms/op 0.63
Array 250000 create 3.0627 ms/op 4.8804 ms/op 0.63
Array 250000 clone - spread 1.3396 ms/op 3.0186 ms/op 0.44
Array 250000 get(125000) 1.0860 ns/op 2.0130 ns/op 0.54
Array 250000 set(125000) 4.1840 ns/op 4.8290 ns/op 0.87
Array 250000 iterate all - loop 167.67 us/op 225.73 us/op 0.74
effectiveBalanceIncrements clone Uint8Array 300000 31.972 us/op 72.262 us/op 0.44
effectiveBalanceIncrements clone MutableVector 300000 421.00 ns/op 1.0550 us/op 0.40
effectiveBalanceIncrements rw all Uint8Array 300000 200.80 us/op 300.81 us/op 0.67
effectiveBalanceIncrements rw all MutableVector 300000 91.810 ms/op 215.06 ms/op 0.43
phase0 afterProcessEpoch - 250000 vs - 7PWei 119.07 ms/op 160.58 ms/op 0.74
phase0 beforeProcessEpoch - 250000 vs - 7PWei 53.202 ms/op 74.027 ms/op 0.72
altair processEpoch - mainnet_e81889 515.12 ms/op 740.81 ms/op 0.70
mainnet_e81889 - altair beforeProcessEpoch 86.806 ms/op 116.89 ms/op 0.74
mainnet_e81889 - altair processJustificationAndFinalization 15.244 us/op 33.374 us/op 0.46
mainnet_e81889 - altair processInactivityUpdates 5.7565 ms/op 10.346 ms/op 0.56
mainnet_e81889 - altair processRewardsAndPenalties 62.513 ms/op 91.911 ms/op 0.68
mainnet_e81889 - altair processRegistryUpdates 2.5650 us/op 5.9580 us/op 0.43
mainnet_e81889 - altair processSlashings 344.00 ns/op 1.0240 us/op 0.34
mainnet_e81889 - altair processEth1DataReset 448.00 ns/op 1.5010 us/op 0.30
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4051 ms/op 2.6398 ms/op 0.53
mainnet_e81889 - altair processSlashingsReset 4.2430 us/op 8.6850 us/op 0.49
mainnet_e81889 - altair processRandaoMixesReset 4.5860 us/op 10.493 us/op 0.44
mainnet_e81889 - altair processHistoricalRootsUpdate 879.00 ns/op 1.3860 us/op 0.63
mainnet_e81889 - altair processParticipationFlagUpdates 2.8480 us/op 4.3380 us/op 0.66
mainnet_e81889 - altair processSyncCommitteeUpdates 793.00 ns/op 1.4150 us/op 0.56
mainnet_e81889 - altair afterProcessEpoch 123.04 ms/op 171.18 ms/op 0.72
capella processEpoch - mainnet_e217614 2.0620 s/op 3.2047 s/op 0.64
mainnet_e217614 - capella beforeProcessEpoch 470.83 ms/op 744.72 ms/op 0.63
mainnet_e217614 - capella processJustificationAndFinalization 20.285 us/op 31.329 us/op 0.65
mainnet_e217614 - capella processInactivityUpdates 18.149 ms/op 27.756 ms/op 0.65
mainnet_e217614 - capella processRewardsAndPenalties 586.99 ms/op 821.96 ms/op 0.71
mainnet_e217614 - capella processRegistryUpdates 18.526 us/op 47.506 us/op 0.39
mainnet_e217614 - capella processSlashings 631.00 ns/op 1.5170 us/op 0.42
mainnet_e217614 - capella processEth1DataReset 462.00 ns/op 1.0690 us/op 0.43
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.2174 ms/op 7.5465 ms/op 0.56
mainnet_e217614 - capella processSlashingsReset 3.6930 us/op 5.6780 us/op 0.65
mainnet_e217614 - capella processRandaoMixesReset 7.0250 us/op 10.646 us/op 0.66
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0900 us/op 1.2500 us/op 0.87
mainnet_e217614 - capella processParticipationFlagUpdates 3.3140 us/op 3.6320 us/op 0.91
mainnet_e217614 - capella afterProcessEpoch 322.75 ms/op 527.18 ms/op 0.61
phase0 processEpoch - mainnet_e58758 529.26 ms/op 711.00 ms/op 0.74
mainnet_e58758 - phase0 beforeProcessEpoch 143.83 ms/op 235.91 ms/op 0.61
mainnet_e58758 - phase0 processJustificationAndFinalization 21.541 us/op 38.247 us/op 0.56
mainnet_e58758 - phase0 processRewardsAndPenalties 72.995 ms/op 78.943 ms/op 0.92
mainnet_e58758 - phase0 processRegistryUpdates 14.803 us/op 21.427 us/op 0.69
mainnet_e58758 - phase0 processSlashings 869.00 ns/op 1.2770 us/op 0.68
mainnet_e58758 - phase0 processEth1DataReset 626.00 ns/op 1.2420 us/op 0.50
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.1137 ms/op 2.6001 ms/op 0.81
mainnet_e58758 - phase0 processSlashingsReset 3.6890 us/op 7.2060 us/op 0.51
mainnet_e58758 - phase0 processRandaoMixesReset 6.0810 us/op 9.6340 us/op 0.63
mainnet_e58758 - phase0 processHistoricalRootsUpdate 632.00 ns/op 1.1990 us/op 0.53
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.7860 us/op 9.0110 us/op 0.53
mainnet_e58758 - phase0 afterProcessEpoch 99.915 ms/op 141.88 ms/op 0.70
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2064 ms/op 3.0409 ms/op 0.40
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.3980 ms/op 3.0245 ms/op 0.46
altair processInactivityUpdates - 250000 normalcase 30.277 ms/op 41.679 ms/op 0.73
altair processInactivityUpdates - 250000 worstcase 30.340 ms/op 47.210 ms/op 0.64
phase0 processRegistryUpdates - 250000 normalcase 12.721 us/op 23.351 us/op 0.54
phase0 processRegistryUpdates - 250000 badcase_full_deposits 777.41 us/op 958.26 us/op 0.81
phase0 processRegistryUpdates - 250000 worstcase 0.5 127.00 ms/op 243.16 ms/op 0.52
altair processRewardsAndPenalties - 250000 normalcase 91.103 ms/op 110.20 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 97.041 ms/op 115.68 ms/op 0.84
phase0 getAttestationDeltas - 250000 normalcase 8.1516 ms/op 15.552 ms/op 0.52
phase0 getAttestationDeltas - 250000 worstcase 7.9216 ms/op 15.657 ms/op 0.51
phase0 processSlashings - 250000 worstcase 130.20 us/op 201.65 us/op 0.65
altair processSyncCommitteeUpdates - 250000 153.92 ms/op 258.40 ms/op 0.60
BeaconState.hashTreeRoot - No change 382.00 ns/op 1.0220 us/op 0.37
BeaconState.hashTreeRoot - 1 full validator 138.19 us/op 232.20 us/op 0.60
BeaconState.hashTreeRoot - 32 full validator 1.7832 ms/op 2.3868 ms/op 0.75
BeaconState.hashTreeRoot - 512 full validator 19.544 ms/op 23.838 ms/op 0.82
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 209.80 us/op 228.59 us/op 0.92
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.3683 ms/op 3.1884 ms/op 0.74
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 31.080 ms/op 61.487 ms/op 0.51
BeaconState.hashTreeRoot - 1 balances 145.04 us/op 181.33 us/op 0.80
BeaconState.hashTreeRoot - 32 balances 1.5062 ms/op 1.6054 ms/op 0.94
BeaconState.hashTreeRoot - 512 balances 12.264 ms/op 16.107 ms/op 0.76
BeaconState.hashTreeRoot - 250000 balances 266.07 ms/op 303.28 ms/op 0.88
aggregationBits - 2048 els - zipIndexesInBitList 26.057 us/op 51.158 us/op 0.51
byteArrayEquals 32 74.618 ns/op 107.92 ns/op 0.69
Buffer.compare 32 56.835 ns/op 92.800 ns/op 0.61
byteArrayEquals 1024 2.0392 us/op 2.7927 us/op 0.73
Buffer.compare 1024 73.979 ns/op 93.983 ns/op 0.79
byteArrayEquals 16384 32.787 us/op 44.605 us/op 0.74
Buffer.compare 16384 274.71 ns/op 369.19 ns/op 0.74
byteArrayEquals 123687377 247.41 ms/op 335.46 ms/op 0.74
Buffer.compare 123687377 6.1084 ms/op 10.121 ms/op 0.60
byteArrayEquals 32 - diff last byte 72.304 ns/op 98.680 ns/op 0.73
Buffer.compare 32 - diff last byte 57.608 ns/op 89.630 ns/op 0.64
byteArrayEquals 1024 - diff last byte 2.0543 us/op 2.7854 us/op 0.74
Buffer.compare 1024 - diff last byte 71.169 ns/op 94.667 ns/op 0.75
byteArrayEquals 16384 - diff last byte 32.799 us/op 44.528 us/op 0.74
Buffer.compare 16384 - diff last byte 275.80 ns/op 359.23 ns/op 0.77
byteArrayEquals 123687377 - diff last byte 247.16 ms/op 337.37 ms/op 0.73
Buffer.compare 123687377 - diff last byte 6.1830 ms/op 10.250 ms/op 0.60
byteArrayEquals 32 - random bytes 5.3410 ns/op 8.1840 ns/op 0.65
Buffer.compare 32 - random bytes 61.831 ns/op 92.012 ns/op 0.67
byteArrayEquals 1024 - random bytes 5.1810 ns/op 7.8110 ns/op 0.66
Buffer.compare 1024 - random bytes 61.074 ns/op 88.934 ns/op 0.69
byteArrayEquals 16384 - random bytes 5.1650 ns/op 7.7870 ns/op 0.66
Buffer.compare 16384 - random bytes 61.267 ns/op 90.205 ns/op 0.68
byteArrayEquals 123687377 - random bytes 8.2700 ns/op 15.680 ns/op 0.53
Buffer.compare 123687377 - random bytes 63.660 ns/op 96.800 ns/op 0.66
regular array get 100000 times 44.543 us/op 60.984 us/op 0.73
wrappedArray get 100000 times 44.214 us/op 60.852 us/op 0.73
arrayWithProxy get 100000 times 14.544 ms/op 26.458 ms/op 0.55
ssz.Root.equals 53.433 ns/op 83.308 ns/op 0.64
byteArrayEquals 52.477 ns/op 82.734 ns/op 0.63
Buffer.compare 10.731 ns/op 16.592 ns/op 0.65
shuffle list - 16384 els 8.3756 ms/op 12.047 ms/op 0.70
shuffle list - 250000 els 124.40 ms/op 175.53 ms/op 0.71
processSlot - 1 slots 19.538 us/op 30.874 us/op 0.63
processSlot - 32 slots 3.8472 ms/op 6.2641 ms/op 0.61
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 63.293 ms/op 90.649 ms/op 0.70
getCommitteeAssignments - req 1 vs - 250000 vc 2.6333 ms/op 4.1630 ms/op 0.63
getCommitteeAssignments - req 100 vs - 250000 vc 3.7933 ms/op 5.9788 ms/op 0.63
getCommitteeAssignments - req 1000 vs - 250000 vc 4.1254 ms/op 6.4321 ms/op 0.64
findModifiedValidators - 10000 modified validators 552.81 ms/op 884.41 ms/op 0.63
findModifiedValidators - 1000 modified validators 424.84 ms/op 682.54 ms/op 0.62
findModifiedValidators - 100 modified validators 405.17 ms/op 664.39 ms/op 0.61
findModifiedValidators - 10 modified validators 408.80 ms/op 718.20 ms/op 0.57
findModifiedValidators - 1 modified validators 395.40 ms/op 678.49 ms/op 0.58
findModifiedValidators - no difference 426.61 ms/op 670.88 ms/op 0.64
compare ViewDUs 4.7566 s/op 7.2137 s/op 0.66
compare each validator Uint8Array 1.9973 s/op 2.7797 s/op 0.72
compare ViewDU to Uint8Array 1.1412 s/op 2.2684 s/op 0.50
migrate state 1000000 validators, 24 modified, 0 new 916.24 ms/op 1.3269 s/op 0.69
migrate state 1000000 validators, 1700 modified, 1000 new 1.2275 s/op 1.8038 s/op 0.68
migrate state 1000000 validators, 3400 modified, 2000 new 1.3923 s/op 2.1289 s/op 0.65
migrate state 1500000 validators, 24 modified, 0 new 848.98 ms/op 1.3643 s/op 0.62
migrate state 1500000 validators, 1700 modified, 1000 new 1.1496 s/op 1.8601 s/op 0.62
migrate state 1500000 validators, 3400 modified, 2000 new 1.4065 s/op 2.2647 s/op 0.62
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.5000 ns/op 7.7800 ns/op 0.58
state getBlockRootAtSlot - 250000 vs - 7PWei 1.0160 us/op 1.5564 us/op 0.65
computeProposers - vc 250000 10.504 ms/op 15.073 ms/op 0.70
computeEpochShuffling - vc 250000 129.28 ms/op 175.50 ms/op 0.74
getNextSyncCommittee - vc 250000 158.38 ms/op 250.79 ms/op 0.63
computeSigningRoot for AttestationData 26.970 us/op 43.694 us/op 0.62
hash AttestationData serialized data then Buffer.toString(base64) 2.3234 us/op 3.3113 us/op 0.70
toHexString serialized data 1.0875 us/op 1.8364 us/op 0.59
Buffer.toString(base64) 206.81 ns/op 412.58 ns/op 0.50

by benchmarkbot/action

jeluard
jeluard previously approved these changes Mar 19, 2024
Copy link
Member

@jeluard jeluard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not an expert in sim tests, but overall the approach sounds sensible to me. And always 👍 removing complexity :)

*/
const result = [`Slot,${nodes.map((n) => n.beacon.id).join(", ,")}`];
result.push(`,${nodes.map((_) => "Withdrawal Amount,Withdrawal Count").join(",")}`);
for (let s = 1; s <= slot; s++) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use const, not let?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then TS would not allow to use s++, you can't mutate a const declared primitive.

let row = `${s}`;
for (const node of nodes) {
const {withdrawalAmount, withdrawalCount} = store[node.beacon.id][s] ?? {};
row += `,${withdrawalAmount ?? "-"},${withdrawalCount ?? "-"}`;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some combination of Array#join and Array#map could be used here. Just for reference, for this PR looks ok.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This iteration approach looks more readable.

const validators: WithdrawalsData["validators"] = {};

for (const withdrawal of withdrawals) {
withdrawalCount++;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it can only ever end up being withdrawals.length ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch. was using different approach earlier so this variable left as counter.

return {
id: `withdrawals_${nodeId}`,
match({forkConfig, epoch, node}) {
if (nodeId === node.id && epoch === forkConfig.CAPELLA_FORK_EPOCH)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised this line pass lints (no curly brasses).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was actually written as single line, but prettier changed it to this format. And our ESlint don't complaint on Prettier formatting.

@@ -0,0 +1,98 @@
/* eslint-disable @typescript-eslint/naming-convention */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to lints, this directive is useless.

@@ -71,39 +71,6 @@ jobs:
ENGINE_PORT: 8551
ETH_PORT: 8661

- name: Pull geth withdrawals
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we need to remove the whole file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet, one other PR in progress then we can remove this file and a lot of utility code along it.

let withdrawalAmount = BigInt(0);
const validators: WithdrawalsData["validators"] = {};

for (const withdrawal of withdrawals) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if it's empty withdrawals? is it scanned through assert() function below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes the assertion works on this logic.

      if (nodeId === node.id && epoch === forkConfig.CAPELLA_FORK_EPOCH)
        return AssertionMatch.Capture | AssertionMatch.Assert;

So even if the above line withdrawals are empty, the assertion will still trigger and results in empty errors array.

twoeths
twoeths previously approved these changes Mar 20, 2024
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me, spent some time to see how capture() and assert() works. Nice work in SimulationTracker @nazarhussain . It's difficult for me to debug it through.

@nazarhussain nazarhussain dismissed stale reviews from twoeths and jeluard via c69d5d5 March 20, 2024 04:14
@nazarhussain
Copy link
Contributor Author

see how capture() and assert() works

Both works based on the match output. One key difference is capture works for each node in the env sequently and assert execute once for all nodes.

@nazarhussain nazarhussain merged commit b68e78e into unstable Mar 20, 2024
19 of 20 checks passed
@nazarhussain nazarhussain deleted the nh/sim-withdrawals branch March 20, 2024 12:44
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants