Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: upgrade geth version in the sim tests #6540

Merged
merged 6 commits into from Mar 13, 2024
Merged

test: upgrade geth version in the sim tests #6540

merged 6 commits into from Mar 13, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Make the tests more stable.

Description

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner March 13, 2024 11:27
@nazarhussain nazarhussain self-assigned this Mar 13, 2024
@nazarhussain nazarhussain changed the title Upgrade geth version in the sim tests test: upgrade geth version in the sim tests Mar 13, 2024
@codecov Codecov
Copy link

codecov bot commented Mar 13, 2024 •
edited

Codecov Report

Merging #6540 (7cf7003) into unstable (1d3bf04) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #6540   +/-   ##
=========================================
  Coverage     61.53%   61.53%           
=========================================
  Files           556      556           
  Lines         58649    58649           
  Branches       1847     1847           
=========================================
  Hits          36089    36089           
  Misses        22520    22520           
  Partials         40       40

jeluard
jeluard requested changes Mar 13, 2024
View reviewed changes
.env.test Outdated Show resolved Hide resolved
.env.test Outdated Show resolved Hide resolved
packages/cli/test/utils/simulation/execution_clients/geth.ts Show resolved Hide resolved
@nazarhussain @jeluard
Apply suggestions from code review
4081f91 
Co-authored-by: Julien <jeluard@users.noreply.github.com>
jeluard
jeluard previously approved these changes Mar 13, 2024
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: e6ea7fd Previous: 1d3bf04 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 939.74 us/op 606.52 us/op 1.55
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 106.05 us/op 86.944 us/op 1.22
BLS verify - blst-native 1.3447 ms/op 1.2966 ms/op 1.04
BLS verifyMultipleSignatures 3 - blst-native 2.8127 ms/op 2.7147 ms/op 1.04
BLS verifyMultipleSignatures 8 - blst-native 6.5138 ms/op 5.9621 ms/op 1.09
BLS verifyMultipleSignatures 32 - blst-native 23.909 ms/op 21.908 ms/op 1.09
BLS verifyMultipleSignatures 64 - blst-native 45.827 ms/op 42.948 ms/op 1.07
BLS verifyMultipleSignatures 128 - blst-native 91.912 ms/op 85.350 ms/op 1.08
BLS deserializing 10000 signatures 943.85 ms/op 882.06 ms/op 1.07
BLS deserializing 100000 signatures 9.2787 s/op 9.3963 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.4848 ms/op 1.3215 ms/op 1.12
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6372 ms/op 1.4888 ms/op 1.10
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4329 ms/op 2.8733 ms/op 0.85
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.6377 ms/op 4.3699 ms/op 0.83
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.9668 ms/op 7.5286 ms/op 0.79
BLS aggregatePubkeys 32 - blst-native 27.276 us/op 27.792 us/op 0.98
BLS aggregatePubkeys 128 - blst-native 103.09 us/op 101.58 us/op 1.01
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 63.872 ms/op 65.766 ms/op 0.97
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 64.035 ms/op 60.030 ms/op 1.07
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 53.365 ms/op 37.133 ms/op 1.44
getSlashingsAndExits - default max 189.33 us/op 182.54 us/op 1.04
getSlashingsAndExits - 2k 454.50 us/op 466.65 us/op 0.97
proposeBlockBody type=full, size=empty 6.1440 ms/op 5.1539 ms/op 1.19
isKnown best case - 1 super set check 536.00 ns/op 459.00 ns/op 1.17
isKnown normal case - 2 super set checks 462.00 ns/op 532.00 ns/op 0.87
isKnown worse case - 16 super set checks 345.00 ns/op 515.00 ns/op 0.67
CheckpointStateCache - add get delete 6.3200 us/op 7.1050 us/op 0.89
validate api signedAggregateAndProof - struct 2.8759 ms/op 2.7724 ms/op 1.04
validate gossip signedAggregateAndProof - struct 2.9756 ms/op 2.7714 ms/op 1.07
validate gossip attestation - vc 640000 1.4266 ms/op 1.3551 ms/op 1.05
batch validate gossip attestation - vc 640000 - chunk 32 180.19 us/op 166.41 us/op 1.08
batch validate gossip attestation - vc 640000 - chunk 64 159.91 us/op 150.07 us/op 1.07
batch validate gossip attestation - vc 640000 - chunk 128 172.81 us/op 149.05 us/op 1.16
batch validate gossip attestation - vc 640000 - chunk 256 176.27 us/op 138.26 us/op 1.27
pickEth1Vote - no votes 1.9390 ms/op 1.3712 ms/op 1.41
pickEth1Vote - max votes 17.526 ms/op 12.003 ms/op 1.46
pickEth1Vote - Eth1Data hashTreeRoot value x2048 28.247 ms/op 24.145 ms/op 1.17
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 43.278 ms/op 29.232 ms/op 1.48
pickEth1Vote - Eth1Data fastSerialize value x2048 960.60 us/op 630.35 us/op 1.52
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.7904 ms/op 4.6213 ms/op 1.47
bytes32 toHexString 963.00 ns/op 506.00 ns/op 1.90
bytes32 Buffer.toString(hex) 389.00 ns/op 288.00 ns/op 1.35
bytes32 Buffer.toString(hex) from Uint8Array 658.00 ns/op 447.00 ns/op 1.47
bytes32 Buffer.toString(hex) + 0x 359.00 ns/op 292.00 ns/op 1.23
Object access 1 prop 0.26900 ns/op 0.16900 ns/op 1.59
Map access 1 prop 0.18700 ns/op 0.15500 ns/op 1.21
Object get x1000 9.0890 ns/op 7.2060 ns/op 1.26
Map get x1000 1.0750 ns/op 0.78800 ns/op 1.36
Object set x1000 92.667 ns/op 54.556 ns/op 1.70
Map set x1000 64.213 ns/op 41.142 ns/op 1.56
Return object 10000 times 0.29340 ns/op 0.25320 ns/op 1.16
Throw Error 10000 times 4.5583 us/op 3.8923 us/op 1.17
fastMsgIdFn sha256 / 200 bytes 3.9880 us/op 3.3700 us/op 1.18
fastMsgIdFn h32 xxhash / 200 bytes 431.00 ns/op 293.00 ns/op 1.47
fastMsgIdFn h64 xxhash / 200 bytes 467.00 ns/op 370.00 ns/op 1.26
fastMsgIdFn sha256 / 1000 bytes 13.352 us/op 11.831 us/op 1.13
fastMsgIdFn h32 xxhash / 1000 bytes 549.00 ns/op 459.00 ns/op 1.20
fastMsgIdFn h64 xxhash / 1000 bytes 551.00 ns/op 460.00 ns/op 1.20
fastMsgIdFn sha256 / 10000 bytes 118.00 us/op 104.04 us/op 1.13
fastMsgIdFn h32 xxhash / 10000 bytes 2.2600 us/op 1.9990 us/op 1.13
fastMsgIdFn h64 xxhash / 10000 bytes 1.5870 us/op 1.3790 us/op 1.15
send data - 1000 256B messages 24.529 ms/op 19.833 ms/op 1.24
send data - 1000 512B messages 27.765 ms/op 26.335 ms/op 1.05
send data - 1000 1024B messages 50.095 ms/op 44.333 ms/op 1.13
send data - 1000 1200B messages 53.547 ms/op 40.265 ms/op 1.33
send data - 1000 2048B messages 61.573 ms/op 51.648 ms/op 1.19
send data - 1000 4096B messages 50.633 ms/op 48.653 ms/op 1.04
send data - 1000 16384B messages 130.09 ms/op 122.53 ms/op 1.06
send data - 1000 65536B messages 581.41 ms/op 487.42 ms/op 1.19
enrSubnets - fastDeserialize 64 bits 1.7700 us/op 1.5230 us/op 1.16
enrSubnets - ssz BitVector 64 bits 593.00 ns/op 526.00 ns/op 1.13
enrSubnets - fastDeserialize 4 bits 259.00 ns/op 221.00 ns/op 1.17
enrSubnets - ssz BitVector 4 bits 615.00 ns/op 529.00 ns/op 1.16
prioritizePeers score -10:0 att 32-0.1 sync 2-0 130.86 us/op 122.66 us/op 1.07
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 163.24 us/op 135.08 us/op 1.21
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 213.89 us/op 194.23 us/op 1.10
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 383.99 us/op 334.36 us/op 1.15
prioritizePeers score 0:0 att 64-1 sync 4-1 476.05 us/op 363.42 us/op 1.31
array of 16000 items push then shift 1.7784 us/op 1.6814 us/op 1.06
LinkedList of 16000 items push then shift 9.5090 ns/op 9.3710 ns/op 1.01
array of 16000 items push then pop 120.18 ns/op 110.40 ns/op 1.09
LinkedList of 16000 items push then pop 10.404 ns/op 8.9310 ns/op 1.16
array of 24000 items push then shift 2.7803 us/op 2.6170 us/op 1.06
LinkedList of 24000 items push then shift 9.4950 ns/op 9.2310 ns/op 1.03
array of 24000 items push then pop 151.22 ns/op 143.61 ns/op 1.05
LinkedList of 24000 items push then pop 9.0970 ns/op 9.0120 ns/op 1.01
intersect bitArray bitLen 8 6.4060 ns/op 5.8330 ns/op 1.10
intersect array and set length 8 91.359 ns/op 65.617 ns/op 1.39
intersect bitArray bitLen 128 37.881 ns/op 35.650 ns/op 1.06
intersect array and set length 128 1.0706 us/op 987.18 ns/op 1.08
bitArray.getTrueBitIndexes() bitLen 128 1.8270 us/op 1.6320 us/op 1.12
bitArray.getTrueBitIndexes() bitLen 248 3.1540 us/op 2.9320 us/op 1.08
bitArray.getTrueBitIndexes() bitLen 512 6.0780 us/op 5.5050 us/op 1.10
Buffer.concat 32 items 1.0670 us/op 1.0500 us/op 1.02
Uint8Array.set 32 items 1.9450 us/op 1.9510 us/op 1.00
Set add up to 64 items then delete first 5.0763 us/op 4.7044 us/op 1.08
OrderedSet add up to 64 items then delete first 6.8795 us/op 6.1379 us/op 1.12
Set add up to 64 items then delete last 5.5835 us/op 5.2804 us/op 1.06
OrderedSet add up to 64 items then delete last 7.0939 us/op 6.7648 us/op 1.05
Set add up to 64 items then delete middle 5.3816 us/op 5.3317 us/op 1.01
OrderedSet add up to 64 items then delete middle 8.3894 us/op 8.0847 us/op 1.04
Set add up to 128 items then delete first 11.510 us/op 10.596 us/op 1.09
OrderedSet add up to 128 items then delete first 15.533 us/op 13.173 us/op 1.18
Set add up to 128 items then delete last 11.517 us/op 10.562 us/op 1.09
OrderedSet add up to 128 items then delete last 14.919 us/op 14.727 us/op 1.01
Set add up to 128 items then delete middle 11.514 us/op 10.492 us/op 1.10
OrderedSet add up to 128 items then delete middle 20.738 us/op 19.213 us/op 1.08
Set add up to 256 items then delete first 21.747 us/op 22.309 us/op 0.97
OrderedSet add up to 256 items then delete first 30.667 us/op 29.751 us/op 1.03
Set add up to 256 items then delete last 22.071 us/op 21.169 us/op 1.04
OrderedSet add up to 256 items then delete last 29.982 us/op 25.525 us/op 1.17
Set add up to 256 items then delete middle 22.304 us/op 20.402 us/op 1.09
OrderedSet add up to 256 items then delete middle 55.125 us/op 50.585 us/op 1.09
transfer serialized Status (84 B) 2.1290 us/op 1.8460 us/op 1.15
copy serialized Status (84 B) 1.6630 us/op 1.4090 us/op 1.18
transfer serialized SignedVoluntaryExit (112 B) 2.4120 us/op 1.9980 us/op 1.21
copy serialized SignedVoluntaryExit (112 B) 1.7570 us/op 1.4650 us/op 1.20
transfer serialized ProposerSlashing (416 B) 3.4990 us/op 2.2340 us/op 1.57
copy serialized ProposerSlashing (416 B) 3.0790 us/op 2.6360 us/op 1.17
transfer serialized Attestation (485 B) 3.1720 us/op 2.7110 us/op 1.17
copy serialized Attestation (485 B) 2.6560 us/op 2.0050 us/op 1.32
transfer serialized AttesterSlashing (33232 B) 2.3960 us/op 2.1280 us/op 1.13
copy serialized AttesterSlashing (33232 B) 16.803 us/op 6.9360 us/op 2.42
transfer serialized Small SignedBeaconBlock (128000 B) 2.4160 us/op 2.3430 us/op 1.03
copy serialized Small SignedBeaconBlock (128000 B) 26.083 us/op 18.168 us/op 1.44
transfer serialized Avg SignedBeaconBlock (200000 B) 3.1200 us/op 2.6890 us/op 1.16
copy serialized Avg SignedBeaconBlock (200000 B) 89.881 us/op 25.701 us/op 3.50
transfer serialized BlobsSidecar (524380 B) 3.0980 us/op 2.9830 us/op 1.04
copy serialized BlobsSidecar (524380 B) 260.18 us/op 92.492 us/op 2.81
transfer serialized Big SignedBeaconBlock (1000000 B) 2.9670 us/op 3.3640 us/op 0.88
copy serialized Big SignedBeaconBlock (1000000 B) 397.06 us/op 157.67 us/op 2.52
pass gossip attestations to forkchoice per slot 4.2344 ms/op 3.9003 ms/op 1.09
forkChoice updateHead vc 100000 bc 64 eq 0 829.50 us/op 691.37 us/op 1.20
forkChoice updateHead vc 600000 bc 64 eq 0 6.7056 ms/op 4.2396 ms/op 1.58
forkChoice updateHead vc 1000000 bc 64 eq 0 7.7175 ms/op 7.3896 ms/op 1.04
forkChoice updateHead vc 600000 bc 320 eq 0 4.6534 ms/op 4.3559 ms/op 1.07
forkChoice updateHead vc 600000 bc 1200 eq 0 4.7132 ms/op 4.3983 ms/op 1.07
forkChoice updateHead vc 600000 bc 7200 eq 0 5.9349 ms/op 5.3140 ms/op 1.12
forkChoice updateHead vc 600000 bc 64 eq 1000 12.253 ms/op 12.571 ms/op 0.97
forkChoice updateHead vc 600000 bc 64 eq 10000 12.864 ms/op 12.371 ms/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 300000 19.378 ms/op 19.303 ms/op 1.00
computeDeltas 500000 validators 300 proto nodes 7.2581 ms/op 6.9228 ms/op 1.05
computeDeltas 500000 validators 1200 proto nodes 7.1318 ms/op 6.6733 ms/op 1.07
computeDeltas 500000 validators 7200 proto nodes 6.8140 ms/op 6.7586 ms/op 1.01
computeDeltas 750000 validators 300 proto nodes 9.6968 ms/op 9.9998 ms/op 0.97
computeDeltas 750000 validators 1200 proto nodes 9.6391 ms/op 10.125 ms/op 0.95
computeDeltas 750000 validators 7200 proto nodes 9.6720 ms/op 9.7364 ms/op 0.99
computeDeltas 1400000 validators 300 proto nodes 18.628 ms/op 18.636 ms/op 1.00
computeDeltas 1400000 validators 1200 proto nodes 19.087 ms/op 19.329 ms/op 0.99
computeDeltas 1400000 validators 7200 proto nodes 18.695 ms/op 19.084 ms/op 0.98
computeDeltas 2100000 validators 300 proto nodes 27.595 ms/op 27.986 ms/op 0.99
computeDeltas 2100000 validators 1200 proto nodes 27.008 ms/op 28.276 ms/op 0.96
computeDeltas 2100000 validators 7200 proto nodes 27.292 ms/op 28.852 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 2.1099 ms/op 2.7005 ms/op 0.78
altair processAttestation - 250000 vs - 7PWei worstcase 3.3266 ms/op 3.6883 ms/op 0.90
altair processAttestation - setStatus - 1/6 committees join 183.76 us/op 195.95 us/op 0.94
altair processAttestation - setStatus - 1/3 committees join 345.97 us/op 367.63 us/op 0.94
altair processAttestation - setStatus - 1/2 committees join 464.90 us/op 498.00 us/op 0.93
altair processAttestation - setStatus - 2/3 committees join 584.84 us/op 609.65 us/op 0.96
altair processAttestation - setStatus - 4/5 committees join 786.75 us/op 834.24 us/op 0.94
altair processAttestation - setStatus - 100% committees join 898.57 us/op 951.09 us/op 0.94
altair processBlock - 250000 vs - 7PWei normalcase 11.308 ms/op 10.379 ms/op 1.09
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.277 ms/op 35.005 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase 41.411 ms/op 43.252 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase hashState 89.727 ms/op 95.435 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei normalcase 2.8698 ms/op 3.0280 ms/op 0.95
phase0 processBlock - 250000 vs - 7PWei worstcase 28.840 ms/op 32.821 ms/op 0.88
altair processEth1Data - 250000 vs - 7PWei normalcase 477.69 us/op 639.16 us/op 0.75
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 13.679 us/op 16.727 us/op 0.82
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 136.56 us/op 73.129 us/op 1.87
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 27.155 us/op 26.832 us/op 1.01
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 19.683 us/op 18.018 us/op 1.09
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 282.49 us/op 168.33 us/op 1.68
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 2.6831 ms/op 1.6084 ms/op 1.67
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.8777 ms/op 2.0008 ms/op 1.44
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.8161 ms/op 2.1590 ms/op 1.30
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.6725 ms/op 3.8998 ms/op 0.94
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.4017 ms/op 2.7327 ms/op 0.88
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.1235 ms/op 5.8912 ms/op 0.87
Tree 40 250000 create 347.08 ms/op 444.23 ms/op 0.78
Tree 40 250000 get(125000) 198.32 ns/op 212.41 ns/op 0.93
Tree 40 250000 set(125000) 986.24 ns/op 1.1003 us/op 0.90
Tree 40 250000 toArray() 18.537 ms/op 23.743 ms/op 0.78
Tree 40 250000 iterate all - toArray() + loop 18.500 ms/op 25.253 ms/op 0.73
Tree 40 250000 iterate all - get(i) 64.198 ms/op 75.761 ms/op 0.85
MutableVector 250000 create 18.629 ms/op 13.586 ms/op 1.37
MutableVector 250000 get(125000) 6.6330 ns/op 7.2280 ns/op 0.92
MutableVector 250000 set(125000) 284.19 ns/op 271.53 ns/op 1.05
MutableVector 250000 toArray() 3.4401 ms/op 3.9413 ms/op 0.87
MutableVector 250000 iterate all - toArray() + loop 4.3551 ms/op 4.3430 ms/op 1.00
MutableVector 250000 iterate all - get(i) 1.5365 ms/op 1.5555 ms/op 0.99
Array 250000 create 3.3161 ms/op 2.9848 ms/op 1.11
Array 250000 clone - spread 1.2133 ms/op 1.2328 ms/op 0.98
Array 250000 get(125000) 0.98800 ns/op 1.0260 ns/op 0.96
Array 250000 set(125000) 3.9180 ns/op 4.1170 ns/op 0.95
Array 250000 iterate all - loop 157.96 us/op 163.89 us/op 0.96
effectiveBalanceIncrements clone Uint8Array 300000 26.754 us/op 28.299 us/op 0.95
effectiveBalanceIncrements clone MutableVector 300000 347.00 ns/op 336.00 ns/op 1.03
effectiveBalanceIncrements rw all Uint8Array 300000 190.73 us/op 205.25 us/op 0.93
effectiveBalanceIncrements rw all MutableVector 300000 79.483 ms/op 86.822 ms/op 0.92
phase0 afterProcessEpoch - 250000 vs - 7PWei 112.71 ms/op 115.92 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 55.346 ms/op 55.969 ms/op 0.99
altair processEpoch - mainnet_e81889 549.65 ms/op 553.06 ms/op 0.99
mainnet_e81889 - altair beforeProcessEpoch 86.170 ms/op 95.674 ms/op 0.90
mainnet_e81889 - altair processJustificationAndFinalization 23.242 us/op 19.921 us/op 1.17
mainnet_e81889 - altair processInactivityUpdates 6.4833 ms/op 8.9464 ms/op 0.72
mainnet_e81889 - altair processRewardsAndPenalties 58.145 ms/op 68.386 ms/op 0.85
mainnet_e81889 - altair processRegistryUpdates 4.0610 us/op 2.7970 us/op 1.45
mainnet_e81889 - altair processSlashings 439.00 ns/op 503.00 ns/op 0.87
mainnet_e81889 - altair processEth1DataReset 543.00 ns/op 483.00 ns/op 1.12
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.7239 ms/op 1.5747 ms/op 1.09
mainnet_e81889 - altair processSlashingsReset 3.4810 us/op 4.3460 us/op 0.80
mainnet_e81889 - altair processRandaoMixesReset 5.9130 us/op 5.3900 us/op 1.10
mainnet_e81889 - altair processHistoricalRootsUpdate 877.00 ns/op 533.00 ns/op 1.65
mainnet_e81889 - altair processParticipationFlagUpdates 3.6660 us/op 1.6230 us/op 2.26
mainnet_e81889 - altair processSyncCommitteeUpdates 1.2270 us/op 668.00 ns/op 1.84
mainnet_e81889 - altair afterProcessEpoch 129.99 ms/op 121.55 ms/op 1.07
capella processEpoch - mainnet_e217614 2.5369 s/op 1.9085 s/op 1.33
mainnet_e217614 - capella beforeProcessEpoch 664.40 ms/op 443.10 ms/op 1.50
mainnet_e217614 - capella processJustificationAndFinalization 29.285 us/op 14.295 us/op 2.05
mainnet_e217614 - capella processInactivityUpdates 26.473 ms/op 19.034 ms/op 1.39
mainnet_e217614 - capella processRewardsAndPenalties 556.21 ms/op 444.03 ms/op 1.25
mainnet_e217614 - capella processRegistryUpdates 50.543 us/op 21.009 us/op 2.41
mainnet_e217614 - capella processSlashings 1.9100 us/op 449.00 ns/op 4.25
mainnet_e217614 - capella processEth1DataReset 1.7110 us/op 522.00 ns/op 3.28
mainnet_e217614 - capella processEffectiveBalanceUpdates 13.640 ms/op 11.271 ms/op 1.21
mainnet_e217614 - capella processSlashingsReset 10.262 us/op 3.6170 us/op 2.84
mainnet_e217614 - capella processRandaoMixesReset 13.103 us/op 5.2970 us/op 2.47
mainnet_e217614 - capella processHistoricalRootsUpdate 2.0200 us/op 411.00 ns/op 4.91
mainnet_e217614 - capella processParticipationFlagUpdates 4.8310 us/op 1.4010 us/op 3.45
mainnet_e217614 - capella afterProcessEpoch 454.48 ms/op 305.36 ms/op 1.49
phase0 processEpoch - mainnet_e58758 784.16 ms/op 475.20 ms/op 1.65
mainnet_e58758 - phase0 beforeProcessEpoch 186.61 ms/op 134.29 ms/op 1.39
mainnet_e58758 - phase0 processJustificationAndFinalization 26.473 us/op 15.148 us/op 1.75
mainnet_e58758 - phase0 processRewardsAndPenalties 62.356 ms/op 35.914 ms/op 1.74
mainnet_e58758 - phase0 processRegistryUpdates 14.996 us/op 11.882 us/op 1.26
mainnet_e58758 - phase0 processSlashings 589.00 ns/op 783.00 ns/op 0.75
mainnet_e58758 - phase0 processEth1DataReset 496.00 ns/op 443.00 ns/op 1.12
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2350 ms/op 1.1262 ms/op 1.98
mainnet_e58758 - phase0 processSlashingsReset 3.1420 us/op 3.4670 us/op 0.91
mainnet_e58758 - phase0 processRandaoMixesReset 4.5600 us/op 5.4630 us/op 0.83
mainnet_e58758 - phase0 processHistoricalRootsUpdate 490.00 ns/op 613.00 ns/op 0.80
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.8570 us/op 4.5390 us/op 0.85
mainnet_e58758 - phase0 afterProcessEpoch 104.70 ms/op 101.76 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3189 ms/op 1.3725 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4643 ms/op 1.5930 ms/op 0.92
altair processInactivityUpdates - 250000 normalcase 30.172 ms/op 30.061 ms/op 1.00
altair processInactivityUpdates - 250000 worstcase 29.926 ms/op 30.052 ms/op 1.00
phase0 processRegistryUpdates - 250000 normalcase 13.066 us/op 11.354 us/op 1.15
phase0 processRegistryUpdates - 250000 badcase_full_deposits 551.87 us/op 422.17 us/op 1.31
phase0 processRegistryUpdates - 250000 worstcase 0.5 178.07 ms/op 159.78 ms/op 1.11
altair processRewardsAndPenalties - 250000 normalcase 62.865 ms/op 62.680 ms/op 1.00
altair processRewardsAndPenalties - 250000 worstcase 59.387 ms/op 59.625 ms/op 1.00
phase0 getAttestationDeltas - 250000 normalcase 11.618 ms/op 10.651 ms/op 1.09
phase0 getAttestationDeltas - 250000 worstcase 12.639 ms/op 9.8365 ms/op 1.28
phase0 processSlashings - 250000 worstcase 138.84 us/op 107.29 us/op 1.29
altair processSyncCommitteeUpdates - 250000 168.86 ms/op 159.21 ms/op 1.06
BeaconState.hashTreeRoot - No change 792.00 ns/op 428.00 ns/op 1.85
BeaconState.hashTreeRoot - 1 full validator 152.69 us/op 152.72 us/op 1.00
BeaconState.hashTreeRoot - 32 full validator 1.6623 ms/op 1.6353 ms/op 1.02
BeaconState.hashTreeRoot - 512 full validator 21.052 ms/op 15.240 ms/op 1.38
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 202.75 us/op 156.21 us/op 1.30
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.6500 ms/op 2.1467 ms/op 1.23
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 32.432 ms/op 28.272 ms/op 1.15
BeaconState.hashTreeRoot - 1 balances 143.85 us/op 125.43 us/op 1.15
BeaconState.hashTreeRoot - 32 balances 1.5145 ms/op 1.1179 ms/op 1.35
BeaconState.hashTreeRoot - 512 balances 15.301 ms/op 11.701 ms/op 1.31
BeaconState.hashTreeRoot - 250000 balances 286.90 ms/op 204.35 ms/op 1.40
aggregationBits - 2048 els - zipIndexesInBitList 46.101 us/op 27.071 us/op 1.70
byteArrayEquals 32 84.377 ns/op 74.898 ns/op 1.13
Buffer.compare 32 62.855 ns/op 56.015 ns/op 1.12
byteArrayEquals 1024 2.4932 us/op 2.0429 us/op 1.22
Buffer.compare 1024 90.957 ns/op 71.668 ns/op 1.27
byteArrayEquals 16384 37.262 us/op 32.521 us/op 1.15
Buffer.compare 16384 281.83 ns/op 258.93 ns/op 1.09
byteArrayEquals 123687377 272.92 ms/op 242.01 ms/op 1.13
Buffer.compare 123687377 9.4838 ms/op 6.7252 ms/op 1.41
byteArrayEquals 32 - diff last byte 109.98 ns/op 69.801 ns/op 1.58
Buffer.compare 32 - diff last byte 70.567 ns/op 54.403 ns/op 1.30
byteArrayEquals 1024 - diff last byte 2.2906 us/op 1.9751 us/op 1.16
Buffer.compare 1024 - diff last byte 82.211 ns/op 69.725 ns/op 1.18
byteArrayEquals 16384 - diff last byte 37.470 us/op 32.378 us/op 1.16
Buffer.compare 16384 - diff last byte 294.73 ns/op 282.57 ns/op 1.04
byteArrayEquals 123687377 - diff last byte 279.81 ms/op 241.36 ms/op 1.16
Buffer.compare 123687377 - diff last byte 13.182 ms/op 6.1268 ms/op 2.15
byteArrayEquals 32 - random bytes 14.592 ns/op 5.0380 ns/op 2.90
Buffer.compare 32 - random bytes 72.463 ns/op 58.905 ns/op 1.23
byteArrayEquals 1024 - random bytes 7.4150 ns/op 5.0110 ns/op 1.48
Buffer.compare 1024 - random bytes 75.589 ns/op 57.802 ns/op 1.31
byteArrayEquals 16384 - random bytes 8.2410 ns/op 4.9590 ns/op 1.66
Buffer.compare 16384 - random bytes 73.279 ns/op 58.934 ns/op 1.24
byteArrayEquals 123687377 - random bytes 17.710 ns/op 8.1400 ns/op 2.18
Buffer.compare 123687377 - random bytes 83.800 ns/op 63.990 ns/op 1.31
regular array get 100000 times 57.869 us/op 43.700 us/op 1.32
wrappedArray get 100000 times 57.683 us/op 44.102 us/op 1.31
arrayWithProxy get 100000 times 15.989 ms/op 13.636 ms/op 1.17
ssz.Root.equals 60.094 ns/op 52.974 ns/op 1.13
byteArrayEquals 62.277 ns/op 52.530 ns/op 1.19
Buffer.compare 13.404 ns/op 10.772 ns/op 1.24
shuffle list - 16384 els 9.8062 ms/op 8.4839 ms/op 1.16
shuffle list - 250000 els 148.69 ms/op 124.26 ms/op 1.20
processSlot - 1 slots 26.041 us/op 16.650 us/op 1.56
processSlot - 32 slots 6.2678 ms/op 2.8626 ms/op 2.19
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 71.417 ms/op 61.049 ms/op 1.17
getCommitteeAssignments - req 1 vs - 250000 vc 3.1643 ms/op 2.6546 ms/op 1.19
getCommitteeAssignments - req 100 vs - 250000 vc 4.4727 ms/op 3.8372 ms/op 1.17
getCommitteeAssignments - req 1000 vs - 250000 vc 4.7535 ms/op 4.1981 ms/op 1.13
findModifiedValidators - 10000 modified validators 615.12 ms/op 525.31 ms/op 1.17
findModifiedValidators - 1000 modified validators 514.10 ms/op 421.12 ms/op 1.22
findModifiedValidators - 100 modified validators 472.88 ms/op 406.26 ms/op 1.16
findModifiedValidators - 10 modified validators 466.44 ms/op 378.59 ms/op 1.23
findModifiedValidators - 1 modified validators 474.12 ms/op 400.56 ms/op 1.18
findModifiedValidators - no difference 440.28 ms/op 407.97 ms/op 1.08
compare ViewDUs 5.1615 s/op 4.3086 s/op 1.20
compare each validator Uint8Array 1.6007 s/op 1.6250 s/op 0.99
compare ViewDU to Uint8Array 1.1255 s/op 1.1095 s/op 1.01
migrate state 1000000 validators, 24 modified, 0 new 910.49 ms/op 807.81 ms/op 1.13
migrate state 1000000 validators, 1700 modified, 1000 new 1.2388 s/op 1.0935 s/op 1.13
migrate state 1000000 validators, 3400 modified, 2000 new 1.3607 s/op 1.3249 s/op 1.03
migrate state 1500000 validators, 24 modified, 0 new 888.32 ms/op 737.66 ms/op 1.20
migrate state 1500000 validators, 1700 modified, 1000 new 1.2139 s/op 1.0808 s/op 1.12
migrate state 1500000 validators, 3400 modified, 2000 new 1.5503 s/op 1.4134 s/op 1.10
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.5900 ns/op 4.7100 ns/op 1.19
state getBlockRootAtSlot - 250000 vs - 7PWei 772.18 ns/op 907.19 ns/op 0.85
computeProposers - vc 250000 11.087 ms/op 9.4320 ms/op 1.18
computeEpochShuffling - vc 250000 139.62 ms/op 134.01 ms/op 1.04
getNextSyncCommittee - vc 250000 189.45 ms/op 172.70 ms/op 1.10
computeSigningRoot for AttestationData 39.150 us/op 31.434 us/op 1.25
hash AttestationData serialized data then Buffer.toString(base64) 2.6950 us/op 2.4171 us/op 1.12
toHexString serialized data 1.7689 us/op 1.4509 us/op 1.22
Buffer.toString(base64) 322.67 ns/op 258.79 ns/op 1.25

by benchmarkbot/action

@nazarhussain nazarhussain merged commit 899f3b4 into unstable Mar 13, 2024
20 checks passed
@nazarhussain nazarhussain deleted the nh/sim-geth-upgrade branch March 13, 2024 12:46
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

@jeluard jeluard jeluard left review comments

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@nazarhussain @wemeetagain @jeluard @nflaig