Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: use Uint32Array for shuffling/committees #6475

Merged
merged 3 commits into from Mar 7, 2024
Merged

feat: use Uint32Array for shuffling/committees #6475

merged 3 commits into from Mar 7, 2024

Conversation

wemeetagain
Copy link
Member

Motivation

Looking at heap snapshots of a running lodestar node, the memory usage of shufflings stood out as larger than necessary.

Description

Replace number[] with Uint32Array for activeIndices, shuffling, and committees in the EpochShuffling.
This has two benefits: tighter packing of the data (TypedArray vs Object) AND data sharing between shuffling and committees.
The tradeoff is that only indices < 2 ^ 32 (4.2 billion) can be represented here, rather than indices < 2 ^ 53. In practice this is plenty and not an issue.

@wemeetagain wemeetagain requested a review from a team as a code owner February 22, 2024 22:52
@codecov Codecov
Copy link

codecov bot commented Feb 22, 2024 •
edited

Codecov Report

Merging #6475 (827ec62) into unstable (8943ab6) will increase coverage by 0.00%.
Report is 4 commits behind head on unstable.
The diff coverage is 72.72%.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #6475   +/-   ##
=========================================
  Coverage     61.72%   61.72%           
=========================================
  Files           555      555           
  Lines         58204    58205    +1     
  Branches       1839     1839           
=========================================
+ Hits          35925    35926    +1     
  Misses        22240    22240           
  Partials         39       39

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 22, 2024 •
edited

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: be4fc64 Previous: e9a3f07 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 700.84 us/op 909.23 us/op 0.77
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 114.84 us/op 176.03 us/op 0.65
BLS verify - blst-native 1.3555 ms/op 1.4517 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 2.8447 ms/op 3.1202 ms/op 0.91
BLS verifyMultipleSignatures 8 - blst-native 6.2644 ms/op 6.4448 ms/op 0.97
BLS verifyMultipleSignatures 32 - blst-native 22.968 ms/op 23.893 ms/op 0.96
BLS verifyMultipleSignatures 64 - blst-native 44.985 ms/op 46.184 ms/op 0.97
BLS verifyMultipleSignatures 128 - blst-native 89.208 ms/op 92.869 ms/op 0.96
BLS deserializing 10000 signatures 909.85 ms/op 965.83 ms/op 0.94
BLS deserializing 100000 signatures 9.6155 s/op 9.5495 s/op 1.01
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3770 ms/op 1.4762 ms/op 0.93
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6012 ms/op 1.6473 ms/op 0.97
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4489 ms/op 2.5651 ms/op 0.95
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.5368 ms/op 3.6115 ms/op 0.98
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.0972 ms/op 6.1982 ms/op 0.98
BLS aggregatePubkeys 32 - blst-native 28.014 us/op 28.007 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 104.53 us/op 110.23 us/op 0.95
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 57.914 ms/op 64.292 ms/op 0.90
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 66.102 ms/op 66.163 ms/op 1.00
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 46.534 ms/op 51.816 ms/op 0.90
getSlashingsAndExits - default max 205.35 us/op 277.56 us/op 0.74
getSlashingsAndExits - 2k 588.16 us/op 719.97 us/op 0.82
proposeBlockBody type=full, size=empty 6.6918 ms/op 6.6901 ms/op 1.00
isKnown best case - 1 super set check 638.00 ns/op 608.00 ns/op 1.05
isKnown normal case - 2 super set checks 456.00 ns/op 647.00 ns/op 0.70
isKnown worse case - 16 super set checks 362.00 ns/op 641.00 ns/op 0.56
CheckpointStateCache - add get delete 5.8160 us/op 6.6130 us/op 0.88
validate api signedAggregateAndProof - struct 2.8644 ms/op 3.0467 ms/op 0.94
validate gossip signedAggregateAndProof - struct 2.8599 ms/op 3.0523 ms/op 0.94
validate gossip attestation - vc 640000 1.3942 ms/op 1.4803 ms/op 0.94
batch validate gossip attestation - vc 640000 - chunk 32 164.42 us/op 190.77 us/op 0.86
batch validate gossip attestation - vc 640000 - chunk 64 150.01 us/op 167.52 us/op 0.90
batch validate gossip attestation - vc 640000 - chunk 128 143.31 us/op 166.57 us/op 0.86
batch validate gossip attestation - vc 640000 - chunk 256 139.52 us/op 150.91 us/op 0.92
pickEth1Vote - no votes 1.5083 ms/op 1.3762 ms/op 1.10
pickEth1Vote - max votes 11.713 ms/op 12.587 ms/op 0.93
pickEth1Vote - Eth1Data hashTreeRoot value x2048 19.597 ms/op 22.743 ms/op 0.86
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 24.504 ms/op 29.622 ms/op 0.83
pickEth1Vote - Eth1Data fastSerialize value x2048 745.02 us/op 791.90 us/op 0.94
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.3538 ms/op 5.9118 ms/op 0.91
bytes32 toHexString 634.00 ns/op 749.00 ns/op 0.85
bytes32 Buffer.toString(hex) 317.00 ns/op 348.00 ns/op 0.91
bytes32 Buffer.toString(hex) from Uint8Array 520.00 ns/op 578.00 ns/op 0.90
bytes32 Buffer.toString(hex) + 0x 307.00 ns/op 319.00 ns/op 0.96
Object access 1 prop 0.20600 ns/op 0.24000 ns/op 0.86
Map access 1 prop 0.17200 ns/op 0.15700 ns/op 1.10
Object get x1000 7.6200 ns/op 7.7110 ns/op 0.99
Map get x1000 0.86700 ns/op 0.88600 ns/op 0.98
Object set x1000 68.228 ns/op 60.314 ns/op 1.13
Map set x1000 50.028 ns/op 49.842 ns/op 1.00
Return object 10000 times 0.25360 ns/op 0.25410 ns/op 1.00
Throw Error 10000 times 4.0376 us/op 4.1639 us/op 0.97
fastMsgIdFn sha256 / 200 bytes 3.4520 us/op 3.5780 us/op 0.96
fastMsgIdFn h32 xxhash / 200 bytes 322.00 ns/op 374.00 ns/op 0.86
fastMsgIdFn h64 xxhash / 200 bytes 386.00 ns/op 425.00 ns/op 0.91
fastMsgIdFn sha256 / 1000 bytes 11.892 us/op 12.372 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 469.00 ns/op 502.00 ns/op 0.93
fastMsgIdFn h64 xxhash / 1000 bytes 453.00 ns/op 495.00 ns/op 0.92
fastMsgIdFn sha256 / 10000 bytes 106.89 us/op 107.18 us/op 1.00
fastMsgIdFn h32 xxhash / 10000 bytes 2.0850 us/op 2.1650 us/op 0.96
fastMsgIdFn h64 xxhash / 10000 bytes 1.4350 us/op 1.5140 us/op 0.95
send data - 1000 256B messages 20.976 ms/op 23.393 ms/op 0.90
send data - 1000 512B messages 29.575 ms/op 33.827 ms/op 0.87
send data - 1000 1024B messages 47.838 ms/op 48.911 ms/op 0.98
send data - 1000 1200B messages 44.876 ms/op 49.969 ms/op 0.90
send data - 1000 2048B messages 52.399 ms/op 60.775 ms/op 0.86
send data - 1000 4096B messages 52.701 ms/op 54.334 ms/op 0.97
send data - 1000 16384B messages 139.56 ms/op 134.98 ms/op 1.03
send data - 1000 65536B messages 535.40 ms/op 499.19 ms/op 1.07
enrSubnets - fastDeserialize 64 bits 1.6830 us/op 1.5350 us/op 1.10
enrSubnets - ssz BitVector 64 bits 541.00 ns/op 554.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 237.00 ns/op 222.00 ns/op 1.07
enrSubnets - ssz BitVector 4 bits 558.00 ns/op 521.00 ns/op 1.07
prioritizePeers score -10:0 att 32-0.1 sync 2-0 125.86 us/op 126.25 us/op 1.00
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 155.38 us/op 150.42 us/op 1.03
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 200.53 us/op 213.34 us/op 0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 349.75 us/op 365.44 us/op 0.96
prioritizePeers score 0:0 att 64-1 sync 4-1 383.59 us/op 419.65 us/op 0.91
array of 16000 items push then shift 1.7759 us/op 1.8476 us/op 0.96
LinkedList of 16000 items push then shift 9.7130 ns/op 9.9920 ns/op 0.97
array of 16000 items push then pop 116.52 ns/op 119.53 ns/op 0.97
LinkedList of 16000 items push then pop 9.5220 ns/op 9.5560 ns/op 1.00
array of 24000 items push then shift 2.7481 us/op 2.7184 us/op 1.01
LinkedList of 24000 items push then shift 9.9240 ns/op 9.5410 ns/op 1.04
array of 24000 items push then pop 160.49 ns/op 158.79 ns/op 1.01
LinkedList of 24000 items push then pop 9.1870 ns/op 9.1590 ns/op 1.00
intersect bitArray bitLen 8 6.3480 ns/op 6.1740 ns/op 1.03
intersect array and set length 8 87.202 ns/op 85.586 ns/op 1.02
intersect bitArray bitLen 128 38.961 ns/op 37.211 ns/op 1.05
intersect array and set length 128 1.1325 us/op 1.1068 us/op 1.02
bitArray.getTrueBitIndexes() bitLen 128 1.7510 us/op 1.6530 us/op 1.06
bitArray.getTrueBitIndexes() bitLen 248 3.0810 us/op 3.1290 us/op 0.98
bitArray.getTrueBitIndexes() bitLen 512 6.5880 us/op 6.5890 us/op 1.00
Buffer.concat 32 items 1.0960 us/op 1.1300 us/op 0.97
Uint8Array.set 32 items 2.0410 us/op 2.1200 us/op 0.96
Set add up to 64 items then delete first 5.3780 us/op 5.1507 us/op 1.04
OrderedSet add up to 64 items then delete first 6.9170 us/op 6.4288 us/op 1.08
Set add up to 64 items then delete last 5.5793 us/op 5.4642 us/op 1.02
OrderedSet add up to 64 items then delete last 7.1394 us/op 7.0399 us/op 1.01
Set add up to 64 items then delete middle 5.5724 us/op 5.5564 us/op 1.00
OrderedSet add up to 64 items then delete middle 9.0841 us/op 8.3533 us/op 1.09
Set add up to 128 items then delete first 11.308 us/op 10.679 us/op 1.06
OrderedSet add up to 128 items then delete first 15.447 us/op 15.074 us/op 1.02
Set add up to 128 items then delete last 11.034 us/op 9.9780 us/op 1.11
OrderedSet add up to 128 items then delete last 14.652 us/op 12.264 us/op 1.19
Set add up to 128 items then delete middle 11.185 us/op 10.612 us/op 1.05
OrderedSet add up to 128 items then delete middle 20.700 us/op 18.836 us/op 1.10
Set add up to 256 items then delete first 22.734 us/op 21.874 us/op 1.04
OrderedSet add up to 256 items then delete first 32.162 us/op 31.049 us/op 1.04
Set add up to 256 items then delete last 22.076 us/op 21.717 us/op 1.02
OrderedSet add up to 256 items then delete last 28.629 us/op 29.498 us/op 0.97
Set add up to 256 items then delete middle 21.423 us/op 21.927 us/op 0.98
OrderedSet add up to 256 items then delete middle 56.082 us/op 54.350 us/op 1.03
transfer serialized Status (84 B) 1.9200 us/op 2.0850 us/op 0.92
copy serialized Status (84 B) 1.4290 us/op 1.5710 us/op 0.91
transfer serialized SignedVoluntaryExit (112 B) 2.0190 us/op 2.1800 us/op 0.93
copy serialized SignedVoluntaryExit (112 B) 1.4750 us/op 1.6280 us/op 0.91
transfer serialized ProposerSlashing (416 B) 2.2630 us/op 2.9100 us/op 0.78
copy serialized ProposerSlashing (416 B) 2.0630 us/op 2.3870 us/op 0.86
transfer serialized Attestation (485 B) 2.3510 us/op 3.0320 us/op 0.78
copy serialized Attestation (485 B) 2.0450 us/op 2.7700 us/op 0.74
transfer serialized AttesterSlashing (33232 B) 2.2080 us/op 2.6820 us/op 0.82
copy serialized AttesterSlashing (33232 B) 7.2310 us/op 10.100 us/op 0.72
transfer serialized Small SignedBeaconBlock (128000 B) 2.8610 us/op 3.3350 us/op 0.86
copy serialized Small SignedBeaconBlock (128000 B) 18.925 us/op 35.557 us/op 0.53
transfer serialized Avg SignedBeaconBlock (200000 B) 3.0350 us/op 3.9050 us/op 0.78
copy serialized Avg SignedBeaconBlock (200000 B) 32.757 us/op 56.749 us/op 0.58
transfer serialized BlobsSidecar (524380 B) 3.5100 us/op 5.2060 us/op 0.67
copy serialized BlobsSidecar (524380 B) 103.49 us/op 157.04 us/op 0.66
transfer serialized Big SignedBeaconBlock (1000000 B) 4.0350 us/op 5.6050 us/op 0.72
copy serialized Big SignedBeaconBlock (1000000 B) 229.08 us/op 266.79 us/op 0.86
pass gossip attestations to forkchoice per slot 4.9175 ms/op 4.4621 ms/op 1.10
forkChoice updateHead vc 100000 bc 64 eq 0 740.99 us/op 857.92 us/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 0 5.6565 ms/op 6.4958 ms/op 0.87
forkChoice updateHead vc 1000000 bc 64 eq 0 8.2432 ms/op 9.9555 ms/op 0.83
forkChoice updateHead vc 600000 bc 320 eq 0 4.5825 ms/op 5.5103 ms/op 0.83
forkChoice updateHead vc 600000 bc 1200 eq 0 4.7089 ms/op 4.9572 ms/op 0.95
forkChoice updateHead vc 600000 bc 7200 eq 0 6.0847 ms/op 7.0997 ms/op 0.86
forkChoice updateHead vc 600000 bc 64 eq 1000 12.360 ms/op 12.016 ms/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 10000 13.040 ms/op 12.652 ms/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 300000 18.391 ms/op 19.217 ms/op 0.96
computeDeltas 500000 validators 300 proto nodes 6.9877 ms/op 7.6705 ms/op 0.91
computeDeltas 500000 validators 1200 proto nodes 6.8255 ms/op 7.7129 ms/op 0.88
computeDeltas 500000 validators 7200 proto nodes 6.6841 ms/op 7.7964 ms/op 0.86
computeDeltas 750000 validators 300 proto nodes 11.914 ms/op 11.200 ms/op 1.06
computeDeltas 750000 validators 1200 proto nodes 9.8009 ms/op 10.992 ms/op 0.89
computeDeltas 750000 validators 7200 proto nodes 10.032 ms/op 11.321 ms/op 0.89
computeDeltas 1400000 validators 300 proto nodes 18.920 ms/op 21.191 ms/op 0.89
computeDeltas 1400000 validators 1200 proto nodes 19.016 ms/op 23.714 ms/op 0.80
computeDeltas 1400000 validators 7200 proto nodes 19.439 ms/op 21.121 ms/op 0.92
computeDeltas 2100000 validators 300 proto nodes 28.140 ms/op 33.867 ms/op 0.83
computeDeltas 2100000 validators 1200 proto nodes 27.266 ms/op 32.824 ms/op 0.83
computeDeltas 2100000 validators 7200 proto nodes 27.863 ms/op 32.033 ms/op 0.87
altair processAttestation - 250000 vs - 7PWei normalcase 2.1450 ms/op 3.5696 ms/op 0.60
altair processAttestation - 250000 vs - 7PWei worstcase 3.3354 ms/op 4.6604 ms/op 0.72
altair processAttestation - setStatus - 1/6 committees join 195.02 us/op 197.61 us/op 0.99
altair processAttestation - setStatus - 1/3 committees join 369.99 us/op 432.89 us/op 0.85
altair processAttestation - setStatus - 1/2 committees join 486.90 us/op 590.04 us/op 0.83
altair processAttestation - setStatus - 2/3 committees join 621.36 us/op 679.38 us/op 0.91
altair processAttestation - setStatus - 4/5 committees join 829.82 us/op 917.64 us/op 0.90
altair processAttestation - setStatus - 100% committees join 980.92 us/op 1.1655 ms/op 0.84
altair processBlock - 250000 vs - 7PWei normalcase 10.512 ms/op 11.978 ms/op 0.88
altair processBlock - 250000 vs - 7PWei normalcase hashState 36.102 ms/op 40.631 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase 38.352 ms/op 51.440 ms/op 0.75
altair processBlock - 250000 vs - 7PWei worstcase hashState 87.261 ms/op 123.08 ms/op 0.71
phase0 processBlock - 250000 vs - 7PWei normalcase 2.4219 ms/op 3.5723 ms/op 0.68
phase0 processBlock - 250000 vs - 7PWei worstcase 38.463 ms/op 41.535 ms/op 0.93
altair processEth1Data - 250000 vs - 7PWei normalcase 565.71 us/op 1.0308 ms/op 0.55
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 8.8070 us/op 18.811 us/op 0.47
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 36.760 us/op 72.328 us/op 0.51
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 18.869 us/op 31.929 us/op 0.59
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 16.024 us/op 17.091 us/op 0.94
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 219.81 us/op 305.01 us/op 0.72
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.5834 ms/op 1.9383 ms/op 0.82
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.5670 ms/op 2.5102 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8394 ms/op 2.1626 ms/op 0.85
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 5.1161 ms/op 4.8736 ms/op 1.05
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 3.1799 ms/op 3.4106 ms/op 0.93
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 7.2594 ms/op 8.4965 ms/op 0.85
Tree 40 250000 create 660.21 ms/op 825.72 ms/op 0.80
Tree 40 250000 get(125000) 227.24 ns/op 237.56 ns/op 0.96
Tree 40 250000 set(125000) 1.1407 us/op 2.6310 us/op 0.43
Tree 40 250000 toArray() 23.594 ms/op 27.695 ms/op 0.85
Tree 40 250000 iterate all - toArray() + loop 24.485 ms/op 28.573 ms/op 0.86
Tree 40 250000 iterate all - get(i) 76.698 ms/op 85.780 ms/op 0.89
MutableVector 250000 create 15.262 ms/op 20.596 ms/op 0.74
MutableVector 250000 get(125000) 6.8730 ns/op 7.1270 ns/op 0.96
MutableVector 250000 set(125000) 459.94 ns/op 481.30 ns/op 0.96
MutableVector 250000 toArray() 4.0806 ms/op 4.5961 ms/op 0.89
MutableVector 250000 iterate all - toArray() + loop 4.4505 ms/op 4.9020 ms/op 0.91
MutableVector 250000 iterate all - get(i) 1.6273 ms/op 1.7877 ms/op 0.91
Array 250000 create 4.2585 ms/op 4.2794 ms/op 1.00
Array 250000 clone - spread 1.4304 ms/op 1.9074 ms/op 0.75
Array 250000 get(125000) 1.4510 ns/op 2.5730 ns/op 0.56
Array 250000 set(125000) 4.6680 ns/op 6.1770 ns/op 0.76
Array 250000 iterate all - loop 207.22 us/op 194.58 us/op 1.06
effectiveBalanceIncrements clone Uint8Array 300000 49.465 us/op 77.572 us/op 0.64
effectiveBalanceIncrements clone MutableVector 300000 395.00 ns/op 500.00 ns/op 0.79
effectiveBalanceIncrements rw all Uint8Array 300000 209.43 us/op 226.13 us/op 0.93
effectiveBalanceIncrements rw all MutableVector 300000 107.10 ms/op 215.42 ms/op 0.50
phase0 afterProcessEpoch - 250000 vs - 7PWei 122.22 ms/op 135.44 ms/op 0.90
phase0 beforeProcessEpoch - 250000 vs - 7PWei 56.439 ms/op 70.349 ms/op 0.80
altair processEpoch - mainnet_e81889 567.22 ms/op 656.01 ms/op 0.86
mainnet_e81889 - altair beforeProcessEpoch 106.36 ms/op 118.40 ms/op 0.90
mainnet_e81889 - altair processJustificationAndFinalization 19.306 us/op 31.417 us/op 0.61
mainnet_e81889 - altair processInactivityUpdates 8.4344 ms/op 10.094 ms/op 0.84
mainnet_e81889 - altair processRewardsAndPenalties 74.916 ms/op 87.516 ms/op 0.86
mainnet_e81889 - altair processRegistryUpdates 3.0880 us/op 5.6340 us/op 0.55
mainnet_e81889 - altair processSlashings 741.00 ns/op 1.4240 us/op 0.52
mainnet_e81889 - altair processEth1DataReset 1.5020 us/op 1.1130 us/op 1.35
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4485 ms/op 2.1467 ms/op 0.67
mainnet_e81889 - altair processSlashingsReset 5.1680 us/op 7.8000 us/op 0.66
mainnet_e81889 - altair processRandaoMixesReset 5.9670 us/op 10.038 us/op 0.59
mainnet_e81889 - altair processHistoricalRootsUpdate 780.00 ns/op 2.7170 us/op 0.29
mainnet_e81889 - altair processParticipationFlagUpdates 2.8580 us/op 4.2580 us/op 0.67
mainnet_e81889 - altair processSyncCommitteeUpdates 950.00 ns/op 1.5170 us/op 0.63
mainnet_e81889 - altair afterProcessEpoch 127.91 ms/op 149.70 ms/op 0.85
capella processEpoch - mainnet_e217614 2.3484 s/op 2.8465 s/op 0.83
mainnet_e217614 - capella beforeProcessEpoch 679.01 ms/op 594.95 ms/op 1.14
mainnet_e217614 - capella processJustificationAndFinalization 26.199 us/op 22.023 us/op 1.19
mainnet_e217614 - capella processInactivityUpdates 31.606 ms/op 26.520 ms/op 1.19
mainnet_e217614 - capella processRewardsAndPenalties 548.23 ms/op 499.88 ms/op 1.10
mainnet_e217614 - capella processRegistryUpdates 22.574 us/op 39.952 us/op 0.57
mainnet_e217614 - capella processSlashings 1.2150 us/op 1.5590 us/op 0.78
mainnet_e217614 - capella processEth1DataReset 626.00 ns/op 849.00 ns/op 0.74
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.9791 ms/op 6.5691 ms/op 0.76
mainnet_e217614 - capella processSlashingsReset 4.8650 us/op 6.5560 us/op 0.74
mainnet_e217614 - capella processRandaoMixesReset 5.8390 us/op 9.9950 us/op 0.58
mainnet_e217614 - capella processHistoricalRootsUpdate 1.0190 us/op 1.5580 us/op 0.65
mainnet_e217614 - capella processParticipationFlagUpdates 3.1830 us/op 3.5570 us/op 0.89
mainnet_e217614 - capella afterProcessEpoch 376.43 ms/op 346.44 ms/op 1.09
phase0 processEpoch - mainnet_e58758 641.79 ms/op 589.26 ms/op 1.09
mainnet_e58758 - phase0 beforeProcessEpoch 220.89 ms/op 194.03 ms/op 1.14
mainnet_e58758 - phase0 processJustificationAndFinalization 30.528 us/op 29.341 us/op 1.04
mainnet_e58758 - phase0 processRewardsAndPenalties 64.956 ms/op 50.575 ms/op 1.28
mainnet_e58758 - phase0 processRegistryUpdates 21.338 us/op 20.884 us/op 1.02
mainnet_e58758 - phase0 processSlashings 1.1170 us/op 937.00 ns/op 1.19
mainnet_e58758 - phase0 processEth1DataReset 1.3250 us/op 1.0290 us/op 1.29
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2326 ms/op 1.6909 ms/op 1.32
mainnet_e58758 - phase0 processSlashingsReset 6.2650 us/op 5.1990 us/op 1.21
mainnet_e58758 - phase0 processRandaoMixesReset 7.8020 us/op 9.1280 us/op 0.85
mainnet_e58758 - phase0 processHistoricalRootsUpdate 812.00 ns/op 1.2100 us/op 0.67
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.4240 us/op 10.924 us/op 0.59
mainnet_e58758 - phase0 afterProcessEpoch 113.01 ms/op 111.41 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6527 ms/op 1.8743 ms/op 1.42
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.9329 ms/op 1.7940 ms/op 1.63
altair processInactivityUpdates - 250000 normalcase 49.288 ms/op 38.670 ms/op 1.27
altair processInactivityUpdates - 250000 worstcase 46.911 ms/op 36.643 ms/op 1.28
phase0 processRegistryUpdates - 250000 normalcase 22.497 us/op 14.716 us/op 1.53
phase0 processRegistryUpdates - 250000 badcase_full_deposits 619.71 us/op 667.65 us/op 0.93
phase0 processRegistryUpdates - 250000 worstcase 0.5 207.00 ms/op 215.66 ms/op 0.96
altair processRewardsAndPenalties - 250000 normalcase 75.179 ms/op 75.289 ms/op 1.00
altair processRewardsAndPenalties - 250000 worstcase 75.180 ms/op 89.431 ms/op 0.84
phase0 getAttestationDeltas - 250000 normalcase 14.681 ms/op 13.079 ms/op 1.12
phase0 getAttestationDeltas - 250000 worstcase 14.464 ms/op 12.186 ms/op 1.19
phase0 processSlashings - 250000 worstcase 148.72 us/op 112.16 us/op 1.33
altair processSyncCommitteeUpdates - 250000 186.05 ms/op 190.95 ms/op 0.97
BeaconState.hashTreeRoot - No change 888.00 ns/op 791.00 ns/op 1.12
BeaconState.hashTreeRoot - 1 full validator 169.39 us/op 183.88 us/op 0.92
BeaconState.hashTreeRoot - 32 full validator 1.7954 ms/op 1.8833 ms/op 0.95
BeaconState.hashTreeRoot - 512 full validator 20.183 ms/op 22.440 ms/op 0.90
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 239.57 us/op 215.96 us/op 1.11
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.8443 ms/op 2.9500 ms/op 0.96
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 38.532 ms/op 32.372 ms/op 1.19
BeaconState.hashTreeRoot - 1 balances 167.07 us/op 153.18 us/op 1.09
BeaconState.hashTreeRoot - 32 balances 1.6838 ms/op 1.5809 ms/op 1.07
BeaconState.hashTreeRoot - 512 balances 15.594 ms/op 14.289 ms/op 1.09
BeaconState.hashTreeRoot - 250000 balances 289.21 ms/op 246.21 ms/op 1.17
aggregationBits - 2048 els - zipIndexesInBitList 55.954 us/op 24.722 us/op 2.26
byteArrayEquals 32 94.209 ns/op 83.157 ns/op 1.13
Buffer.compare 32 61.632 ns/op 62.619 ns/op 0.98
byteArrayEquals 1024 2.2211 us/op 2.2774 us/op 0.98
Buffer.compare 1024 78.178 ns/op 78.280 ns/op 1.00
byteArrayEquals 16384 38.932 us/op 38.385 us/op 1.01
Buffer.compare 16384 341.42 ns/op 292.68 ns/op 1.17
byteArrayEquals 123687377 278.92 ms/op 269.92 ms/op 1.03
Buffer.compare 123687377 13.763 ms/op 7.5315 ms/op 1.83
byteArrayEquals 32 - diff last byte 75.600 ns/op 78.045 ns/op 0.97
Buffer.compare 32 - diff last byte 58.220 ns/op 61.319 ns/op 0.95
byteArrayEquals 1024 - diff last byte 2.1559 us/op 2.1967 us/op 0.98
Buffer.compare 1024 - diff last byte 77.913 ns/op 80.672 ns/op 0.97
byteArrayEquals 16384 - diff last byte 34.643 us/op 36.444 us/op 0.95
Buffer.compare 16384 - diff last byte 281.86 ns/op 284.54 ns/op 0.99
byteArrayEquals 123687377 - diff last byte 286.49 ms/op 267.23 ms/op 1.07
Buffer.compare 123687377 - diff last byte 15.848 ms/op 8.5816 ms/op 1.85
byteArrayEquals 32 - random bytes 7.4540 ns/op 6.2170 ns/op 1.20
Buffer.compare 32 - random bytes 67.161 ns/op 67.467 ns/op 1.00
byteArrayEquals 1024 - random bytes 7.5460 ns/op 6.4710 ns/op 1.17
Buffer.compare 1024 - random bytes 66.299 ns/op 69.105 ns/op 0.96
byteArrayEquals 16384 - random bytes 7.2410 ns/op 6.6690 ns/op 1.09
Buffer.compare 16384 - random bytes 64.819 ns/op 71.251 ns/op 0.91
byteArrayEquals 123687377 - random bytes 22.350 ns/op 11.310 ns/op 1.98
Buffer.compare 123687377 - random bytes 84.730 ns/op 83.640 ns/op 1.01
regular array get 100000 times 49.664 us/op 55.872 us/op 0.89
wrappedArray get 100000 times 48.278 us/op 53.157 us/op 0.91
arrayWithProxy get 100000 times 15.467 ms/op 16.169 ms/op 0.96
ssz.Root.equals 65.362 ns/op 60.413 ns/op 1.08
byteArrayEquals 58.452 ns/op 58.454 ns/op 1.00
Buffer.compare 14.222 ns/op 12.766 ns/op 1.11
shuffle list - 16384 els 9.1953 ms/op 7.8010 ms/op 1.18
shuffle list - 250000 els 138.12 ms/op 119.74 ms/op 1.15
processSlot - 1 slots 20.776 us/op 20.431 us/op 1.02
processSlot - 32 slots 3.8408 ms/op 4.2163 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 72.442 ms/op 73.941 ms/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 2.8308 ms/op 2.9873 ms/op 0.95
getCommitteeAssignments - req 100 vs - 250000 vc 4.0920 ms/op 4.3612 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3498 ms/op 4.9161 ms/op 0.88
findModifiedValidators - 10000 modified validators 638.81 ms/op 707.72 ms/op 0.90
findModifiedValidators - 1000 modified validators 524.21 ms/op 581.54 ms/op 0.90
findModifiedValidators - 100 modified validators 487.07 ms/op 527.96 ms/op 0.92
findModifiedValidators - 10 modified validators 445.81 ms/op 524.89 ms/op 0.85
findModifiedValidators - 1 modified validators 503.70 ms/op 492.73 ms/op 1.02
findModifiedValidators - no difference 517.55 ms/op 521.52 ms/op 0.99
compare ViewDUs 5.3716 s/op 5.4488 s/op 0.99
compare each validator Uint8Array 2.1879 s/op 2.2079 s/op 0.99
compare ViewDU to Uint8Array 1.4970 s/op 1.5405 s/op 0.97
migrate state 1000000 validators, 24 modified, 0 new 1.0728 s/op 1.0034 s/op 1.07
migrate state 1000000 validators, 1700 modified, 1000 new 1.3016 s/op 1.5094 s/op 0.86
migrate state 1000000 validators, 3400 modified, 2000 new 1.4996 s/op 1.7061 s/op 0.88
migrate state 1500000 validators, 24 modified, 0 new 961.01 ms/op 945.28 ms/op 1.02
migrate state 1500000 validators, 1700 modified, 1000 new 1.2688 s/op 1.4068 s/op 0.90
migrate state 1500000 validators, 3400 modified, 2000 new 1.8686 s/op 1.5829 s/op 1.18
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.2400 ns/op 5.6000 ns/op 1.11
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3915 us/op 801.24 ns/op 1.74
computeProposers - vc 250000 14.979 ms/op 11.073 ms/op 1.35
computeEpochShuffling - vc 250000 157.89 ms/op 115.78 ms/op 1.36
getNextSyncCommittee - vc 250000 193.97 ms/op 198.24 ms/op 0.98
computeSigningRoot for AttestationData 39.370 us/op 32.855 us/op 1.20
hash AttestationData serialized data then Buffer.toString(base64) 2.8727 us/op 2.5981 us/op 1.11
toHexString serialized data 1.9893 us/op 1.4861 us/op 1.34
Buffer.toString(base64) 291.23 ns/op 310.45 ns/op 0.94

by benchmarkbot/action

@twoeths
Copy link
Contributor

twoeths commented Feb 23, 2024

looks good to me, dropped a minor comment @wemeetagain

nflaig
nflaig approved these changes Mar 5, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

matthewkeil
matthewkeil approved these changes Mar 6, 2024
View reviewed changes
Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet!!! 🎸

@wemeetagain wemeetagain merged commit 36f50cf into unstable Mar 7, 2024
20 checks passed
@wemeetagain wemeetagain deleted the cayman/uint32array-committee branch March 7, 2024 14:56
@wemeetagain
Copy link
Member Author

🎉 This PR is included in v1.17.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

@matthewkeil matthewkeil matthewkeil approved these changes

@nflaig nflaig nflaig approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@wemeetagain @twoeths @matthewkeil @nflaig