Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: temporarily reduce the sync committee participation limit for sim tests #6433

Merged
merged 1 commit into from Feb 16, 2024
Merged

test: temporarily reduce the sync committee participation limit for sim tests #6433

merged 1 commit into from Feb 16, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Make the sim tests run more stable.

Description

Related to #6432, can't easily identified what's wrong in sync committee participation. Until we fix that issue following configuration will help to keep the sim tests pass.

Steps to test or reproduce

  • Run sim tests.

@nazarhussain nazarhussain requested a review from a team as a code owner February 15, 2024 19:02
@nazarhussain nazarhussain self-assigned this Feb 15, 2024
@codecov Codecov
Copy link

codecov bot commented Feb 15, 2024

Codecov Report

Merging #6433 (50fd922) into unstable (48871eb) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files 
@@            Coverage Diff            @@
##           unstable    #6433   +/-   ##
=========================================
  Coverage     61.70%   61.70%           
=========================================
  Files           553      553           
  Lines         57858    57858           
  Branches       1829     1829           
=========================================
  Hits          35702    35702           
  Misses        22119    22119           
  Partials         37       37

@github-actions GitHub Actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: f2681e1 Previous: 48871eb Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 606.66 us/op 758.14 us/op 0.80
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 112.33 us/op 92.295 us/op 1.22
BLS verify - blst-native 1.4116 ms/op 1.1083 ms/op 1.27
BLS verifyMultipleSignatures 3 - blst-native 2.8787 ms/op 2.3561 ms/op 1.22
BLS verifyMultipleSignatures 8 - blst-native 6.3084 ms/op 5.2966 ms/op 1.19
BLS verifyMultipleSignatures 32 - blst-native 23.085 ms/op 19.663 ms/op 1.17
BLS verifyMultipleSignatures 64 - blst-native 45.466 ms/op 39.021 ms/op 1.17
BLS verifyMultipleSignatures 128 - blst-native 94.113 ms/op 76.918 ms/op 1.22
BLS deserializing 10000 signatures 993.61 ms/op 809.54 ms/op 1.23
BLS deserializing 100000 signatures 9.4256 s/op 8.4377 s/op 1.12
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.4577 ms/op 1.1323 ms/op 1.29
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.6356 ms/op 1.3259 ms/op 1.23
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.4235 ms/op 2.6066 ms/op 0.93
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.8132 ms/op 3.0350 ms/op 1.26
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.8807 ms/op 6.4674 ms/op 0.91
BLS aggregatePubkeys 32 - blst-native 27.574 us/op 24.561 us/op 1.12
BLS aggregatePubkeys 128 - blst-native 107.65 us/op 89.737 us/op 1.20
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 68.841 ms/op 43.123 ms/op 1.60
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 63.165 ms/op 42.176 ms/op 1.50
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 59.332 ms/op 27.155 ms/op 2.18
getSlashingsAndExits - default max 163.63 us/op 199.65 us/op 0.82
getSlashingsAndExits - 2k 559.06 us/op 346.45 us/op 1.61
proposeBlockBody type=full, size=empty 6.8179 ms/op 3.9476 ms/op 1.73
isKnown best case - 1 super set check 646.00 ns/op 521.00 ns/op 1.24
isKnown normal case - 2 super set checks 440.00 ns/op 582.00 ns/op 0.76
isKnown worse case - 16 super set checks 565.00 ns/op 442.00 ns/op 1.28
CheckpointStateCache - add get delete 6.6400 us/op 4.7870 us/op 1.39
validate api signedAggregateAndProof - struct 3.0818 ms/op 2.6261 ms/op 1.17
validate gossip signedAggregateAndProof - struct 3.1100 ms/op 2.5193 ms/op 1.23
validate gossip attestation - vc 640000 1.4464 ms/op 1.2475 ms/op 1.16
batch validate gossip attestation - vc 640000 - chunk 32 175.01 us/op 157.84 us/op 1.11
batch validate gossip attestation - vc 640000 - chunk 64 160.83 us/op 139.93 us/op 1.15
batch validate gossip attestation - vc 640000 - chunk 128 158.51 us/op 139.37 us/op 1.14
batch validate gossip attestation - vc 640000 - chunk 256 149.69 us/op 133.15 us/op 1.12
pickEth1Vote - no votes 1.3469 ms/op 1.1784 ms/op 1.14
pickEth1Vote - max votes 11.956 ms/op 11.333 ms/op 1.05
pickEth1Vote - Eth1Data hashTreeRoot value x2048 21.535 ms/op 16.476 ms/op 1.31
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 30.851 ms/op 24.542 ms/op 1.26
pickEth1Vote - Eth1Data fastSerialize value x2048 784.13 us/op 532.01 us/op 1.47
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.3579 ms/op 3.7902 ms/op 1.94
bytes32 toHexString 645.00 ns/op 515.00 ns/op 1.25
bytes32 Buffer.toString(hex) 336.00 ns/op 373.00 ns/op 0.90
bytes32 Buffer.toString(hex) from Uint8Array 582.00 ns/op 597.00 ns/op 0.97
bytes32 Buffer.toString(hex) + 0x 353.00 ns/op 356.00 ns/op 0.99
Object access 1 prop 0.21300 ns/op 0.24400 ns/op 0.87
Map access 1 prop 0.17000 ns/op 0.20500 ns/op 0.83
Object get x1000 8.6720 ns/op 5.3920 ns/op 1.61
Map get x1000 0.93000 ns/op 0.87600 ns/op 1.06
Object set x1000 69.587 ns/op 33.061 ns/op 2.10
Map set x1000 54.616 ns/op 21.104 ns/op 2.59
Return object 10000 times 0.26700 ns/op 0.25150 ns/op 1.06
Throw Error 10000 times 4.3546 us/op 3.1602 us/op 1.38
fastMsgIdFn sha256 / 200 bytes 3.6970 us/op 2.4430 us/op 1.51
fastMsgIdFn h32 xxhash / 200 bytes 354.00 ns/op 403.00 ns/op 0.88
fastMsgIdFn h64 xxhash / 200 bytes 397.00 ns/op 421.00 ns/op 0.94
fastMsgIdFn sha256 / 1000 bytes 12.362 us/op 6.6730 us/op 1.85
fastMsgIdFn h32 xxhash / 1000 bytes 493.00 ns/op 525.00 ns/op 0.94
fastMsgIdFn h64 xxhash / 1000 bytes 489.00 ns/op 498.00 ns/op 0.98
fastMsgIdFn sha256 / 10000 bytes 117.09 us/op 60.789 us/op 1.93
fastMsgIdFn h32 xxhash / 10000 bytes 2.1850 us/op 1.9720 us/op 1.11
fastMsgIdFn h64 xxhash / 10000 bytes 1.5070 us/op 1.4410 us/op 1.05
send data - 1000 256B messages 23.709 ms/op 18.922 ms/op 1.25
send data - 1000 512B messages 29.089 ms/op 20.272 ms/op 1.43
send data - 1000 1024B messages 43.975 ms/op 29.557 ms/op 1.49
send data - 1000 1200B messages 38.466 ms/op 36.768 ms/op 1.05
send data - 1000 2048B messages 49.931 ms/op 42.062 ms/op 1.19
send data - 1000 4096B messages 32.799 ms/op 37.755 ms/op 0.87
send data - 1000 16384B messages 126.70 ms/op 106.91 ms/op 1.19
send data - 1000 65536B messages 461.01 ms/op 405.20 ms/op 1.14
enrSubnets - fastDeserialize 64 bits 1.4010 us/op 1.3370 us/op 1.05
enrSubnets - ssz BitVector 64 bits 468.00 ns/op 695.00 ns/op 0.67
enrSubnets - fastDeserialize 4 bits 181.00 ns/op 297.00 ns/op 0.61
enrSubnets - ssz BitVector 4 bits 455.00 ns/op 646.00 ns/op 0.70
prioritizePeers score -10:0 att 32-0.1 sync 2-0 112.90 us/op 95.234 us/op 1.19
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 144.29 us/op 132.27 us/op 1.09
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 198.44 us/op 158.70 us/op 1.25
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 341.32 us/op 250.55 us/op 1.36
prioritizePeers score 0:0 att 64-1 sync 4-1 355.15 us/op 272.70 us/op 1.30
array of 16000 items push then shift 1.6692 us/op 1.5466 us/op 1.08
LinkedList of 16000 items push then shift 8.9780 ns/op 7.8760 ns/op 1.14
array of 16000 items push then pop 108.38 ns/op 124.05 ns/op 0.87
LinkedList of 16000 items push then pop 8.8670 ns/op 6.1830 ns/op 1.43
array of 24000 items push then shift 2.5373 us/op 2.1954 us/op 1.16
LinkedList of 24000 items push then shift 9.1640 ns/op 9.8020 ns/op 0.93
array of 24000 items push then pop 141.98 ns/op 208.78 ns/op 0.68
LinkedList of 24000 items push then pop 8.8630 ns/op 10.845 ns/op 0.82
intersect bitArray bitLen 8 6.1340 ns/op 6.8830 ns/op 0.89
intersect array and set length 8 66.531 ns/op 121.68 ns/op 0.55
intersect bitArray bitLen 128 37.038 ns/op 35.100 ns/op 1.06
intersect array and set length 128 985.25 ns/op 1.1287 us/op 0.87
bitArray.getTrueBitIndexes() bitLen 128 1.6120 us/op 2.2320 us/op 0.72
bitArray.getTrueBitIndexes() bitLen 248 2.6630 us/op 3.7780 us/op 0.70
bitArray.getTrueBitIndexes() bitLen 512 5.1390 us/op 8.0170 us/op 0.64
Buffer.concat 32 items 999.00 ns/op 1.3330 us/op 0.75
Uint8Array.set 32 items 1.7140 us/op 2.8400 us/op 0.60
Set add up to 64 items then delete first 4.5044 us/op 2.9216 us/op 1.54
OrderedSet add up to 64 items then delete first 5.9610 us/op 3.6613 us/op 1.63
Set add up to 64 items then delete last 4.8061 us/op 2.9581 us/op 1.62
OrderedSet add up to 64 items then delete last 5.9857 us/op 4.2445 us/op 1.41
Set add up to 64 items then delete middle 4.7742 us/op 3.4176 us/op 1.40
OrderedSet add up to 64 items then delete middle 7.7394 us/op 6.7024 us/op 1.15
Set add up to 128 items then delete first 9.8322 us/op 6.4419 us/op 1.53
OrderedSet add up to 128 items then delete first 12.932 us/op 7.3323 us/op 1.76
Set add up to 128 items then delete last 9.6992 us/op 4.5780 us/op 2.12
OrderedSet add up to 128 items then delete last 12.333 us/op 7.6131 us/op 1.62
Set add up to 128 items then delete middle 9.7431 us/op 3.9294 us/op 2.48
OrderedSet add up to 128 items then delete middle 19.551 us/op 11.784 us/op 1.66
Set add up to 256 items then delete first 20.185 us/op 8.6266 us/op 2.34
OrderedSet add up to 256 items then delete first 26.825 us/op 12.942 us/op 2.07
Set add up to 256 items then delete last 19.592 us/op 7.7866 us/op 2.52
OrderedSet add up to 256 items then delete last 24.614 us/op 12.296 us/op 2.00
Set add up to 256 items then delete middle 19.782 us/op 8.3097 us/op 2.38
OrderedSet add up to 256 items then delete middle 48.123 us/op 38.937 us/op 1.24
transfer serialized Status (84 B) 1.7650 us/op 2.0800 us/op 0.85
copy serialized Status (84 B) 1.3540 us/op 1.4540 us/op 0.93
transfer serialized SignedVoluntaryExit (112 B) 1.8430 us/op 2.1620 us/op 0.85
copy serialized SignedVoluntaryExit (112 B) 1.4080 us/op 1.6720 us/op 0.84
transfer serialized ProposerSlashing (416 B) 2.2130 us/op 3.2360 us/op 0.68
copy serialized ProposerSlashing (416 B) 1.8340 us/op 3.2120 us/op 0.57
transfer serialized Attestation (485 B) 2.2950 us/op 3.4140 us/op 0.67
copy serialized Attestation (485 B) 2.9690 us/op 3.3150 us/op 0.90
transfer serialized AttesterSlashing (33232 B) 2.8140 us/op 3.3720 us/op 0.83
copy serialized AttesterSlashing (33232 B) 7.7040 us/op 11.271 us/op 0.68
transfer serialized Small SignedBeaconBlock (128000 B) 2.9920 us/op 3.8700 us/op 0.77
copy serialized Small SignedBeaconBlock (128000 B) 22.330 us/op 35.131 us/op 0.64
transfer serialized Avg SignedBeaconBlock (200000 B) 3.1020 us/op 3.5610 us/op 0.87
copy serialized Avg SignedBeaconBlock (200000 B) 27.685 us/op 47.851 us/op 0.58
transfer serialized BlobsSidecar (524380 B) 3.3900 us/op 4.4720 us/op 0.76
copy serialized BlobsSidecar (524380 B) 86.308 us/op 158.47 us/op 0.54
transfer serialized Big SignedBeaconBlock (1000000 B) 3.3170 us/op 7.5160 us/op 0.44
copy serialized Big SignedBeaconBlock (1000000 B) 169.26 us/op 302.40 us/op 0.56
pass gossip attestations to forkchoice per slot 3.9352 ms/op 3.0524 ms/op 1.29
forkChoice updateHead vc 100000 bc 64 eq 0 692.64 us/op 522.25 us/op 1.33
forkChoice updateHead vc 600000 bc 64 eq 0 4.1947 ms/op 2.9886 ms/op 1.40
forkChoice updateHead vc 1000000 bc 64 eq 0 7.4065 ms/op 5.8806 ms/op 1.26
forkChoice updateHead vc 600000 bc 320 eq 0 4.4400 ms/op 3.0157 ms/op 1.47
forkChoice updateHead vc 600000 bc 1200 eq 0 4.3928 ms/op 3.0336 ms/op 1.45
forkChoice updateHead vc 600000 bc 7200 eq 0 5.5174 ms/op 4.1313 ms/op 1.34
forkChoice updateHead vc 600000 bc 64 eq 1000 11.647 ms/op 11.157 ms/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 10000 12.469 ms/op 10.182 ms/op 1.22
forkChoice updateHead vc 600000 bc 64 eq 300000 18.642 ms/op 13.632 ms/op 1.37
computeDeltas 500000 validators 300 proto nodes 7.1670 ms/op 4.0071 ms/op 1.79
computeDeltas 500000 validators 1200 proto nodes 7.2942 ms/op 3.6535 ms/op 2.00
computeDeltas 500000 validators 7200 proto nodes 7.0524 ms/op 3.7829 ms/op 1.86
computeDeltas 750000 validators 300 proto nodes 11.016 ms/op 5.1149 ms/op 2.15
computeDeltas 750000 validators 1200 proto nodes 10.625 ms/op 4.6389 ms/op 2.29
computeDeltas 750000 validators 7200 proto nodes 10.688 ms/op 4.6019 ms/op 2.32
computeDeltas 1400000 validators 300 proto nodes 20.683 ms/op 9.4312 ms/op 2.19
computeDeltas 1400000 validators 1200 proto nodes 19.986 ms/op 9.6095 ms/op 2.08
computeDeltas 1400000 validators 7200 proto nodes 19.499 ms/op 10.074 ms/op 1.94
computeDeltas 2100000 validators 300 proto nodes 29.990 ms/op 14.677 ms/op 2.04
computeDeltas 2100000 validators 1200 proto nodes 29.632 ms/op 15.848 ms/op 1.87
computeDeltas 2100000 validators 7200 proto nodes 29.633 ms/op 18.039 ms/op 1.64
altair processAttestation - 250000 vs - 7PWei normalcase 3.0907 ms/op 3.1350 ms/op 0.99
altair processAttestation - 250000 vs - 7PWei worstcase 4.4402 ms/op 3.7405 ms/op 1.19
altair processAttestation - setStatus - 1/6 committees join 198.60 us/op 97.986 us/op 2.03
altair processAttestation - setStatus - 1/3 committees join 406.05 us/op 240.70 us/op 1.69
altair processAttestation - setStatus - 1/2 committees join 516.49 us/op 302.16 us/op 1.71
altair processAttestation - setStatus - 2/3 committees join 606.24 us/op 372.79 us/op 1.63
altair processAttestation - setStatus - 4/5 committees join 815.59 us/op 628.13 us/op 1.30
altair processAttestation - setStatus - 100% committees join 1.0008 ms/op 641.49 us/op 1.56
altair processBlock - 250000 vs - 7PWei normalcase 10.365 ms/op 9.1848 ms/op 1.13
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.293 ms/op 34.660 ms/op 1.08
altair processBlock - 250000 vs - 7PWei worstcase 45.305 ms/op 34.586 ms/op 1.31
altair processBlock - 250000 vs - 7PWei worstcase hashState 106.84 ms/op 91.891 ms/op 1.16
phase0 processBlock - 250000 vs - 7PWei normalcase 4.2881 ms/op 2.3592 ms/op 1.82
phase0 processBlock - 250000 vs - 7PWei worstcase 35.443 ms/op 26.174 ms/op 1.35
altair processEth1Data - 250000 vs - 7PWei normalcase 656.59 us/op 421.50 us/op 1.56
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 15.071 us/op 18.317 us/op 0.82
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 89.839 us/op 60.016 us/op 1.50
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 22.266 us/op 17.991 us/op 1.24
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 22.200 us/op 15.231 us/op 1.46
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 261.00 us/op 126.94 us/op 2.06
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.7671 ms/op 1.2901 ms/op 1.37
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.3501 ms/op 1.1218 ms/op 2.09
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8755 ms/op 1.2881 ms/op 1.46
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.2629 ms/op 3.3564 ms/op 1.27
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.7653 ms/op 2.1130 ms/op 1.31
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 6.8678 ms/op 5.3513 ms/op 1.28
Tree 40 250000 create 575.35 ms/op 270.96 ms/op 2.12
Tree 40 250000 get(125000) 223.98 ns/op 118.23 ns/op 1.89
Tree 40 250000 set(125000) 1.3077 us/op 810.04 ns/op 1.61
Tree 40 250000 toArray() 25.131 ms/op 22.064 ms/op 1.14
Tree 40 250000 iterate all - toArray() + loop 25.123 ms/op 23.855 ms/op 1.05
Tree 40 250000 iterate all - get(i) 78.302 ms/op 54.022 ms/op 1.45
MutableVector 250000 create 18.536 ms/op 10.786 ms/op 1.72
MutableVector 250000 get(125000) 6.9880 ns/op 5.9010 ns/op 1.18
MutableVector 250000 set(125000) 350.89 ns/op 281.66 ns/op 1.25
MutableVector 250000 toArray() 4.2325 ms/op 3.7240 ms/op 1.14
MutableVector 250000 iterate all - toArray() + loop 4.3155 ms/op 3.8455 ms/op 1.12
MutableVector 250000 iterate all - get(i) 1.7274 ms/op 1.3368 ms/op 1.29
Array 250000 create 4.2661 ms/op 3.5855 ms/op 1.19
Array 250000 clone - spread 1.5397 ms/op 1.2814 ms/op 1.20
Array 250000 get(125000) 1.2420 ns/op 1.3290 ns/op 0.93
Array 250000 set(125000) 5.4710 ns/op 2.0000 ns/op 2.74
Array 250000 iterate all - loop 179.71 us/op 153.75 us/op 1.17
effectiveBalanceIncrements clone Uint8Array 300000 47.730 us/op 20.913 us/op 2.28
effectiveBalanceIncrements clone MutableVector 300000 462.00 ns/op 331.00 ns/op 1.40
effectiveBalanceIncrements rw all Uint8Array 300000 215.42 us/op 189.45 us/op 1.14
effectiveBalanceIncrements rw all MutableVector 300000 110.49 ms/op 72.634 ms/op 1.52
phase0 afterProcessEpoch - 250000 vs - 7PWei 126.97 ms/op 78.250 ms/op 1.62
phase0 beforeProcessEpoch - 250000 vs - 7PWei 46.543 ms/op 35.272 ms/op 1.32
altair processEpoch - mainnet_e81889 615.08 ms/op 515.76 ms/op 1.19
mainnet_e81889 - altair beforeProcessEpoch 97.976 ms/op 88.649 ms/op 1.11
mainnet_e81889 - altair processJustificationAndFinalization 19.493 us/op 13.992 us/op 1.39
mainnet_e81889 - altair processInactivityUpdates 7.0551 ms/op 5.5629 ms/op 1.27
mainnet_e81889 - altair processRewardsAndPenalties 70.441 ms/op 56.582 ms/op 1.24
mainnet_e81889 - altair processRegistryUpdates 3.2740 us/op 2.7280 us/op 1.20
mainnet_e81889 - altair processSlashings 643.00 ns/op 748.00 ns/op 0.86
mainnet_e81889 - altair processEth1DataReset 801.00 ns/op 720.00 ns/op 1.11
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.8013 ms/op 1.1073 ms/op 1.63
mainnet_e81889 - altair processSlashingsReset 4.4330 us/op 3.2160 us/op 1.38
mainnet_e81889 - altair processRandaoMixesReset 6.5800 us/op 5.9650 us/op 1.10
mainnet_e81889 - altair processHistoricalRootsUpdate 1.4550 us/op 909.00 ns/op 1.60
mainnet_e81889 - altair processParticipationFlagUpdates 3.7650 us/op 1.6100 us/op 2.34
mainnet_e81889 - altair processSyncCommitteeUpdates 923.00 ns/op 768.00 ns/op 1.20
mainnet_e81889 - altair afterProcessEpoch 130.46 ms/op 83.548 ms/op 1.56
capella processEpoch - mainnet_e217614 2.5771 s/op 2.0995 s/op 1.23
mainnet_e217614 - capella beforeProcessEpoch 528.21 ms/op 467.77 ms/op 1.13
mainnet_e217614 - capella processJustificationAndFinalization 34.975 us/op 12.993 us/op 2.69
mainnet_e217614 - capella processInactivityUpdates 43.298 ms/op 16.072 ms/op 2.69
mainnet_e217614 - capella processRewardsAndPenalties 442.55 ms/op 393.88 ms/op 1.12
mainnet_e217614 - capella processRegistryUpdates 33.916 us/op 25.251 us/op 1.34
mainnet_e217614 - capella processSlashings 733.00 ns/op 739.00 ns/op 0.99
mainnet_e217614 - capella processEth1DataReset 480.00 ns/op 1.0180 us/op 0.47
mainnet_e217614 - capella processEffectiveBalanceUpdates 6.3895 ms/op 12.742 ms/op 0.50
mainnet_e217614 - capella processSlashingsReset 4.4150 us/op 4.6860 us/op 0.94
mainnet_e217614 - capella processRandaoMixesReset 14.259 us/op 6.7400 us/op 2.12
mainnet_e217614 - capella processHistoricalRootsUpdate 1.1470 us/op 879.00 ns/op 1.30
mainnet_e217614 - capella processParticipationFlagUpdates 2.7970 us/op 2.4250 us/op 1.15
mainnet_e217614 - capella afterProcessEpoch 333.59 ms/op 205.00 ms/op 1.63
phase0 processEpoch - mainnet_e58758 629.78 ms/op 498.34 ms/op 1.26
mainnet_e58758 - phase0 beforeProcessEpoch 216.75 ms/op 165.73 ms/op 1.31
mainnet_e58758 - phase0 processJustificationAndFinalization 34.886 us/op 14.808 us/op 2.36
mainnet_e58758 - phase0 processRewardsAndPenalties 60.416 ms/op 51.630 ms/op 1.17
mainnet_e58758 - phase0 processRegistryUpdates 25.045 us/op 16.142 us/op 1.55
mainnet_e58758 - phase0 processSlashings 1.5370 us/op 994.00 ns/op 1.55
mainnet_e58758 - phase0 processEth1DataReset 1.0970 us/op 1.1250 us/op 0.98
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.4391 ms/op 1.3352 ms/op 1.08
mainnet_e58758 - phase0 processSlashingsReset 7.3090 us/op 5.8320 us/op 1.25
mainnet_e58758 - phase0 processRandaoMixesReset 12.785 us/op 7.6690 us/op 1.67
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.6010 us/op 1.0400 us/op 1.54
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.502 us/op 7.5610 us/op 1.39
mainnet_e58758 - phase0 afterProcessEpoch 112.41 ms/op 70.582 ms/op 1.59
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.0596 ms/op 1.5791 ms/op 1.30
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.7221 ms/op 1.6662 ms/op 1.63
altair processInactivityUpdates - 250000 normalcase 39.889 ms/op 27.727 ms/op 1.44
altair processInactivityUpdates - 250000 worstcase 36.325 ms/op 35.173 ms/op 1.03
phase0 processRegistryUpdates - 250000 normalcase 20.989 us/op 11.967 us/op 1.75
phase0 processRegistryUpdates - 250000 badcase_full_deposits 605.18 us/op 474.87 us/op 1.27
phase0 processRegistryUpdates - 250000 worstcase 0.5 169.02 ms/op 179.92 ms/op 0.94
altair processRewardsAndPenalties - 250000 normalcase 58.248 ms/op 60.953 ms/op 0.96
altair processRewardsAndPenalties - 250000 worstcase 57.595 ms/op 56.319 ms/op 1.02
phase0 getAttestationDeltas - 250000 normalcase 14.120 ms/op 6.0526 ms/op 2.33
phase0 getAttestationDeltas - 250000 worstcase 14.163 ms/op 5.6972 ms/op 2.49
phase0 processSlashings - 250000 worstcase 133.31 us/op 93.402 us/op 1.43
altair processSyncCommitteeUpdates - 250000 187.77 ms/op 102.49 ms/op 1.83
BeaconState.hashTreeRoot - No change 920.00 ns/op 420.00 ns/op 2.19
BeaconState.hashTreeRoot - 1 full validator 165.93 us/op 113.51 us/op 1.46
BeaconState.hashTreeRoot - 32 full validator 2.2007 ms/op 1.2663 ms/op 1.74
BeaconState.hashTreeRoot - 512 full validator 20.284 ms/op 15.406 ms/op 1.32
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 188.38 us/op 164.44 us/op 1.15
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.8041 ms/op 2.1307 ms/op 1.32
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 40.857 ms/op 26.401 ms/op 1.55
BeaconState.hashTreeRoot - 1 balances 172.18 us/op 125.16 us/op 1.38
BeaconState.hashTreeRoot - 32 balances 1.3968 ms/op 987.37 us/op 1.41
BeaconState.hashTreeRoot - 512 balances 15.287 ms/op 10.205 ms/op 1.50
BeaconState.hashTreeRoot - 250000 balances 264.47 ms/op 173.17 ms/op 1.53
aggregationBits - 2048 els - zipIndexesInBitList 31.737 us/op 17.319 us/op 1.83
byteArrayEquals 32 87.674 ns/op 63.772 ns/op 1.37
Buffer.compare 32 62.774 ns/op 35.902 ns/op 1.75
byteArrayEquals 1024 2.3087 us/op 1.8409 us/op 1.25
Buffer.compare 1024 79.563 ns/op 43.210 ns/op 1.84
byteArrayEquals 16384 36.803 us/op 26.831 us/op 1.37
Buffer.compare 16384 275.36 ns/op 195.49 ns/op 1.41
byteArrayEquals 123687377 277.02 ms/op 220.17 ms/op 1.26
Buffer.compare 123687377 9.3208 ms/op 7.8307 ms/op 1.19
byteArrayEquals 32 - diff last byte 86.328 ns/op 77.238 ns/op 1.12
Buffer.compare 32 - diff last byte 63.450 ns/op 43.242 ns/op 1.47
byteArrayEquals 1024 - diff last byte 2.2483 us/op 2.0613 us/op 1.09
Buffer.compare 1024 - diff last byte 78.428 ns/op 49.612 ns/op 1.58
byteArrayEquals 16384 - diff last byte 35.638 us/op 30.843 us/op 1.16
Buffer.compare 16384 - diff last byte 279.82 ns/op 245.68 ns/op 1.14
byteArrayEquals 123687377 - diff last byte 272.61 ms/op 252.03 ms/op 1.08
Buffer.compare 123687377 - diff last byte 8.2572 ms/op 8.8645 ms/op 0.93
byteArrayEquals 32 - random bytes 6.5680 ns/op 5.9770 ns/op 1.10
Buffer.compare 32 - random bytes 64.810 ns/op 43.625 ns/op 1.49
byteArrayEquals 1024 - random bytes 6.5180 ns/op 6.3830 ns/op 1.02
Buffer.compare 1024 - random bytes 67.691 ns/op 39.351 ns/op 1.72
byteArrayEquals 16384 - random bytes 6.6250 ns/op 6.9330 ns/op 0.96
Buffer.compare 16384 - random bytes 69.574 ns/op 48.180 ns/op 1.44
byteArrayEquals 123687377 - random bytes 8.8700 ns/op 21.870 ns/op 0.41
Buffer.compare 123687377 - random bytes 63.850 ns/op 57.270 ns/op 1.11
regular array get 100000 times 44.567 us/op 44.694 us/op 1.00
wrappedArray get 100000 times 44.566 us/op 44.156 us/op 1.01
arrayWithProxy get 100000 times 14.142 ms/op 11.230 ms/op 1.26
ssz.Root.equals 59.031 ns/op 63.617 ns/op 0.93
byteArrayEquals 53.886 ns/op 57.682 ns/op 0.93
Buffer.compare 11.043 ns/op 12.603 ns/op 0.88
shuffle list - 16384 els 7.2749 ms/op 5.2027 ms/op 1.40
shuffle list - 250000 els 101.59 ms/op 82.107 ms/op 1.24
processSlot - 1 slots 18.029 us/op 22.154 us/op 0.81
processSlot - 32 slots 3.5073 ms/op 3.5070 ms/op 1.00
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 60.543 ms/op 60.619 ms/op 1.00
getCommitteeAssignments - req 1 vs - 250000 vc 2.6349 ms/op 2.5782 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 3.8898 ms/op 3.6556 ms/op 1.06
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3164 ms/op 4.0519 ms/op 1.07
findModifiedValidators - 10000 modified validators 613.55 ms/op 570.34 ms/op 1.08
findModifiedValidators - 1000 modified validators 493.27 ms/op 439.44 ms/op 1.12
findModifiedValidators - 100 modified validators 425.68 ms/op 518.95 ms/op 0.82
findModifiedValidators - 10 modified validators 482.32 ms/op 427.94 ms/op 1.13
findModifiedValidators - 1 modified validators 431.70 ms/op 436.18 ms/op 0.99
findModifiedValidators - no difference 501.32 ms/op 423.64 ms/op 1.18
compare ViewDUs 5.3828 s/op 5.7393 s/op 0.94
compare each validator Uint8Array 1.8931 s/op 1.6006 s/op 1.18
compare ViewDU to Uint8Array 1.5573 s/op 1.2256 s/op 1.27
migrate state 1000000 validators, 24 modified, 0 new 1.0000 s/op 1.0049 s/op 1.00
migrate state 1000000 validators, 1700 modified, 1000 new 1.3101 s/op 1.2535 s/op 1.05
migrate state 1000000 validators, 3400 modified, 2000 new 1.4787 s/op 1.2214 s/op 1.21
migrate state 1500000 validators, 24 modified, 0 new 862.06 ms/op 653.97 ms/op 1.32
migrate state 1500000 validators, 1700 modified, 1000 new 1.1340 s/op 966.24 ms/op 1.17
migrate state 1500000 validators, 3400 modified, 2000 new 1.4884 s/op 1.1814 s/op 1.26
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.3900 ns/op 5.0800 ns/op 1.06
state getBlockRootAtSlot - 250000 vs - 7PWei 695.21 ns/op 951.49 ns/op 0.73
computeProposers - vc 250000 9.6824 ms/op 7.2573 ms/op 1.33
computeEpochShuffling - vc 250000 110.78 ms/op 68.604 ms/op 1.61
getNextSyncCommittee - vc 250000 187.98 ms/op 122.62 ms/op 1.53
computeSigningRoot for AttestationData 32.156 us/op 26.212 us/op 1.23
hash AttestationData serialized data then Buffer.toString(base64) 2.4926 us/op 1.3404 us/op 1.86
toHexString serialized data 1.6822 us/op 792.24 ns/op 2.12
Buffer.toString(base64) 286.36 ns/op 176.87 ns/op 1.62

by benchmarkbot/action

nflaig
nflaig approved these changes Feb 16, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not ideal but better than having sim tests constantly fail

@nazarhussain nazarhussain merged commit 5047410 into unstable Feb 16, 2024
14 of 15 checks passed
@nazarhussain nazarhussain deleted the nh/sync-comittee-test branch February 16, 2024 09:11
@nazarhussain
Copy link
Contributor Author

Not ideal but better than having sim tests constantly fail

The reference issue with logs are mentioned in the PR. We need to debug and fix the issue separately.

For reference this issue is not new and I observed this behavior many times earlier as well. But in multi_fork test because we have 5 nodes, so low participation for 1 or 2 nodes in few slot does not reduce the epoch participation below 90%. In multi-client test we have only 2 nodes, so why this impact a lot in this test.

@nflaig
Copy link
Member

nflaig commented Feb 16, 2024

But in multi_fork test because we have 5 nodes, so low participation for 1 or 2 nodes in few slot does not reduce the epoch participation below 90%. In multi-client test we have only 2 nodes, so why this impact a lot in this test.

Maybe the runner is just overloaded, and some nodes are resource starved?

@wemeetagain
Copy link
Member

🎉 This PR is included in v1.16.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

Assignees

@nazarhussain nazarhussain

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@nazarhussain @nflaig @wemeetagain