Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rename prover CLI start command as proxy and make it default #6428

Merged
merged 3 commits into from Feb 16, 2024

Conversation

jeluard
Copy link
Member

@jeluard jeluard commented Feb 14, 2024

Motivation

prover CLI has a single command, start, rename it as proxy and make it the default for better UX

Description

It's now possible to start the prover using npx @chainsafe/prover proxy ... or npx @chainsafe/prover ... (previously was npx @chainsafe/prover start ...).
This is a breaking change as the old start command won't work anymore.

@jeluard jeluard requested a review from a team as a code owner February 14, 2024 14:15
Copy link

codecov bot commented Feb 14, 2024

Codecov Report

Merging #6428 (33e22c6) into unstable (cd7421e) will decrease coverage by 0.03%.
Report is 9 commits behind head on unstable.
The diff coverage is n/a.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #6428      +/-   ##
============================================
- Coverage     61.72%   61.70%   -0.03%     
============================================
  Files           553      553              
  Lines         57862    57858       -4     
  Branches       1829     1829              
============================================
- Hits          35717    35702      -15     
- Misses        22108    22119      +11     
  Partials         37       37              

Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this restrict the extensibility of the CLI as it will match any subcommand?

I'd agree that start is not a good name for the sub command, maybe just rename it to proxy? We also need to be mindful when renaming commands as it breaks current usage but I don't know of anybody even uses the prover as a proxy server.

Copy link
Contributor

github-actions bot commented Feb 14, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: e6c52d3 Previous: 48871eb Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 371.07 us/op 758.14 us/op 0.49
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 54.673 us/op 92.295 us/op 0.59
BLS verify - blst-native 1.1179 ms/op 1.1083 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.4652 ms/op 2.3561 ms/op 1.05
BLS verifyMultipleSignatures 8 - blst-native 5.2801 ms/op 5.2966 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 19.305 ms/op 19.663 ms/op 0.98
BLS verifyMultipleSignatures 64 - blst-native 38.195 ms/op 39.021 ms/op 0.98
BLS verifyMultipleSignatures 128 - blst-native 78.876 ms/op 76.918 ms/op 1.03
BLS deserializing 10000 signatures 817.99 ms/op 809.54 ms/op 1.01
BLS deserializing 100000 signatures 8.3242 s/op 8.4377 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1222 ms/op 1.1323 ms/op 0.99
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2695 ms/op 1.3259 ms/op 0.96
BLS verifyMultipleSignatures - same message - 32 - blst-native 1.9953 ms/op 2.6066 ms/op 0.77
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.7505 ms/op 3.0350 ms/op 1.24
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.0981 ms/op 6.4674 ms/op 0.79
BLS aggregatePubkeys 32 - blst-native 22.670 us/op 24.561 us/op 0.92
BLS aggregatePubkeys 128 - blst-native 88.973 us/op 89.737 us/op 0.99
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 43.271 ms/op 43.123 ms/op 1.00
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 40.621 ms/op 42.176 ms/op 0.96
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 25.192 ms/op 27.155 ms/op 0.93
getSlashingsAndExits - default max 116.63 us/op 199.65 us/op 0.58
getSlashingsAndExits - 2k 322.54 us/op 346.45 us/op 0.93
proposeBlockBody type=full, size=empty 3.8263 ms/op 3.9476 ms/op 0.97
isKnown best case - 1 super set check 367.00 ns/op 521.00 ns/op 0.70
isKnown normal case - 2 super set checks 338.00 ns/op 582.00 ns/op 0.58
isKnown worse case - 16 super set checks 325.00 ns/op 442.00 ns/op 0.74
CheckpointStateCache - add get delete 4.1640 us/op 4.7870 us/op 0.87
validate api signedAggregateAndProof - struct 2.3850 ms/op 2.6261 ms/op 0.91
validate gossip signedAggregateAndProof - struct 2.3869 ms/op 2.5193 ms/op 0.95
validate gossip attestation - vc 640000 1.1417 ms/op 1.2475 ms/op 0.92
batch validate gossip attestation - vc 640000 - chunk 32 137.81 us/op 157.84 us/op 0.87
batch validate gossip attestation - vc 640000 - chunk 64 120.13 us/op 139.93 us/op 0.86
batch validate gossip attestation - vc 640000 - chunk 128 115.66 us/op 139.37 us/op 0.83
batch validate gossip attestation - vc 640000 - chunk 256 116.20 us/op 133.15 us/op 0.87
pickEth1Vote - no votes 954.27 us/op 1.1784 ms/op 0.81
pickEth1Vote - max votes 12.115 ms/op 11.333 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.593 ms/op 16.476 ms/op 0.95
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.645 ms/op 24.542 ms/op 0.96
pickEth1Vote - Eth1Data fastSerialize value x2048 434.04 us/op 532.01 us/op 0.82
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.7130 ms/op 3.7902 ms/op 1.51
bytes32 toHexString 451.00 ns/op 515.00 ns/op 0.88
bytes32 Buffer.toString(hex) 315.00 ns/op 373.00 ns/op 0.84
bytes32 Buffer.toString(hex) from Uint8Array 420.00 ns/op 597.00 ns/op 0.70
bytes32 Buffer.toString(hex) + 0x 309.00 ns/op 356.00 ns/op 0.87
Object access 1 prop 0.18800 ns/op 0.24400 ns/op 0.77
Map access 1 prop 0.18800 ns/op 0.20500 ns/op 0.92
Object get x1000 5.4290 ns/op 5.3920 ns/op 1.01
Map get x1000 0.73800 ns/op 0.87600 ns/op 0.84
Object set x1000 26.720 ns/op 33.061 ns/op 0.81
Map set x1000 17.245 ns/op 21.104 ns/op 0.82
Return object 10000 times 0.22920 ns/op 0.25150 ns/op 0.91
Throw Error 10000 times 2.9347 us/op 3.1602 us/op 0.93
fastMsgIdFn sha256 / 200 bytes 2.0160 us/op 2.4430 us/op 0.83
fastMsgIdFn h32 xxhash / 200 bytes 288.00 ns/op 403.00 ns/op 0.71
fastMsgIdFn h64 xxhash / 200 bytes 341.00 ns/op 421.00 ns/op 0.81
fastMsgIdFn sha256 / 1000 bytes 6.4950 us/op 6.6730 us/op 0.97
fastMsgIdFn h32 xxhash / 1000 bytes 419.00 ns/op 525.00 ns/op 0.80
fastMsgIdFn h64 xxhash / 1000 bytes 455.00 ns/op 498.00 ns/op 0.91
fastMsgIdFn sha256 / 10000 bytes 55.065 us/op 60.789 us/op 0.91
fastMsgIdFn h32 xxhash / 10000 bytes 1.8780 us/op 1.9720 us/op 0.95
fastMsgIdFn h64 xxhash / 10000 bytes 1.2880 us/op 1.4410 us/op 0.89
send data - 1000 256B messages 12.029 ms/op 18.922 ms/op 0.64
send data - 1000 512B messages 15.184 ms/op 20.272 ms/op 0.75
send data - 1000 1024B messages 26.273 ms/op 29.557 ms/op 0.89
send data - 1000 1200B messages 28.759 ms/op 36.768 ms/op 0.78
send data - 1000 2048B messages 37.655 ms/op 42.062 ms/op 0.90
send data - 1000 4096B messages 34.959 ms/op 37.755 ms/op 0.93
send data - 1000 16384B messages 92.065 ms/op 106.91 ms/op 0.86
send data - 1000 65536B messages 392.14 ms/op 405.20 ms/op 0.97
enrSubnets - fastDeserialize 64 bits 1.3990 us/op 1.3370 us/op 1.05
enrSubnets - ssz BitVector 64 bits 553.00 ns/op 695.00 ns/op 0.80
enrSubnets - fastDeserialize 4 bits 243.00 ns/op 297.00 ns/op 0.82
enrSubnets - ssz BitVector 4 bits 456.00 ns/op 646.00 ns/op 0.71
prioritizePeers score -10:0 att 32-0.1 sync 2-0 89.795 us/op 95.234 us/op 0.94
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 108.18 us/op 132.27 us/op 0.82
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 145.52 us/op 158.70 us/op 0.92
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 230.20 us/op 250.55 us/op 0.92
prioritizePeers score 0:0 att 64-1 sync 4-1 234.56 us/op 272.70 us/op 0.86
array of 16000 items push then shift 1.2906 us/op 1.5466 us/op 0.83
LinkedList of 16000 items push then shift 7.4450 ns/op 7.8760 ns/op 0.95
array of 16000 items push then pop 99.399 ns/op 124.05 ns/op 0.80
LinkedList of 16000 items push then pop 5.9990 ns/op 6.1830 ns/op 0.97
array of 24000 items push then shift 1.8816 us/op 2.1954 us/op 0.86
LinkedList of 24000 items push then shift 7.1430 ns/op 9.8020 ns/op 0.73
array of 24000 items push then pop 137.31 ns/op 208.78 ns/op 0.66
LinkedList of 24000 items push then pop 5.7190 ns/op 10.845 ns/op 0.53
intersect bitArray bitLen 8 4.8810 ns/op 6.8830 ns/op 0.71
intersect array and set length 8 49.291 ns/op 121.68 ns/op 0.41
intersect bitArray bitLen 128 29.576 ns/op 35.100 ns/op 0.84
intersect array and set length 128 709.39 ns/op 1.1287 us/op 0.63
bitArray.getTrueBitIndexes() bitLen 128 1.4850 us/op 2.2320 us/op 0.67
bitArray.getTrueBitIndexes() bitLen 248 2.5760 us/op 3.7780 us/op 0.68
bitArray.getTrueBitIndexes() bitLen 512 5.5230 us/op 8.0170 us/op 0.69
Buffer.concat 32 items 977.00 ns/op 1.3330 us/op 0.73
Uint8Array.set 32 items 2.4460 us/op 2.8400 us/op 0.86
Set add up to 64 items then delete first 2.3708 us/op 2.9216 us/op 0.81
OrderedSet add up to 64 items then delete first 2.8384 us/op 3.6613 us/op 0.78
Set add up to 64 items then delete last 2.0029 us/op 2.9581 us/op 0.68
OrderedSet add up to 64 items then delete last 3.0082 us/op 4.2445 us/op 0.71
Set add up to 64 items then delete middle 2.0167 us/op 3.4176 us/op 0.59
OrderedSet add up to 64 items then delete middle 4.2477 us/op 6.7024 us/op 0.63
Set add up to 128 items then delete first 3.8993 us/op 6.4419 us/op 0.61
OrderedSet add up to 128 items then delete first 6.0565 us/op 7.3323 us/op 0.83
Set add up to 128 items then delete last 3.7918 us/op 4.5780 us/op 0.83
OrderedSet add up to 128 items then delete last 5.7304 us/op 7.6131 us/op 0.75
Set add up to 128 items then delete middle 3.7916 us/op 3.9294 us/op 0.96
OrderedSet add up to 128 items then delete middle 10.745 us/op 11.784 us/op 0.91
Set add up to 256 items then delete first 7.6351 us/op 8.6266 us/op 0.89
OrderedSet add up to 256 items then delete first 12.061 us/op 12.942 us/op 0.93
Set add up to 256 items then delete last 7.4103 us/op 7.7866 us/op 0.95
OrderedSet add up to 256 items then delete last 11.425 us/op 12.296 us/op 0.93
Set add up to 256 items then delete middle 7.4431 us/op 8.3097 us/op 0.90
OrderedSet add up to 256 items then delete middle 30.618 us/op 38.937 us/op 0.79
transfer serialized Status (84 B) 1.3400 us/op 2.0800 us/op 0.64
copy serialized Status (84 B) 1.0370 us/op 1.4540 us/op 0.71
transfer serialized SignedVoluntaryExit (112 B) 1.3400 us/op 2.1620 us/op 0.62
copy serialized SignedVoluntaryExit (112 B) 1.0290 us/op 1.6720 us/op 0.62
transfer serialized ProposerSlashing (416 B) 1.4470 us/op 3.2360 us/op 0.45
copy serialized ProposerSlashing (416 B) 1.2880 us/op 3.2120 us/op 0.40
transfer serialized Attestation (485 B) 1.4740 us/op 3.4140 us/op 0.43
copy serialized Attestation (485 B) 1.2650 us/op 3.3150 us/op 0.38
transfer serialized AttesterSlashing (33232 B) 1.5970 us/op 3.3720 us/op 0.47
copy serialized AttesterSlashing (33232 B) 3.2870 us/op 11.271 us/op 0.29
transfer serialized Small SignedBeaconBlock (128000 B) 1.6630 us/op 3.8700 us/op 0.43
copy serialized Small SignedBeaconBlock (128000 B) 8.5140 us/op 35.131 us/op 0.24
transfer serialized Avg SignedBeaconBlock (200000 B) 1.6850 us/op 3.5610 us/op 0.47
copy serialized Avg SignedBeaconBlock (200000 B) 12.249 us/op 47.851 us/op 0.26
transfer serialized BlobsSidecar (524380 B) 2.2140 us/op 4.4720 us/op 0.50
copy serialized BlobsSidecar (524380 B) 76.466 us/op 158.47 us/op 0.48
transfer serialized Big SignedBeaconBlock (1000000 B) 2.4960 us/op 7.5160 us/op 0.33
copy serialized Big SignedBeaconBlock (1000000 B) 137.66 us/op 302.40 us/op 0.46
pass gossip attestations to forkchoice per slot 2.8381 ms/op 3.0524 ms/op 0.93
forkChoice updateHead vc 100000 bc 64 eq 0 432.21 us/op 522.25 us/op 0.83
forkChoice updateHead vc 600000 bc 64 eq 0 2.6129 ms/op 2.9886 ms/op 0.87
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3062 ms/op 5.8806 ms/op 0.73
forkChoice updateHead vc 600000 bc 320 eq 0 2.6952 ms/op 3.0157 ms/op 0.89
forkChoice updateHead vc 600000 bc 1200 eq 0 2.8175 ms/op 3.0336 ms/op 0.93
forkChoice updateHead vc 600000 bc 7200 eq 0 3.3804 ms/op 4.1313 ms/op 0.82
forkChoice updateHead vc 600000 bc 64 eq 1000 9.9565 ms/op 11.157 ms/op 0.89
forkChoice updateHead vc 600000 bc 64 eq 10000 9.9335 ms/op 10.182 ms/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 300000 11.746 ms/op 13.632 ms/op 0.86
computeDeltas 500000 validators 300 proto nodes 3.2476 ms/op 4.0071 ms/op 0.81
computeDeltas 500000 validators 1200 proto nodes 2.9305 ms/op 3.6535 ms/op 0.80
computeDeltas 500000 validators 7200 proto nodes 2.9523 ms/op 3.7829 ms/op 0.78
computeDeltas 750000 validators 300 proto nodes 4.3577 ms/op 5.1149 ms/op 0.85
computeDeltas 750000 validators 1200 proto nodes 4.3968 ms/op 4.6389 ms/op 0.95
computeDeltas 750000 validators 7200 proto nodes 4.5572 ms/op 4.6019 ms/op 0.99
computeDeltas 1400000 validators 300 proto nodes 9.3118 ms/op 9.4312 ms/op 0.99
computeDeltas 1400000 validators 1200 proto nodes 8.6133 ms/op 9.6095 ms/op 0.90
computeDeltas 1400000 validators 7200 proto nodes 9.0403 ms/op 10.074 ms/op 0.90
computeDeltas 2100000 validators 300 proto nodes 13.112 ms/op 14.677 ms/op 0.89
computeDeltas 2100000 validators 1200 proto nodes 13.159 ms/op 15.848 ms/op 0.83
computeDeltas 2100000 validators 7200 proto nodes 13.842 ms/op 18.039 ms/op 0.77
altair processAttestation - 250000 vs - 7PWei normalcase 2.1357 ms/op 3.1350 ms/op 0.68
altair processAttestation - 250000 vs - 7PWei worstcase 2.5706 ms/op 3.7405 ms/op 0.69
altair processAttestation - setStatus - 1/6 committees join 87.005 us/op 97.986 us/op 0.89
altair processAttestation - setStatus - 1/3 committees join 160.44 us/op 240.70 us/op 0.67
altair processAttestation - setStatus - 1/2 committees join 250.45 us/op 302.16 us/op 0.83
altair processAttestation - setStatus - 2/3 committees join 335.96 us/op 372.79 us/op 0.90
altair processAttestation - setStatus - 4/5 committees join 419.78 us/op 628.13 us/op 0.67
altair processAttestation - setStatus - 100% committees join 538.67 us/op 641.49 us/op 0.84
altair processBlock - 250000 vs - 7PWei normalcase 9.7970 ms/op 9.1848 ms/op 1.07
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.050 ms/op 34.660 ms/op 0.72
altair processBlock - 250000 vs - 7PWei worstcase 34.356 ms/op 34.586 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.866 ms/op 91.891 ms/op 1.08
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3233 ms/op 2.3592 ms/op 0.98
phase0 processBlock - 250000 vs - 7PWei worstcase 29.872 ms/op 26.174 ms/op 1.14
altair processEth1Data - 250000 vs - 7PWei normalcase 593.40 us/op 421.50 us/op 1.41
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 13.587 us/op 18.317 us/op 0.74
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 68.039 us/op 60.016 us/op 1.13
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 20.959 us/op 17.991 us/op 1.16
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 17.548 us/op 15.231 us/op 1.15
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 164.98 us/op 126.94 us/op 1.30
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0245 ms/op 1.2901 ms/op 0.79
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.2866 ms/op 1.1218 ms/op 1.15
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1437 ms/op 1.2881 ms/op 0.89
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.3193 ms/op 3.3564 ms/op 1.29
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.8874 ms/op 2.1130 ms/op 0.89
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.5355 ms/op 5.3513 ms/op 0.85
Tree 40 250000 create 249.18 ms/op 270.96 ms/op 0.92
Tree 40 250000 get(125000) 126.13 ns/op 118.23 ns/op 1.07
Tree 40 250000 set(125000) 781.69 ns/op 810.04 ns/op 0.96
Tree 40 250000 toArray() 20.453 ms/op 22.064 ms/op 0.93
Tree 40 250000 iterate all - toArray() + loop 21.869 ms/op 23.855 ms/op 0.92
Tree 40 250000 iterate all - get(i) 54.272 ms/op 54.022 ms/op 1.00
MutableVector 250000 create 13.886 ms/op 10.786 ms/op 1.29
MutableVector 250000 get(125000) 5.9320 ns/op 5.9010 ns/op 1.01
MutableVector 250000 set(125000) 445.40 ns/op 281.66 ns/op 1.58
MutableVector 250000 toArray() 3.2693 ms/op 3.7240 ms/op 0.88
MutableVector 250000 iterate all - toArray() + loop 3.1493 ms/op 3.8455 ms/op 0.82
MutableVector 250000 iterate all - get(i) 1.3661 ms/op 1.3368 ms/op 1.02
Array 250000 create 3.0166 ms/op 3.5855 ms/op 0.84
Array 250000 clone - spread 1.1804 ms/op 1.2814 ms/op 0.92
Array 250000 get(125000) 1.0520 ns/op 1.3290 ns/op 0.79
Array 250000 set(125000) 1.2530 ns/op 2.0000 ns/op 0.63
Array 250000 iterate all - loop 160.18 us/op 153.75 us/op 1.04
effectiveBalanceIncrements clone Uint8Array 300000 14.904 us/op 20.913 us/op 0.71
effectiveBalanceIncrements clone MutableVector 300000 406.00 ns/op 331.00 ns/op 1.23
effectiveBalanceIncrements rw all Uint8Array 300000 190.49 us/op 189.45 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 70.239 ms/op 72.634 ms/op 0.97
phase0 afterProcessEpoch - 250000 vs - 7PWei 80.965 ms/op 78.250 ms/op 1.03
phase0 beforeProcessEpoch - 250000 vs - 7PWei 53.129 ms/op 35.272 ms/op 1.51
altair processEpoch - mainnet_e81889 444.29 ms/op 515.76 ms/op 0.86
mainnet_e81889 - altair beforeProcessEpoch 86.700 ms/op 88.649 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 15.907 us/op 13.992 us/op 1.14
mainnet_e81889 - altair processInactivityUpdates 6.3713 ms/op 5.5629 ms/op 1.15
mainnet_e81889 - altair processRewardsAndPenalties 59.128 ms/op 56.582 ms/op 1.04
mainnet_e81889 - altair processRegistryUpdates 2.6350 us/op 2.7280 us/op 0.97
mainnet_e81889 - altair processSlashings 681.00 ns/op 748.00 ns/op 0.91
mainnet_e81889 - altair processEth1DataReset 873.00 ns/op 720.00 ns/op 1.21
mainnet_e81889 - altair processEffectiveBalanceUpdates 6.0742 ms/op 1.1073 ms/op 5.49
mainnet_e81889 - altair processSlashingsReset 4.0470 us/op 3.2160 us/op 1.26
mainnet_e81889 - altair processRandaoMixesReset 5.9580 us/op 5.9650 us/op 1.00
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2960 us/op 909.00 ns/op 1.43
mainnet_e81889 - altair processParticipationFlagUpdates 2.9640 us/op 1.6100 us/op 1.84
mainnet_e81889 - altair processSyncCommitteeUpdates 1.8580 us/op 768.00 ns/op 2.42
mainnet_e81889 - altair afterProcessEpoch 82.866 ms/op 83.548 ms/op 0.99
capella processEpoch - mainnet_e217614 2.2452 s/op 2.0995 s/op 1.07
mainnet_e217614 - capella beforeProcessEpoch 500.43 ms/op 467.77 ms/op 1.07
mainnet_e217614 - capella processJustificationAndFinalization 15.677 us/op 12.993 us/op 1.21
mainnet_e217614 - capella processInactivityUpdates 20.889 ms/op 16.072 ms/op 1.30
mainnet_e217614 - capella processRewardsAndPenalties 368.71 ms/op 393.88 ms/op 0.94
mainnet_e217614 - capella processRegistryUpdates 34.451 us/op 25.251 us/op 1.36
mainnet_e217614 - capella processSlashings 852.00 ns/op 739.00 ns/op 1.15
mainnet_e217614 - capella processEth1DataReset 582.00 ns/op 1.0180 us/op 0.57
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.7275 ms/op 12.742 ms/op 0.29
mainnet_e217614 - capella processSlashingsReset 3.1650 us/op 4.6860 us/op 0.68
mainnet_e217614 - capella processRandaoMixesReset 4.2460 us/op 6.7400 us/op 0.63
mainnet_e217614 - capella processHistoricalRootsUpdate 840.00 ns/op 879.00 ns/op 0.96
mainnet_e217614 - capella processParticipationFlagUpdates 2.7540 us/op 2.4250 us/op 1.14
mainnet_e217614 - capella afterProcessEpoch 233.71 ms/op 205.00 ms/op 1.14
phase0 processEpoch - mainnet_e58758 392.77 ms/op 498.34 ms/op 0.79
mainnet_e58758 - phase0 beforeProcessEpoch 125.47 ms/op 165.73 ms/op 0.76
mainnet_e58758 - phase0 processJustificationAndFinalization 17.608 us/op 14.808 us/op 1.19
mainnet_e58758 - phase0 processRewardsAndPenalties 51.544 ms/op 51.630 ms/op 1.00
mainnet_e58758 - phase0 processRegistryUpdates 10.651 us/op 16.142 us/op 0.66
mainnet_e58758 - phase0 processSlashings 710.00 ns/op 994.00 ns/op 0.71
mainnet_e58758 - phase0 processEth1DataReset 681.00 ns/op 1.1250 us/op 0.61
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 914.89 us/op 1.3352 ms/op 0.69
mainnet_e58758 - phase0 processSlashingsReset 2.7620 us/op 5.8320 us/op 0.47
mainnet_e58758 - phase0 processRandaoMixesReset 4.3430 us/op 7.6690 us/op 0.57
mainnet_e58758 - phase0 processHistoricalRootsUpdate 687.00 ns/op 1.0400 us/op 0.66
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.5650 us/op 7.5610 us/op 0.34
mainnet_e58758 - phase0 afterProcessEpoch 63.783 ms/op 70.582 ms/op 0.90
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0263 ms/op 1.5791 ms/op 0.65
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.1016 ms/op 1.6662 ms/op 0.66
altair processInactivityUpdates - 250000 normalcase 23.964 ms/op 27.727 ms/op 0.86
altair processInactivityUpdates - 250000 worstcase 23.785 ms/op 35.173 ms/op 0.68
phase0 processRegistryUpdates - 250000 normalcase 6.6190 us/op 11.967 us/op 0.55
phase0 processRegistryUpdates - 250000 badcase_full_deposits 367.66 us/op 474.87 us/op 0.77
phase0 processRegistryUpdates - 250000 worstcase 0.5 120.25 ms/op 179.92 ms/op 0.67
altair processRewardsAndPenalties - 250000 normalcase 50.333 ms/op 60.953 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 51.881 ms/op 56.319 ms/op 0.92
phase0 getAttestationDeltas - 250000 normalcase 6.0611 ms/op 6.0526 ms/op 1.00
phase0 getAttestationDeltas - 250000 worstcase 6.1087 ms/op 5.6972 ms/op 1.07
phase0 processSlashings - 250000 worstcase 53.990 us/op 93.402 us/op 0.58
altair processSyncCommitteeUpdates - 250000 111.48 ms/op 102.49 ms/op 1.09
BeaconState.hashTreeRoot - No change 479.00 ns/op 420.00 ns/op 1.14
BeaconState.hashTreeRoot - 1 full validator 126.36 us/op 113.51 us/op 1.11
BeaconState.hashTreeRoot - 32 full validator 1.4052 ms/op 1.2663 ms/op 1.11
BeaconState.hashTreeRoot - 512 full validator 13.928 ms/op 15.406 ms/op 0.90
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 155.26 us/op 164.44 us/op 0.94
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.6524 ms/op 2.1307 ms/op 0.78
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 30.310 ms/op 26.401 ms/op 1.15
BeaconState.hashTreeRoot - 1 balances 128.34 us/op 125.16 us/op 1.03
BeaconState.hashTreeRoot - 32 balances 1.2316 ms/op 987.37 us/op 1.25
BeaconState.hashTreeRoot - 512 balances 12.294 ms/op 10.205 ms/op 1.20
BeaconState.hashTreeRoot - 250000 balances 201.29 ms/op 173.17 ms/op 1.16
aggregationBits - 2048 els - zipIndexesInBitList 10.549 us/op 17.319 us/op 0.61
byteArrayEquals 32 64.866 ns/op 63.772 ns/op 1.02
Buffer.compare 32 37.731 ns/op 35.902 ns/op 1.05
byteArrayEquals 1024 1.7389 us/op 1.8409 us/op 0.94
Buffer.compare 1024 51.941 ns/op 43.210 ns/op 1.20
byteArrayEquals 16384 28.276 us/op 26.831 us/op 1.05
Buffer.compare 16384 201.59 ns/op 195.49 ns/op 1.03
byteArrayEquals 123687377 195.49 ms/op 220.17 ms/op 0.89
Buffer.compare 123687377 5.3077 ms/op 7.8307 ms/op 0.68
byteArrayEquals 32 - diff last byte 58.614 ns/op 77.238 ns/op 0.76
Buffer.compare 32 - diff last byte 35.328 ns/op 43.242 ns/op 0.82
byteArrayEquals 1024 - diff last byte 1.6442 us/op 2.0613 us/op 0.80
Buffer.compare 1024 - diff last byte 50.070 ns/op 49.612 ns/op 1.01
byteArrayEquals 16384 - diff last byte 26.328 us/op 30.843 us/op 0.85
Buffer.compare 16384 - diff last byte 190.03 ns/op 245.68 ns/op 0.77
byteArrayEquals 123687377 - diff last byte 201.89 ms/op 252.03 ms/op 0.80
Buffer.compare 123687377 - diff last byte 4.0137 ms/op 8.8645 ms/op 0.45
byteArrayEquals 32 - random bytes 4.5810 ns/op 5.9770 ns/op 0.77
Buffer.compare 32 - random bytes 40.193 ns/op 43.625 ns/op 0.92
byteArrayEquals 1024 - random bytes 4.5160 ns/op 6.3830 ns/op 0.71
Buffer.compare 1024 - random bytes 38.290 ns/op 39.351 ns/op 0.97
byteArrayEquals 16384 - random bytes 4.4860 ns/op 6.9330 ns/op 0.65
Buffer.compare 16384 - random bytes 38.623 ns/op 48.180 ns/op 0.80
byteArrayEquals 123687377 - random bytes 8.0500 ns/op 21.870 ns/op 0.37
Buffer.compare 123687377 - random bytes 44.000 ns/op 57.270 ns/op 0.77
regular array get 100000 times 42.239 us/op 44.694 us/op 0.95
wrappedArray get 100000 times 40.560 us/op 44.156 us/op 0.92
arrayWithProxy get 100000 times 10.600 ms/op 11.230 ms/op 0.94
ssz.Root.equals 51.399 ns/op 63.617 ns/op 0.81
byteArrayEquals 53.180 ns/op 57.682 ns/op 0.92
Buffer.compare 8.9930 ns/op 12.603 ns/op 0.71
shuffle list - 16384 els 4.6423 ms/op 5.2027 ms/op 0.89
shuffle list - 250000 els 68.137 ms/op 82.107 ms/op 0.83
processSlot - 1 slots 14.989 us/op 22.154 us/op 0.68
processSlot - 32 slots 3.4817 ms/op 3.5070 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 53.538 ms/op 60.619 ms/op 0.88
getCommitteeAssignments - req 1 vs - 250000 vc 2.2713 ms/op 2.5782 ms/op 0.88
getCommitteeAssignments - req 100 vs - 250000 vc 3.3253 ms/op 3.6556 ms/op 0.91
getCommitteeAssignments - req 1000 vs - 250000 vc 3.5585 ms/op 4.0519 ms/op 0.88
findModifiedValidators - 10000 modified validators 437.15 ms/op 570.34 ms/op 0.77
findModifiedValidators - 1000 modified validators 338.82 ms/op 439.44 ms/op 0.77
findModifiedValidators - 100 modified validators 312.21 ms/op 518.95 ms/op 0.60
findModifiedValidators - 10 modified validators 342.09 ms/op 427.94 ms/op 0.80
findModifiedValidators - 1 modified validators 335.95 ms/op 436.18 ms/op 0.77
findModifiedValidators - no difference 348.84 ms/op 423.64 ms/op 0.82
compare ViewDUs 3.9103 s/op 5.7393 s/op 0.68
compare each validator Uint8Array 1.2941 s/op 1.6006 s/op 0.81
compare ViewDU to Uint8Array 872.16 ms/op 1.2256 s/op 0.71
migrate state 1000000 validators, 24 modified, 0 new 701.11 ms/op 1.0049 s/op 0.70
migrate state 1000000 validators, 1700 modified, 1000 new 901.11 ms/op 1.2535 s/op 0.72
migrate state 1000000 validators, 3400 modified, 2000 new 1.2635 s/op 1.2214 s/op 1.03
migrate state 1500000 validators, 24 modified, 0 new 740.26 ms/op 653.97 ms/op 1.13
migrate state 1500000 validators, 1700 modified, 1000 new 874.36 ms/op 966.24 ms/op 0.90
migrate state 1500000 validators, 3400 modified, 2000 new 1.2526 s/op 1.1814 s/op 1.06
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3900 ns/op 5.0800 ns/op 0.86
state getBlockRootAtSlot - 250000 vs - 7PWei 692.73 ns/op 951.49 ns/op 0.73
computeProposers - vc 250000 6.0758 ms/op 7.2573 ms/op 0.84
computeEpochShuffling - vc 250000 68.816 ms/op 68.604 ms/op 1.00
getNextSyncCommittee - vc 250000 107.13 ms/op 122.62 ms/op 0.87
computeSigningRoot for AttestationData 24.592 us/op 26.212 us/op 0.94
hash AttestationData serialized data then Buffer.toString(base64) 1.2634 us/op 1.3404 us/op 0.94
toHexString serialized data 830.17 ns/op 792.24 ns/op 1.05
Buffer.toString(base64) 155.73 ns/op 176.87 ns/op 0.88

by benchmarkbot/action

@jeluard
Copy link
Member Author

jeluard commented Feb 14, 2024

I am under the impression that there are no other commands planned in the foreseeable future. Maybe @nazarhussain can comment on that.

@nazarhussain
Copy link
Contributor

I am under the impression that there are no other commands planned in the foreseeable future. Maybe @nazarhussain can comment on that.

My plans were to provide some utility commands like verified-balance, storage-value etc in future. So you run, sync perform that task and show result to user and shutdown.

But don't have any planned schedule for such features.

@jeluard
Copy link
Member Author

jeluard commented Feb 14, 2024

My plans were to provide some utility commands like verified-balance, storage-value etc in future. So you run, sync perform that task and show result to user and shutdown.

Great ideas! Ok, I will just rename to proxy then, to be future proof.

@@ -48,6 +48,9 @@ export function getLodestarProverCli(): yargs.Argv {
registerCommandToYargs(prover, cmd);
}

// Register the proxy command as the default one
registerCommandToYargs(prover, {...proverProxyStartCommand, command: '*'});
Copy link
Member

@nflaig nflaig Feb 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if you run lodestar-prover --help, I would assume it no longer prints out global flags and a list of available commands.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It prints the global help screen, as before. lodestar-prover proxy --help prints the help for the proxy command as expected.

Copy link
Contributor

@nazarhussain nazarhussain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

That will be a breaking change, we need to discuss and finalize the release strategy for prover in such cases. Opened a discussion point for next meeting. #6434 (comment)

@nflaig
Copy link
Member

nflaig commented Feb 16, 2024

The PR description and title needs to be updated to properly reflect the changes. We might also wanna label this PR as breaking change and add a BREAKING CHANGE note as well with the changes required

@jeluard jeluard changed the title feat: make start command default for prover CLI feat: rename prover CLI start command as proxy and make it default Feb 16, 2024
@nflaig nflaig added the meta-breaking-change Introduces breaking changes to DB, Validator, Beacon Node, or CLI interfaces. Handle with care! label Feb 16, 2024
@jeluard jeluard merged commit a02ea75 into unstable Feb 16, 2024
18 of 19 checks passed
@jeluard jeluard deleted the jeluard/prover-start-command branch February 16, 2024 09:52
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.16.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
meta-breaking-change Introduces breaking changes to DB, Validator, Beacon Node, or CLI interfaces. Handle with care!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants