Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: modify minimal preset's field elements per blob to mainnet for dev runs #5484

Merged
merged 1 commit into from May 16, 2023

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented May 12, 2023

As part of by parts integration of free the blobs PR

For devnets started using cli in minimal and for the merge interop/inline tests, the FIELD_ELEMENTS_PER_BLOB needs to be set to 4096 which is the mainnet value

because all kzg libraries and ELs process the blobs on this size and is a hardcoded value in them. We only need minimal value for spec tests which does not use the cli path or will not be providing DEV_RUN flag on test setup

@g11tech g11tech requested a review from a team as a code owner May 12, 2023 09:11
@g11tech g11tech changed the title refactor: Modify minimal preset's field elements per blob to mainnet for dev runs fix: Modify minimal preset's field elements per blob to mainnet for dev runs May 12, 2023
@g11tech g11tech changed the title fix: Modify minimal preset's field elements per blob to mainnet for dev runs fix: modify minimal preset's field elements per blob to mainnet for dev runs May 12, 2023
@github-actions
Copy link
Contributor

github-actions bot commented May 12, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 3151190 Previous: f16192f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 736.91 us/op 582.80 us/op 1.26
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 52.310 us/op 47.418 us/op 1.10
BLS verify - blst-native 1.2483 ms/op 1.2367 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.5539 ms/op 2.5355 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.4358 ms/op 5.4240 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 19.831 ms/op 19.947 ms/op 0.99
BLS aggregatePubkeys 32 - blst-native 26.716 us/op 26.403 us/op 1.01
BLS aggregatePubkeys 128 - blst-native 104.23 us/op 102.59 us/op 1.02
getAttestationsForBlock 67.015 ms/op 57.316 ms/op 1.17
isKnown best case - 1 super set check 279.00 ns/op 258.00 ns/op 1.08
isKnown normal case - 2 super set checks 268.00 ns/op 256.00 ns/op 1.05
isKnown worse case - 16 super set checks 265.00 ns/op 256.00 ns/op 1.04
CheckpointStateCache - add get delete 5.8800 us/op 5.0360 us/op 1.17
validate gossip signedAggregateAndProof - struct 2.8572 ms/op 2.7976 ms/op 1.02
validate gossip attestation - struct 1.3689 ms/op 1.3351 ms/op 1.03
pickEth1Vote - no votes 1.4290 ms/op 1.3224 ms/op 1.08
pickEth1Vote - max votes 11.160 ms/op 10.312 ms/op 1.08
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.8319 ms/op 8.6933 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.998 ms/op 15.630 ms/op 1.22
pickEth1Vote - Eth1Data fastSerialize value x2048 768.06 us/op 655.19 us/op 1.17
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.6769 ms/op 4.7735 ms/op 1.19
bytes32 toHexString 598.00 ns/op 502.00 ns/op 1.19
bytes32 Buffer.toString(hex) 427.00 ns/op 387.00 ns/op 1.10
bytes32 Buffer.toString(hex) from Uint8Array 684.00 ns/op 608.00 ns/op 1.13
bytes32 Buffer.toString(hex) + 0x 438.00 ns/op 385.00 ns/op 1.14
Object access 1 prop 0.19200 ns/op 0.16800 ns/op 1.14
Map access 1 prop 0.16900 ns/op 0.15800 ns/op 1.07
Object get x1000 6.9600 ns/op 6.4530 ns/op 1.08
Map get x1000 0.69500 ns/op 0.63200 ns/op 1.10
Object set x1000 71.988 ns/op 54.563 ns/op 1.32
Map set x1000 56.266 ns/op 45.029 ns/op 1.25
Return object 10000 times 0.28080 ns/op 0.24070 ns/op 1.17
Throw Error 10000 times 4.6021 us/op 4.2722 us/op 1.08
fastMsgIdFn sha256 / 200 bytes 3.6520 us/op 3.5330 us/op 1.03
fastMsgIdFn h32 xxhash / 200 bytes 329.00 ns/op 307.00 ns/op 1.07
fastMsgIdFn h64 xxhash / 200 bytes 488.00 ns/op 453.00 ns/op 1.08
fastMsgIdFn sha256 / 1000 bytes 13.319 us/op 11.877 us/op 1.12
fastMsgIdFn h32 xxhash / 1000 bytes 480.00 ns/op 444.00 ns/op 1.08
fastMsgIdFn h64 xxhash / 1000 bytes 577.00 ns/op 497.00 ns/op 1.16
fastMsgIdFn sha256 / 10000 bytes 110.27 us/op 104.42 us/op 1.06
fastMsgIdFn h32 xxhash / 10000 bytes 2.1010 us/op 1.9650 us/op 1.07
fastMsgIdFn h64 xxhash / 10000 bytes 1.4910 us/op 1.4350 us/op 1.04
enrSubnets - fastDeserialize 64 bits 1.7200 us/op 1.5440 us/op 1.11
enrSubnets - ssz BitVector 64 bits 603.00 ns/op 573.00 ns/op 1.05
enrSubnets - fastDeserialize 4 bits 214.00 ns/op 193.00 ns/op 1.11
enrSubnets - ssz BitVector 4 bits 635.00 ns/op 568.00 ns/op 1.12
prioritizePeers score -10:0 att 32-0.1 sync 2-0 126.54 us/op 136.55 us/op 0.93
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 174.59 us/op 149.15 us/op 1.17
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 212.08 us/op 184.55 us/op 1.15
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 379.48 us/op 365.61 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 454.60 us/op 429.36 us/op 1.06
array of 16000 items push then shift 1.7984 us/op 1.6736 us/op 1.07
LinkedList of 16000 items push then shift 9.7820 ns/op 9.0760 ns/op 1.08
array of 16000 items push then pop 118.95 ns/op 105.07 ns/op 1.13
LinkedList of 16000 items push then pop 9.1540 ns/op 8.8730 ns/op 1.03
array of 24000 items push then shift 2.4653 us/op 2.4396 us/op 1.01
LinkedList of 24000 items push then shift 9.4390 ns/op 9.3760 ns/op 1.01
array of 24000 items push then pop 86.408 ns/op 84.169 ns/op 1.03
LinkedList of 24000 items push then pop 9.1600 ns/op 8.9610 ns/op 1.02
intersect bitArray bitLen 8 14.131 ns/op 13.626 ns/op 1.04
intersect array and set length 8 87.042 ns/op 80.898 ns/op 1.08
intersect bitArray bitLen 128 45.995 ns/op 44.903 ns/op 1.02
intersect array and set length 128 1.2054 us/op 1.0916 us/op 1.10
Buffer.concat 32 items 3.0520 us/op 2.7020 us/op 1.13
Uint8Array.set 32 items 3.0520 us/op 2.2210 us/op 1.37
pass gossip attestations to forkchoice per slot 3.2101 ms/op 3.0223 ms/op 1.06
computeDeltas 3.3054 ms/op 3.0590 ms/op 1.08
computeProposerBoostScoreFromBalances 1.8565 ms/op 1.8228 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 2.8066 ms/op 2.4860 ms/op 1.13
altair processAttestation - 250000 vs - 7PWei worstcase 3.7479 ms/op 3.7923 ms/op 0.99
altair processAttestation - setStatus - 1/6 committees join 150.78 us/op 152.44 us/op 0.99
altair processAttestation - setStatus - 1/3 committees join 289.12 us/op 296.15 us/op 0.98
altair processAttestation - setStatus - 1/2 committees join 382.67 us/op 384.53 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 474.36 us/op 482.61 us/op 0.98
altair processAttestation - setStatus - 4/5 committees join 673.14 us/op 685.63 us/op 0.98
altair processAttestation - setStatus - 100% committees join 776.46 us/op 795.06 us/op 0.98
altair processBlock - 250000 vs - 7PWei normalcase 19.221 ms/op 17.646 ms/op 1.09
altair processBlock - 250000 vs - 7PWei normalcase hashState 28.393 ms/op 25.944 ms/op 1.09
altair processBlock - 250000 vs - 7PWei worstcase 61.778 ms/op 53.275 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase hashState 75.091 ms/op 72.105 ms/op 1.04
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2683 ms/op 2.1154 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei worstcase 31.325 ms/op 30.769 ms/op 1.02
altair processEth1Data - 250000 vs - 7PWei normalcase 511.92 us/op 472.38 us/op 1.08
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 9.5460 us/op 8.7070 us/op 1.10
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 29.027 us/op 25.858 us/op 1.12
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 11.528 us/op 10.289 us/op 1.12
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 9.5430 us/op 7.6790 us/op 1.24
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 110.03 us/op 102.32 us/op 1.08
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 656.97 us/op 668.99 us/op 0.98
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.0326 ms/op 907.82 us/op 1.14
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 1.0265 ms/op 898.99 us/op 1.14
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.4681 ms/op 2.4786 ms/op 1.00
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.9009 ms/op 1.5140 ms/op 1.26
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 4.3142 ms/op 4.2446 ms/op 1.02
Tree 40 250000 create 340.41 ms/op 321.17 ms/op 1.06
Tree 40 250000 get(125000) 196.82 ns/op 192.80 ns/op 1.02
Tree 40 250000 set(125000) 1.1025 us/op 995.58 ns/op 1.11
Tree 40 250000 toArray() 21.834 ms/op 22.402 ms/op 0.97
Tree 40 250000 iterate all - toArray() + loop 21.732 ms/op 21.971 ms/op 0.99
Tree 40 250000 iterate all - get(i) 75.930 ms/op 76.429 ms/op 0.99
MutableVector 250000 create 11.274 ms/op 10.751 ms/op 1.05
MutableVector 250000 get(125000) 6.7420 ns/op 6.5410 ns/op 1.03
MutableVector 250000 set(125000) 280.77 ns/op 292.25 ns/op 0.96
MutableVector 250000 toArray() 3.8510 ms/op 4.0216 ms/op 0.96
MutableVector 250000 iterate all - toArray() + loop 3.6381 ms/op 3.8995 ms/op 0.93
MutableVector 250000 iterate all - get(i) 1.5481 ms/op 1.5679 ms/op 0.99
Array 250000 create 2.8391 ms/op 3.1348 ms/op 0.91
Array 250000 clone - spread 1.1379 ms/op 1.2059 ms/op 0.94
Array 250000 get(125000) 0.57700 ns/op 0.59000 ns/op 0.98
Array 250000 set(125000) 0.64700 ns/op 0.66400 ns/op 0.97
Array 250000 iterate all - loop 109.42 us/op 84.890 us/op 1.29
effectiveBalanceIncrements clone Uint8Array 300000 28.878 us/op 35.256 us/op 0.82
effectiveBalanceIncrements clone MutableVector 300000 357.00 ns/op 359.00 ns/op 0.99
effectiveBalanceIncrements rw all Uint8Array 300000 169.34 us/op 167.84 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 82.461 ms/op 84.030 ms/op 0.98
phase0 afterProcessEpoch - 250000 vs - 7PWei 117.93 ms/op 119.18 ms/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 43.704 ms/op 46.193 ms/op 0.95
altair processEpoch - mainnet_e81889 319.30 ms/op 342.49 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 70.942 ms/op 66.937 ms/op 1.06
mainnet_e81889 - altair processJustificationAndFinalization 22.596 us/op 19.285 us/op 1.17
mainnet_e81889 - altair processInactivityUpdates 7.5413 ms/op 6.3116 ms/op 1.19
mainnet_e81889 - altair processRewardsAndPenalties 78.900 ms/op 66.321 ms/op 1.19
mainnet_e81889 - altair processRegistryUpdates 3.2250 us/op 2.4290 us/op 1.33
mainnet_e81889 - altair processSlashings 658.00 ns/op 510.00 ns/op 1.29
mainnet_e81889 - altair processEth1DataReset 650.00 ns/op 555.00 ns/op 1.17
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6231 ms/op 1.2658 ms/op 1.28
mainnet_e81889 - altair processSlashingsReset 5.8920 us/op 4.1660 us/op 1.41
mainnet_e81889 - altair processRandaoMixesReset 6.5670 us/op 9.3450 us/op 0.70
mainnet_e81889 - altair processHistoricalRootsUpdate 1.9830 us/op 776.00 ns/op 2.56
mainnet_e81889 - altair processParticipationFlagUpdates 3.4970 us/op 2.7170 us/op 1.29
mainnet_e81889 - altair processSyncCommitteeUpdates 963.00 ns/op 538.00 ns/op 1.79
mainnet_e81889 - altair afterProcessEpoch 132.66 ms/op 140.81 ms/op 0.94
phase0 processEpoch - mainnet_e58758 388.82 ms/op 407.04 ms/op 0.96
mainnet_e58758 - phase0 beforeProcessEpoch 154.77 ms/op 161.28 ms/op 0.96
mainnet_e58758 - phase0 processJustificationAndFinalization 22.396 us/op 27.081 us/op 0.83
mainnet_e58758 - phase0 processRewardsAndPenalties 71.597 ms/op 68.396 ms/op 1.05
mainnet_e58758 - phase0 processRegistryUpdates 11.956 us/op 8.8610 us/op 1.35
mainnet_e58758 - phase0 processSlashings 1.2350 us/op 555.00 ns/op 2.23
mainnet_e58758 - phase0 processEth1DataReset 1.3410 us/op 634.00 ns/op 2.12
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.8891 ms/op 1.0737 ms/op 1.76
mainnet_e58758 - phase0 processSlashingsReset 6.2050 us/op 5.6450 us/op 1.10
mainnet_e58758 - phase0 processRandaoMixesReset 11.774 us/op 5.6100 us/op 2.10
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.9010 us/op 665.00 ns/op 2.86
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.911 us/op 4.3340 us/op 2.52
mainnet_e58758 - phase0 afterProcessEpoch 118.62 ms/op 103.90 ms/op 1.14
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.1273 ms/op 1.2790 ms/op 1.66
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.2995 ms/op 1.7130 ms/op 1.93
altair processInactivityUpdates - 250000 normalcase 34.522 ms/op 23.314 ms/op 1.48
altair processInactivityUpdates - 250000 worstcase 34.476 ms/op 32.914 ms/op 1.05
phase0 processRegistryUpdates - 250000 normalcase 16.398 us/op 11.136 us/op 1.47
phase0 processRegistryUpdates - 250000 badcase_full_deposits 466.03 us/op 296.42 us/op 1.57
phase0 processRegistryUpdates - 250000 worstcase 0.5 173.77 ms/op 144.32 ms/op 1.20
altair processRewardsAndPenalties - 250000 normalcase 76.070 ms/op 69.283 ms/op 1.10
altair processRewardsAndPenalties - 250000 worstcase 79.226 ms/op 80.707 ms/op 0.98
phase0 getAttestationDeltas - 250000 normalcase 9.2994 ms/op 8.9571 ms/op 1.04
phase0 getAttestationDeltas - 250000 worstcase 10.551 ms/op 9.0213 ms/op 1.17
phase0 processSlashings - 250000 worstcase 3.9796 ms/op 3.9344 ms/op 1.01
altair processSyncCommitteeUpdates - 250000 197.50 ms/op 201.07 ms/op 0.98
BeaconState.hashTreeRoot - No change 323.00 ns/op 419.00 ns/op 0.77
BeaconState.hashTreeRoot - 1 full validator 61.714 us/op 53.308 us/op 1.16
BeaconState.hashTreeRoot - 32 full validator 600.22 us/op 568.69 us/op 1.06
BeaconState.hashTreeRoot - 512 full validator 6.5709 ms/op 5.7692 ms/op 1.14
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 73.518 us/op 65.842 us/op 1.12
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 930.64 us/op 1.0458 ms/op 0.89
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 13.929 ms/op 12.841 ms/op 1.08
BeaconState.hashTreeRoot - 1 balances 57.256 us/op 50.865 us/op 1.13
BeaconState.hashTreeRoot - 32 balances 522.82 us/op 486.69 us/op 1.07
BeaconState.hashTreeRoot - 512 balances 5.0626 ms/op 4.9453 ms/op 1.02
BeaconState.hashTreeRoot - 250000 balances 92.023 ms/op 73.738 ms/op 1.25
aggregationBits - 2048 els - zipIndexesInBitList 23.373 us/op 22.443 us/op 1.04
regular array get 100000 times 36.670 us/op 35.159 us/op 1.04
wrappedArray get 100000 times 37.002 us/op 35.876 us/op 1.03
arrayWithProxy get 100000 times 18.595 ms/op 18.571 ms/op 1.00
ssz.Root.equals 642.00 ns/op 642.00 ns/op 1.00
byteArrayEquals 651.00 ns/op 661.00 ns/op 0.98
shuffle list - 16384 els 7.2958 ms/op 7.5527 ms/op 0.97
shuffle list - 250000 els 107.84 ms/op 111.20 ms/op 0.97
processSlot - 1 slots 10.319 us/op 10.953 us/op 0.94
processSlot - 32 slots 1.4987 ms/op 1.6528 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 40.703 ms/op 44.160 ms/op 0.92
getCommitteeAssignments - req 1 vs - 250000 vc 3.1456 ms/op 3.2454 ms/op 0.97
getCommitteeAssignments - req 100 vs - 250000 vc 4.6175 ms/op 4.7237 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.7756 ms/op 4.8346 ms/op 0.99
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.5800 ns/op 5.6300 ns/op 0.99
state getBlockRootAtSlot - 250000 vs - 7PWei 813.06 ns/op 823.22 ns/op 0.99
computeProposers - vc 250000 11.696 ms/op 12.106 ms/op 0.97
computeEpochShuffling - vc 250000 109.63 ms/op 118.47 ms/op 0.93
getNextSyncCommittee - vc 250000 199.33 ms/op 208.58 ms/op 0.96
computeSigningRoot for AttestationData 14.798 us/op 16.699 us/op 0.89
hash AttestationData serialized data then Buffer.toString(base64) 2.6489 us/op 2.6583 us/op 1.00
toHexString serialized data 1.1672 us/op 1.6126 us/op 0.72
Buffer.toString(base64) 357.97 ns/op 375.19 ns/op 0.95

by benchmarkbot/action

@@ -44,6 +47,9 @@ else if (network) {
else if (process.argv[2] === "dev") {
process.env.LODESTAR_PRESET = "minimal";
process.env.LODESTAR_NETWORK = "dev";
// "c-kzg" has hardcoded the mainnet value, do not use presets
// eslint-disable-next-line @typescript-eslint/naming-convention
setActivePreset(PresetName.minimal, {FIELD_ELEMENTS_PER_BLOB: 4096});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this in the production code path just for a local test of us is pretty ugly. Please use presetFile functionality in that test to override the values on the test runtime only.

Copy link
Contributor Author

@g11tech g11tech May 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this for running lodestar via --dev network flag on cli (mostly via prebuild/docker images), test override is a different file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you detail more who / where this is necessary? Is someone else dependant on this fix?

Copy link
Contributor Author

@g11tech g11tech May 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally its used in ethereumjs sims for 4844 (https://github.com/ethereumjs/ethereumjs-monorepo/blob/master/packages/client/test/sim/4844.md) , also we would want our --dev network to be 4844 compatible with EL clients

@dapplion dapplion merged commit f417c35 into unstable May 16, 2023
12 of 14 checks passed
@dapplion dapplion deleted the ftb/dev-preset branch May 16, 2023 05:41
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants