Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EIP-4844: Add c-kzg library and current trusted setup #4851

Merged
merged 1 commit into from
Dec 5, 2022
Merged

EIP-4844: Add c-kzg library and current trusted setup #4851

merged 1 commit into from
Dec 5, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Dec 5, 2022

Motivation

EIP-4844 needs a new crypto backend, which will co-exist to BLST. It also needs access to a network specific trusted setup, which is a list of G1 and G2 points

Description

  • Add c-kzg library and current trusted setup

@dapplion dapplion requested a review from a team as a code owner December 5, 2022 10:02
@github-actions
Copy link
Contributor

github-actions bot commented Dec 5, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 8a351b6 Previous: 1fde79d Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.3218 ms/op 2.3952 ms/op 0.97
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 82.186 us/op 79.363 us/op 1.04
BLS verify - blst-native 2.7311 ms/op 1.9986 ms/op 1.37
BLS verifyMultipleSignatures 3 - blst-native 5.4415 ms/op 4.2332 ms/op 1.29
BLS verifyMultipleSignatures 8 - blst-native 11.929 ms/op 8.8027 ms/op 1.36
BLS verifyMultipleSignatures 32 - blst-native 43.759 ms/op 34.314 ms/op 1.28
BLS aggregatePubkeys 32 - blst-native 59.071 us/op 48.821 us/op 1.21
BLS aggregatePubkeys 128 - blst-native 228.55 us/op 163.45 us/op 1.40
getAttestationsForBlock 111.01 ms/op 102.00 ms/op 1.09
isKnown best case - 1 super set check 497.00 ns/op 436.00 ns/op 1.14
isKnown normal case - 2 super set checks 471.00 ns/op 421.00 ns/op 1.12
isKnown worse case - 16 super set checks 481.00 ns/op 442.00 ns/op 1.09
CheckpointStateCache - add get delete 10.272 us/op 9.4070 us/op 1.09
validate gossip signedAggregateAndProof - struct 5.9766 ms/op 4.7969 ms/op 1.25
validate gossip attestation - struct 2.8721 ms/op 2.0977 ms/op 1.37
pickEth1Vote - no votes 2.4850 ms/op 2.0058 ms/op 1.24
pickEth1Vote - max votes 23.933 ms/op 21.064 ms/op 1.14
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.901 ms/op 11.254 ms/op 1.24
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.871 ms/op 21.036 ms/op 1.13
pickEth1Vote - Eth1Data fastSerialize value x2048 1.7564 ms/op 1.6766 ms/op 1.05
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.986 ms/op 15.477 ms/op 0.97
bytes32 toHexString 1.2760 us/op 1.1290 us/op 1.13
bytes32 Buffer.toString(hex) 818.00 ns/op 746.00 ns/op 1.10
bytes32 Buffer.toString(hex) from Uint8Array 1.0800 us/op 1.0470 us/op 1.03
bytes32 Buffer.toString(hex) + 0x 781.00 ns/op 688.00 ns/op 1.14
Object access 1 prop 0.42300 ns/op 0.34100 ns/op 1.24
Map access 1 prop 0.33600 ns/op 0.27500 ns/op 1.22
Object get x1000 16.238 ns/op 13.368 ns/op 1.21
Map get x1000 0.96400 ns/op 0.75700 ns/op 1.27
Object set x1000 115.15 ns/op 85.592 ns/op 1.35
Map set x1000 79.948 ns/op 63.109 ns/op 1.27
Return object 10000 times 0.41330 ns/op 0.39670 ns/op 1.04
Throw Error 10000 times 7.0900 us/op 6.4669 us/op 1.10
fastMsgIdFn sha256 / 200 bytes 5.3870 us/op 5.1520 us/op 1.05
fastMsgIdFn h32 xxhash / 200 bytes 647.00 ns/op 579.00 ns/op 1.12
fastMsgIdFn h64 xxhash / 200 bytes 810.00 ns/op 721.00 ns/op 1.12
fastMsgIdFn sha256 / 1000 bytes 17.041 us/op 14.306 us/op 1.19
fastMsgIdFn h32 xxhash / 1000 bytes 791.00 ns/op 667.00 ns/op 1.19
fastMsgIdFn h64 xxhash / 1000 bytes 892.00 ns/op 881.00 ns/op 1.01
fastMsgIdFn sha256 / 10000 bytes 148.27 us/op 120.15 us/op 1.23
fastMsgIdFn h32 xxhash / 10000 bytes 2.5330 us/op 2.2780 us/op 1.11
fastMsgIdFn h64 xxhash / 10000 bytes 2.0990 us/op 1.7690 us/op 1.19
enrSubnets - fastDeserialize 64 bits 3.1580 us/op 2.7610 us/op 1.14
enrSubnets - ssz BitVector 64 bits 864.00 ns/op 861.00 ns/op 1.00
enrSubnets - fastDeserialize 4 bits 459.00 ns/op 364.00 ns/op 1.26
enrSubnets - ssz BitVector 4 bits 885.00 ns/op 695.00 ns/op 1.27
prioritizePeers score -10:0 att 32-0.1 sync 2-0 106.71 us/op 90.494 us/op 1.18
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 148.64 us/op 122.51 us/op 1.21
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 277.98 us/op 228.55 us/op 1.22
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 608.64 us/op 498.16 us/op 1.22
prioritizePeers score 0:0 att 64-1 sync 4-1 608.22 us/op 482.54 us/op 1.26
RateTracker 1000000 limit, 1 obj count per request 207.91 ns/op 163.35 ns/op 1.27
RateTracker 1000000 limit, 2 obj count per request 157.32 ns/op 116.16 ns/op 1.35
RateTracker 1000000 limit, 4 obj count per request 130.37 ns/op 97.823 ns/op 1.33
RateTracker 1000000 limit, 8 obj count per request 114.10 ns/op 89.142 ns/op 1.28
RateTracker with prune 5.3820 us/op 4.0900 us/op 1.32
array of 16000 items push then shift 5.2047 us/op 4.2745 us/op 1.22
LinkedList of 16000 items push then shift 20.119 ns/op 14.993 ns/op 1.34
array of 16000 items push then pop 256.06 ns/op 212.11 ns/op 1.21
LinkedList of 16000 items push then pop 19.020 ns/op 13.661 ns/op 1.39
array of 24000 items push then shift 7.5311 us/op 6.3622 us/op 1.18
LinkedList of 24000 items push then shift 19.872 ns/op 16.140 ns/op 1.23
array of 24000 items push then pop 245.05 ns/op 203.90 ns/op 1.20
LinkedList of 24000 items push then pop 18.532 ns/op 14.571 ns/op 1.27
intersect bitArray bitLen 8 12.052 ns/op 9.7590 ns/op 1.23
intersect array and set length 8 193.98 ns/op 152.06 ns/op 1.28
intersect bitArray bitLen 128 70.180 ns/op 57.488 ns/op 1.22
intersect array and set length 128 2.3874 us/op 1.7687 us/op 1.35
Buffer.concat 32 items 2.6650 ns/op 1.8720 ns/op 1.42
pass gossip attestations to forkchoice per slot 4.5501 ms/op 3.5583 ms/op 1.28
computeDeltas 5.9262 ms/op 5.1914 ms/op 1.14
computeProposerBoostScoreFromBalances 861.30 us/op 712.25 us/op 1.21
altair processAttestation - 250000 vs - 7PWei normalcase 6.2421 ms/op 3.7455 ms/op 1.67
altair processAttestation - 250000 vs - 7PWei worstcase 8.0394 ms/op 5.8901 ms/op 1.36
altair processAttestation - setStatus - 1/6 committees join 245.36 us/op 200.75 us/op 1.22
altair processAttestation - setStatus - 1/3 committees join 471.14 us/op 402.99 us/op 1.17
altair processAttestation - setStatus - 1/2 committees join 677.08 us/op 575.29 us/op 1.18
altair processAttestation - setStatus - 2/3 committees join 875.68 us/op 757.32 us/op 1.16
altair processAttestation - setStatus - 4/5 committees join 1.2175 ms/op 1.0110 ms/op 1.20
altair processAttestation - setStatus - 100% committees join 1.5811 ms/op 1.3092 ms/op 1.21
altair processBlock - 250000 vs - 7PWei normalcase 30.998 ms/op 26.738 ms/op 1.16
altair processBlock - 250000 vs - 7PWei normalcase hashState 42.701 ms/op 43.711 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase 119.05 ms/op 79.531 ms/op 1.50
altair processBlock - 250000 vs - 7PWei worstcase hashState 121.30 ms/op 98.296 ms/op 1.23
phase0 processBlock - 250000 vs - 7PWei normalcase 4.2761 ms/op 3.5202 ms/op 1.21
phase0 processBlock - 250000 vs - 7PWei worstcase 62.948 ms/op 43.190 ms/op 1.46
altair processEth1Data - 250000 vs - 7PWei normalcase 1.0529 ms/op 734.92 us/op 1.43
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 19.806 us/op 20.075 us/op 0.99
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 48.742 us/op 45.087 us/op 1.08
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 25.084 us/op 25.600 us/op 0.98
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 22.375 us/op 21.210 us/op 1.05
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 159.69 us/op 134.01 us/op 1.19
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 1.2902 ms/op 1.1139 ms/op 1.16
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 141069 18.156 ms/op 13.075 ms/op 1.39
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 250000 28.404 ms/op 23.415 ms/op 1.21
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 250000 98.320 ms/op 76.872 ms/op 1.28
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 250000 57.051 ms/op 44.208 ms/op 1.29
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 250000 144.84 ms/op 123.31 ms/op 1.17
Tree 40 250000 create 1.0618 s/op 757.50 ms/op 1.40
Tree 40 250000 get(125000) 330.67 ns/op 257.99 ns/op 1.28
Tree 40 250000 set(125000) 3.0944 us/op 2.1617 us/op 1.43
Tree 40 250000 toArray() 34.499 ms/op 32.374 ms/op 1.07
Tree 40 250000 iterate all - toArray() + loop 34.856 ms/op 32.960 ms/op 1.06
Tree 40 250000 iterate all - get(i) 133.42 ms/op 108.83 ms/op 1.23
MutableVector 250000 create 16.928 ms/op 13.369 ms/op 1.27
MutableVector 250000 get(125000) 14.524 ns/op 11.575 ns/op 1.25
MutableVector 250000 set(125000) 844.26 ns/op 545.94 ns/op 1.55
MutableVector 250000 toArray() 7.5527 ms/op 6.2579 ms/op 1.21
MutableVector 250000 iterate all - toArray() + loop 7.7005 ms/op 6.2043 ms/op 1.24
MutableVector 250000 iterate all - get(i) 3.6613 ms/op 2.6749 ms/op 1.37
Array 250000 create 7.5100 ms/op 5.6554 ms/op 1.33
Array 250000 clone - spread 4.3883 ms/op 3.1076 ms/op 1.41
Array 250000 get(125000) 1.7580 ns/op 1.3990 ns/op 1.26
Array 250000 set(125000) 1.8230 ns/op 1.3240 ns/op 1.38
Array 250000 iterate all - loop 137.67 us/op 119.56 us/op 1.15
effectiveBalanceIncrements clone Uint8Array 300000 216.02 us/op 64.298 us/op 3.36
effectiveBalanceIncrements clone MutableVector 300000 885.00 ns/op 934.00 ns/op 0.95
effectiveBalanceIncrements rw all Uint8Array 300000 292.62 us/op 242.30 us/op 1.21
effectiveBalanceIncrements rw all MutableVector 300000 215.17 ms/op 192.29 ms/op 1.12
phase0 afterProcessEpoch - 250000 vs - 7PWei 215.68 ms/op 165.34 ms/op 1.30
phase0 beforeProcessEpoch - 250000 vs - 7PWei 76.254 ms/op 99.949 ms/op 0.76
altair processEpoch - mainnet_e81889 665.14 ms/op 538.90 ms/op 1.23
mainnet_e81889 - altair beforeProcessEpoch 160.81 ms/op 130.98 ms/op 1.23
mainnet_e81889 - altair processJustificationAndFinalization 73.546 us/op 69.371 us/op 1.06
mainnet_e81889 - altair processInactivityUpdates 11.530 ms/op 9.3178 ms/op 1.24
mainnet_e81889 - altair processRewardsAndPenalties 99.278 ms/op 89.595 ms/op 1.11
mainnet_e81889 - altair processRegistryUpdates 12.754 us/op 14.945 us/op 0.85
mainnet_e81889 - altair processSlashings 4.0640 us/op 4.1410 us/op 0.98
mainnet_e81889 - altair processEth1DataReset 3.9400 us/op 4.3760 us/op 0.90
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4173 ms/op 1.9670 ms/op 1.23
mainnet_e81889 - altair processSlashingsReset 24.844 us/op 27.487 us/op 0.90
mainnet_e81889 - altair processRandaoMixesReset 22.564 us/op 25.159 us/op 0.90
mainnet_e81889 - altair processHistoricalRootsUpdate 4.3870 us/op 3.9940 us/op 1.10
mainnet_e81889 - altair processParticipationFlagUpdates 15.240 us/op 15.278 us/op 1.00
mainnet_e81889 - altair processSyncCommitteeUpdates 3.9760 us/op 3.5080 us/op 1.13
mainnet_e81889 - altair afterProcessEpoch 207.67 ms/op 172.46 ms/op 1.20
phase0 processEpoch - mainnet_e58758 633.12 ms/op 493.85 ms/op 1.28
mainnet_e58758 - phase0 beforeProcessEpoch 293.77 ms/op 260.13 ms/op 1.13
mainnet_e58758 - phase0 processJustificationAndFinalization 69.334 us/op 63.352 us/op 1.09
mainnet_e58758 - phase0 processRewardsAndPenalties 152.68 ms/op 112.07 ms/op 1.36
mainnet_e58758 - phase0 processRegistryUpdates 36.010 us/op 34.207 us/op 1.05
mainnet_e58758 - phase0 processSlashings 3.3000 us/op 3.9930 us/op 0.83
mainnet_e58758 - phase0 processEth1DataReset 3.5390 us/op 3.8360 us/op 0.92
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.7375 ms/op 1.9498 ms/op 1.40
mainnet_e58758 - phase0 processSlashingsReset 16.284 us/op 19.124 us/op 0.85
mainnet_e58758 - phase0 processRandaoMixesReset 25.201 us/op 26.058 us/op 0.97
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.7790 us/op 4.3150 us/op 1.11
mainnet_e58758 - phase0 processParticipationRecordUpdates 24.501 us/op 24.125 us/op 1.02
mainnet_e58758 - phase0 afterProcessEpoch 173.61 ms/op 134.75 ms/op 1.29
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.4530 ms/op 1.9822 ms/op 1.24
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.8191 ms/op 2.2041 ms/op 1.28
altair processInactivityUpdates - 250000 normalcase 49.359 ms/op 47.645 ms/op 1.04
altair processInactivityUpdates - 250000 worstcase 60.736 ms/op 55.703 ms/op 1.09
phase0 processRegistryUpdates - 250000 normalcase 27.519 us/op 29.695 us/op 0.93
phase0 processRegistryUpdates - 250000 badcase_full_deposits 511.31 us/op 448.64 us/op 1.14
phase0 processRegistryUpdates - 250000 worstcase 0.5 240.16 ms/op 228.74 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 144.18 ms/op 135.80 ms/op 1.06
altair processRewardsAndPenalties - 250000 worstcase 97.962 ms/op 80.118 ms/op 1.22
phase0 getAttestationDeltas - 250000 normalcase 14.614 ms/op 11.512 ms/op 1.27
phase0 getAttestationDeltas - 250000 worstcase 14.968 ms/op 11.690 ms/op 1.28
phase0 processSlashings - 250000 worstcase 7.5287 ms/op 5.8091 ms/op 1.30
altair processSyncCommitteeUpdates - 250000 345.99 ms/op 267.40 ms/op 1.29
BeaconState.hashTreeRoot - No change 602.00 ns/op 460.00 ns/op 1.31
BeaconState.hashTreeRoot - 1 full validator 74.695 us/op 61.382 us/op 1.22
BeaconState.hashTreeRoot - 32 full validator 749.66 us/op 624.11 us/op 1.20
BeaconState.hashTreeRoot - 512 full validator 8.3652 ms/op 6.7505 ms/op 1.24
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 109.68 us/op 84.040 us/op 1.31
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5431 ms/op 1.1719 ms/op 1.32
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.743 ms/op 14.657 ms/op 1.35
BeaconState.hashTreeRoot - 1 balances 81.399 us/op 58.336 us/op 1.40
BeaconState.hashTreeRoot - 32 balances 755.49 us/op 561.78 us/op 1.34
BeaconState.hashTreeRoot - 512 balances 7.5078 ms/op 5.7629 ms/op 1.30
BeaconState.hashTreeRoot - 250000 balances 119.39 ms/op 88.042 ms/op 1.36
aggregationBits - 2048 els - zipIndexesInBitList 34.113 us/op 21.679 us/op 1.57
regular array get 100000 times 56.346 us/op 45.457 us/op 1.24
wrappedArray get 100000 times 56.907 us/op 46.295 us/op 1.23
arrayWithProxy get 100000 times 35.282 ms/op 27.715 ms/op 1.27
ssz.Root.equals 607.00 ns/op 466.00 ns/op 1.30
byteArrayEquals 572.00 ns/op 413.00 ns/op 1.38
shuffle list - 16384 els 11.804 ms/op 9.1569 ms/op 1.29
shuffle list - 250000 els 177.26 ms/op 137.88 ms/op 1.29
processSlot - 1 slots 17.098 us/op 13.045 us/op 1.31
processSlot - 32 slots 2.3929 ms/op 1.8176 ms/op 1.32
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 673.68 us/op 357.18 us/op 1.89
getCommitteeAssignments - req 1 vs - 250000 vc 5.4148 ms/op 4.2755 ms/op 1.27
getCommitteeAssignments - req 100 vs - 250000 vc 7.7288 ms/op 6.1270 ms/op 1.26
getCommitteeAssignments - req 1000 vs - 250000 vc 8.3520 ms/op 6.4460 ms/op 1.30
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.2200 ns/op 7.8900 ns/op 1.17
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3007 us/op 924.44 ns/op 1.41
computeProposers - vc 250000 20.732 ms/op 15.780 ms/op 1.31
computeEpochShuffling - vc 250000 177.90 ms/op 144.12 ms/op 1.23
getNextSyncCommittee - vc 250000 335.98 ms/op 268.06 ms/op 1.25

by benchmarkbot/action

@dapplion dapplion added the spec-deneb Issues targeting the Deneb spec version label Dec 5, 2022
g11tech
g11tech reviewed Dec 5, 2022
View reviewed changes
packages/beacon-node/src/util/kzg.ts
// Solutions: https://stackoverflow.com/questions/46745014/alternative-for-dirname-in-node-js-when-using-es6-modules
// eslint-disable-next-line @typescript-eslint/naming-convention
const __dirname = path.dirname(fileURLToPath(import.meta.url));
export const TRUSTED_SETUP_BIN_FILEPATH = path.join(__dirname, "../../trusted_setup.bin");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this be network dependent or same setup will work for all networks (and then the question of chains like gnosis etc)

Copy link
Contributor Author

@dapplion dapplion Dec 5, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still unknown, probably it will be shared since it's only dependant on the size of blobs. I would bet big on being the same file for all eth2 networks

g11tech
g11tech approved these changes Dec 5, 2022
View reviewed changes
Copy link
Contributor

@g11tech g11tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, just a query regarding the trusted bin customization w.r.t. networks

@g11tech g11tech merged commit 29dd662 into unstable Dec 5, 2022
@g11tech g11tech deleted the dapplion/c-kzg branch December 5, 2022 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

@g11tech g11tech g11tech approved these changes

Assignees
No one assigned
Labels
spec-deneb Issues targeting the Deneb spec version
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@dapplion @twoeths @g11tech