Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: generalize blobs to data for extension ready for ils and/or data columns #6693

Merged
merged 1 commit into from
Jun 14, 2024
Merged

feat: generalize blobs to data for extension ready for ils and/or data columns #6693

merged 1 commit into from
Jun 14, 2024

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Apr 21, 2024

to avoid redoing the same work on different branches and have a clean way to extend data requirements

Base automatically changed from blobsource to unstable April 26, 2024 09:09
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 1, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: f9efe3a Previous: 79a008f Ratio
Buffer.compare 123687377 - diff last byte 3.7085 ms/op 11.597 ms/op 0.32
Full benchmark results
Benchmark suite Current: f9efe3a Previous: 79a008f Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 388.71 us/op 833.95 us/op 0.47
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 43.606 us/op 56.171 us/op 0.78
BLS verify - blst-native 1.1276 ms/op 1.2478 ms/op 0.90
BLS verifyMultipleSignatures 3 - blst-native 2.4170 ms/op 2.6527 ms/op 0.91
BLS verifyMultipleSignatures 8 - blst-native 5.3110 ms/op 5.7293 ms/op 0.93
BLS verifyMultipleSignatures 32 - blst-native 19.484 ms/op 21.174 ms/op 0.92
BLS verifyMultipleSignatures 64 - blst-native 38.858 ms/op 42.233 ms/op 0.92
BLS verifyMultipleSignatures 128 - blst-native 76.935 ms/op 83.381 ms/op 0.92
BLS deserializing 10000 signatures 803.87 ms/op 877.12 ms/op 0.92
BLS deserializing 100000 signatures 8.0561 s/op 8.8252 s/op 0.91
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1564 ms/op 1.3225 ms/op 0.87
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.3131 ms/op 1.4227 ms/op 0.92
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.0519 ms/op 2.2437 ms/op 0.91
BLS verifyMultipleSignatures - same message - 64 - blst-native 2.9813 ms/op 3.3226 ms/op 0.90
BLS verifyMultipleSignatures - same message - 128 - blst-native 6.3242 ms/op 6.3214 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 23.787 us/op 26.519 us/op 0.90
BLS aggregatePubkeys 128 - blst-native 96.601 us/op 98.487 us/op 0.98
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 68.482 ms/op 63.318 ms/op 1.08
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 64.429 ms/op 58.653 ms/op 1.10
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 33.260 ms/op 41.573 ms/op 0.80
getSlashingsAndExits - default max 110.12 us/op 93.943 us/op 1.17
getSlashingsAndExits - 2k 432.74 us/op 331.49 us/op 1.31
proposeBlockBody type=full, size=empty 5.1707 ms/op 6.8611 ms/op 0.75
isKnown best case - 1 super set check 753.00 ns/op 329.00 ns/op 2.29
isKnown normal case - 2 super set checks 787.00 ns/op 466.00 ns/op 1.69
isKnown worse case - 16 super set checks 873.00 ns/op 482.00 ns/op 1.81
InMemoryCheckpointStateCache - add get delete 6.1490 us/op 5.7610 us/op 1.07
validate api signedAggregateAndProof - struct 2.4693 ms/op 2.8071 ms/op 0.88
validate gossip signedAggregateAndProof - struct 2.4845 ms/op 2.7019 ms/op 0.92
validate gossip attestation - vc 640000 1.2668 ms/op 1.3269 ms/op 0.95
batch validate gossip attestation - vc 640000 - chunk 32 147.43 us/op 162.81 us/op 0.91
batch validate gossip attestation - vc 640000 - chunk 64 128.49 us/op 143.17 us/op 0.90
batch validate gossip attestation - vc 640000 - chunk 128 117.14 us/op 130.16 us/op 0.90
batch validate gossip attestation - vc 640000 - chunk 256 119.83 us/op 123.58 us/op 0.97
pickEth1Vote - no votes 1.0422 ms/op 1.0488 ms/op 0.99
pickEth1Vote - max votes 8.6632 ms/op 7.9891 ms/op 1.08
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.852 ms/op 14.818 ms/op 0.80
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 22.584 ms/op 23.854 ms/op 0.95
pickEth1Vote - Eth1Data fastSerialize value x2048 405.54 us/op 479.11 us/op 0.85
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.2657 ms/op 5.1068 ms/op 0.64
bytes32 toHexString 629.00 ns/op 422.00 ns/op 1.49
bytes32 Buffer.toString(hex) 444.00 ns/op 237.00 ns/op 1.87
bytes32 Buffer.toString(hex) from Uint8Array 517.00 ns/op 364.00 ns/op 1.42
bytes32 Buffer.toString(hex) + 0x 426.00 ns/op 247.00 ns/op 1.72
Object access 1 prop 0.32400 ns/op 0.13300 ns/op 2.44
Map access 1 prop 0.32300 ns/op 0.12900 ns/op 2.50
Object get x1000 5.3310 ns/op 5.9850 ns/op 0.89
Map get x1000 5.8560 ns/op 6.5630 ns/op 0.89
Object set x1000 26.027 ns/op 32.182 ns/op 0.81
Map set x1000 19.551 ns/op 21.657 ns/op 0.90
Return object 10000 times 0.29690 ns/op 0.28590 ns/op 1.04
Throw Error 10000 times 2.6787 us/op 3.3440 us/op 0.80
fastMsgIdFn sha256 / 200 bytes 2.0030 us/op 2.1890 us/op 0.92
fastMsgIdFn h32 xxhash / 200 bytes 439.00 ns/op 228.00 ns/op 1.93
fastMsgIdFn h64 xxhash / 200 bytes 455.00 ns/op 259.00 ns/op 1.76
fastMsgIdFn sha256 / 1000 bytes 5.9500 us/op 7.0500 us/op 0.84
fastMsgIdFn h32 xxhash / 1000 bytes 562.00 ns/op 345.00 ns/op 1.63
fastMsgIdFn h64 xxhash / 1000 bytes 535.00 ns/op 378.00 ns/op 1.42
fastMsgIdFn sha256 / 10000 bytes 50.789 us/op 62.629 us/op 0.81
fastMsgIdFn h32 xxhash / 10000 bytes 1.9670 us/op 1.8490 us/op 1.06
fastMsgIdFn h64 xxhash / 10000 bytes 1.3550 us/op 1.2320 us/op 1.10
send data - 1000 256B messages 9.7418 ms/op 11.757 ms/op 0.83
send data - 1000 512B messages 12.625 ms/op 16.479 ms/op 0.77
send data - 1000 1024B messages 21.616 ms/op 25.631 ms/op 0.84
send data - 1000 1200B messages 22.957 ms/op 26.106 ms/op 0.88
send data - 1000 2048B messages 29.210 ms/op 31.816 ms/op 0.92
send data - 1000 4096B messages 25.622 ms/op 31.423 ms/op 0.82
send data - 1000 16384B messages 66.236 ms/op 72.633 ms/op 0.91
send data - 1000 65536B messages 253.20 ms/op 189.60 ms/op 1.34
enrSubnets - fastDeserialize 64 bits 1.0620 us/op 1.0640 us/op 1.00
enrSubnets - ssz BitVector 64 bits 510.00 ns/op 342.00 ns/op 1.49
enrSubnets - fastDeserialize 4 bits 317.00 ns/op 144.00 ns/op 2.20
enrSubnets - ssz BitVector 4 bits 517.00 ns/op 353.00 ns/op 1.46
prioritizePeers score -10:0 att 32-0.1 sync 2-0 125.17 us/op 160.84 us/op 0.78
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 141.45 us/op 141.91 us/op 1.00
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 288.04 us/op 391.30 us/op 0.74
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 521.91 us/op 536.76 us/op 0.97
prioritizePeers score 0:0 att 64-1 sync 4-1 529.54 us/op 858.86 us/op 0.62
array of 16000 items push then shift 1.2923 us/op 1.6176 us/op 0.80
LinkedList of 16000 items push then shift 6.4870 ns/op 6.9310 ns/op 0.94
array of 16000 items push then pop 77.452 ns/op 107.24 ns/op 0.72
LinkedList of 16000 items push then pop 6.2990 ns/op 6.8380 ns/op 0.92
array of 24000 items push then shift 1.8896 us/op 2.3653 us/op 0.80
LinkedList of 24000 items push then shift 6.5220 ns/op 7.1240 ns/op 0.92
array of 24000 items push then pop 104.58 ns/op 142.02 ns/op 0.74
LinkedList of 24000 items push then pop 6.1660 ns/op 6.9350 ns/op 0.89
intersect bitArray bitLen 8 5.4020 ns/op 6.3350 ns/op 0.85
intersect array and set length 8 39.468 ns/op 43.129 ns/op 0.92
intersect bitArray bitLen 128 26.365 ns/op 29.685 ns/op 0.89
intersect array and set length 128 586.04 ns/op 652.90 ns/op 0.90
bitArray.getTrueBitIndexes() bitLen 128 1.8790 us/op 2.1860 us/op 0.86
bitArray.getTrueBitIndexes() bitLen 248 2.7290 us/op 3.6820 us/op 0.74
bitArray.getTrueBitIndexes() bitLen 512 5.0970 us/op 8.9240 us/op 0.57
Buffer.concat 32 items 1.0160 us/op 903.00 ns/op 1.13
Uint8Array.set 32 items 1.2080 us/op 1.4520 us/op 0.83
Buffer.copy 1.3610 us/op 1.7930 us/op 0.76
Uint8Array.set - with subarray 1.9480 us/op 2.5020 us/op 0.78
Uint8Array.set - without subarray 1.1860 us/op 1.4000 us/op 0.85
Set add up to 64 items then delete first 1.8158 us/op 2.1843 us/op 0.83
OrderedSet add up to 64 items then delete first 2.8442 us/op 3.3676 us/op 0.84
Set add up to 64 items then delete last 2.0586 us/op 2.4991 us/op 0.82
OrderedSet add up to 64 items then delete last 3.1334 us/op 3.8725 us/op 0.81
Set add up to 64 items then delete middle 1.8819 us/op 2.5099 us/op 0.75
OrderedSet add up to 64 items then delete middle 4.1567 us/op 5.3161 us/op 0.78
Set add up to 128 items then delete first 3.7335 us/op 5.0901 us/op 0.73
OrderedSet add up to 128 items then delete first 5.8783 us/op 7.2938 us/op 0.81
Set add up to 128 items then delete last 3.5867 us/op 5.0363 us/op 0.71
OrderedSet add up to 128 items then delete last 5.5333 us/op 7.6692 us/op 0.72
Set add up to 128 items then delete middle 3.6020 us/op 4.9403 us/op 0.73
OrderedSet add up to 128 items then delete middle 10.936 us/op 13.530 us/op 0.81
Set add up to 256 items then delete first 7.3802 us/op 10.174 us/op 0.73
OrderedSet add up to 256 items then delete first 11.864 us/op 15.311 us/op 0.77
Set add up to 256 items then delete last 7.1377 us/op 10.014 us/op 0.71
OrderedSet add up to 256 items then delete last 11.204 us/op 15.630 us/op 0.72
Set add up to 256 items then delete middle 7.1647 us/op 9.9302 us/op 0.72
OrderedSet add up to 256 items then delete middle 32.333 us/op 40.250 us/op 0.80
transfer serialized Status (84 B) 1.4700 us/op 1.3640 us/op 1.08
copy serialized Status (84 B) 1.2130 us/op 1.0860 us/op 1.12
transfer serialized SignedVoluntaryExit (112 B) 1.5690 us/op 1.4430 us/op 1.09
copy serialized SignedVoluntaryExit (112 B) 1.2890 us/op 1.1920 us/op 1.08
transfer serialized ProposerSlashing (416 B) 2.4690 us/op 1.9460 us/op 1.27
copy serialized ProposerSlashing (416 B) 2.1870 us/op 2.4370 us/op 0.90
transfer serialized Attestation (485 B) 2.3120 us/op 2.7760 us/op 0.83
copy serialized Attestation (485 B) 2.0170 us/op 2.7890 us/op 0.72
transfer serialized AttesterSlashing (33232 B) 2.6380 us/op 2.7500 us/op 0.96
copy serialized AttesterSlashing (33232 B) 5.7200 us/op 6.0150 us/op 0.95
transfer serialized Small SignedBeaconBlock (128000 B) 3.1090 us/op 3.2390 us/op 0.96
copy serialized Small SignedBeaconBlock (128000 B) 14.275 us/op 15.743 us/op 0.91
transfer serialized Avg SignedBeaconBlock (200000 B) 3.2050 us/op 3.6950 us/op 0.87
copy serialized Avg SignedBeaconBlock (200000 B) 19.970 us/op 22.120 us/op 0.90
transfer serialized BlobsSidecar (524380 B) 3.0240 us/op 2.8020 us/op 1.08
copy serialized BlobsSidecar (524380 B) 72.552 us/op 86.776 us/op 0.84
transfer serialized Big SignedBeaconBlock (1000000 B) 3.3150 us/op 2.7570 us/op 1.20
copy serialized Big SignedBeaconBlock (1000000 B) 334.70 us/op 162.14 us/op 2.06
pass gossip attestations to forkchoice per slot 2.6181 ms/op 3.0460 ms/op 0.86
forkChoice updateHead vc 100000 bc 64 eq 0 438.31 us/op 591.27 us/op 0.74
forkChoice updateHead vc 600000 bc 64 eq 0 2.5000 ms/op 3.0077 ms/op 0.83
forkChoice updateHead vc 1000000 bc 64 eq 0 4.0591 ms/op 5.2597 ms/op 0.77
forkChoice updateHead vc 600000 bc 320 eq 0 2.5313 ms/op 3.0898 ms/op 0.82
forkChoice updateHead vc 600000 bc 1200 eq 0 2.3588 ms/op 3.0462 ms/op 0.77
forkChoice updateHead vc 600000 bc 7200 eq 0 2.8122 ms/op 3.6595 ms/op 0.77
forkChoice updateHead vc 600000 bc 64 eq 1000 9.4866 ms/op 10.523 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 10000 9.3987 ms/op 10.266 ms/op 0.92
forkChoice updateHead vc 600000 bc 64 eq 300000 11.569 ms/op 13.873 ms/op 0.83
computeDeltas 500000 validators 300 proto nodes 3.0042 ms/op 3.4315 ms/op 0.88
computeDeltas 500000 validators 1200 proto nodes 2.9577 ms/op 3.5891 ms/op 0.82
computeDeltas 500000 validators 7200 proto nodes 2.8957 ms/op 3.7876 ms/op 0.76
computeDeltas 750000 validators 300 proto nodes 4.6800 ms/op 5.3141 ms/op 0.88
computeDeltas 750000 validators 1200 proto nodes 4.6551 ms/op 5.3298 ms/op 0.87
computeDeltas 750000 validators 7200 proto nodes 4.5660 ms/op 5.4053 ms/op 0.84
computeDeltas 1400000 validators 300 proto nodes 8.5752 ms/op 10.523 ms/op 0.81
computeDeltas 1400000 validators 1200 proto nodes 8.8554 ms/op 10.430 ms/op 0.85
computeDeltas 1400000 validators 7200 proto nodes 8.6474 ms/op 10.625 ms/op 0.81
computeDeltas 2100000 validators 300 proto nodes 12.843 ms/op 15.185 ms/op 0.85
computeDeltas 2100000 validators 1200 proto nodes 13.328 ms/op 15.373 ms/op 0.87
computeDeltas 2100000 validators 7200 proto nodes 12.892 ms/op 15.884 ms/op 0.81
altair processAttestation - 250000 vs - 7PWei normalcase 1.9094 ms/op 2.0140 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei worstcase 3.0036 ms/op 3.1434 ms/op 0.96
altair processAttestation - setStatus - 1/6 committees join 81.334 us/op 95.102 us/op 0.86
altair processAttestation - setStatus - 1/3 committees join 159.98 us/op 180.71 us/op 0.89
altair processAttestation - setStatus - 1/2 committees join 191.68 us/op 265.85 us/op 0.72
altair processAttestation - setStatus - 2/3 committees join 274.13 us/op 380.42 us/op 0.72
altair processAttestation - setStatus - 4/5 committees join 388.37 us/op 630.01 us/op 0.62
altair processAttestation - setStatus - 100% committees join 480.71 us/op 630.36 us/op 0.76
altair processBlock - 250000 vs - 7PWei normalcase 2.9866 ms/op 4.6620 ms/op 0.64
altair processBlock - 250000 vs - 7PWei normalcase hashState 20.723 ms/op 35.393 ms/op 0.59
altair processBlock - 250000 vs - 7PWei worstcase 38.698 ms/op 44.712 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase hashState 88.001 ms/op 93.354 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9839 ms/op 2.3917 ms/op 0.83
phase0 processBlock - 250000 vs - 7PWei worstcase 22.951 ms/op 29.674 ms/op 0.77
altair processEth1Data - 250000 vs - 7PWei normalcase 242.07 us/op 326.65 us/op 0.74
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 4.8180 us/op 6.6400 us/op 0.73
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 20.654 us/op 39.964 us/op 0.52
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 7.1070 us/op 13.259 us/op 0.54
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.1500 us/op 9.2520 us/op 0.56
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 75.627 us/op 151.03 us/op 0.50
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 756.13 us/op 902.02 us/op 0.84
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.1408 ms/op 1.1800 ms/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 914.53 us/op 1.3445 ms/op 0.68
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 1.9739 ms/op 2.9854 ms/op 0.66
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.2623 ms/op 2.0715 ms/op 0.61
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 2.9330 ms/op 4.7484 ms/op 0.62
Tree 40 250000 create 178.64 ms/op 375.08 ms/op 0.48
Tree 40 250000 get(125000) 101.30 ns/op 166.56 ns/op 0.61
Tree 40 250000 set(125000) 540.30 ns/op 1.1776 us/op 0.46
Tree 40 250000 toArray() 14.631 ms/op 24.204 ms/op 0.60
Tree 40 250000 iterate all - toArray() + loop 9.9636 ms/op 23.558 ms/op 0.42
Tree 40 250000 iterate all - get(i) 41.157 ms/op 61.863 ms/op 0.67
MutableVector 250000 create 7.6827 ms/op 13.937 ms/op 0.55
MutableVector 250000 get(125000) 5.7730 ns/op 6.4410 ns/op 0.90
MutableVector 250000 set(125000) 179.79 ns/op 247.39 ns/op 0.73
MutableVector 250000 toArray() 2.5953 ms/op 5.6296 ms/op 0.46
MutableVector 250000 iterate all - toArray() + loop 2.6854 ms/op 5.3455 ms/op 0.50
MutableVector 250000 iterate all - get(i) 1.4057 ms/op 1.6770 ms/op 0.84
Array 250000 create 2.3366 ms/op 3.8123 ms/op 0.61
Array 250000 clone - spread 1.3167 ms/op 1.6623 ms/op 0.79
Array 250000 get(125000) 0.57400 ns/op 0.45600 ns/op 1.26
Array 250000 set(125000) 0.58600 ns/op 0.48800 ns/op 1.20
Array 250000 iterate all - loop 76.867 us/op 95.503 us/op 0.80
effectiveBalanceIncrements clone Uint8Array 300000 14.175 us/op 59.593 us/op 0.24
effectiveBalanceIncrements clone MutableVector 300000 311.00 ns/op 142.00 ns/op 2.19
effectiveBalanceIncrements rw all Uint8Array 300000 166.70 us/op 211.49 us/op 0.79
effectiveBalanceIncrements rw all MutableVector 300000 58.734 ms/op 96.278 ms/op 0.61
phase0 afterProcessEpoch - 250000 vs - 7PWei 75.601 ms/op 92.346 ms/op 0.82
phase0 beforeProcessEpoch - 250000 vs - 7PWei 34.737 ms/op 45.496 ms/op 0.76
altair processEpoch - mainnet_e81889 361.86 ms/op 400.15 ms/op 0.90
mainnet_e81889 - altair beforeProcessEpoch 56.701 ms/op 52.582 ms/op 1.08
mainnet_e81889 - altair processJustificationAndFinalization 10.863 us/op 17.869 us/op 0.61
mainnet_e81889 - altair processInactivityUpdates 4.1920 ms/op 6.9136 ms/op 0.61
mainnet_e81889 - altair processRewardsAndPenalties 45.636 ms/op 47.580 ms/op 0.96
mainnet_e81889 - altair processRegistryUpdates 1.7460 us/op 1.9980 us/op 0.87
mainnet_e81889 - altair processSlashings 811.00 ns/op 379.00 ns/op 2.14
mainnet_e81889 - altair processEth1DataReset 756.00 ns/op 345.00 ns/op 2.19
mainnet_e81889 - altair processEffectiveBalanceUpdates 831.41 us/op 1.6721 ms/op 0.50
mainnet_e81889 - altair processSlashingsReset 2.4080 us/op 4.2700 us/op 0.56
mainnet_e81889 - altair processRandaoMixesReset 3.1450 us/op 4.6480 us/op 0.68
mainnet_e81889 - altair processHistoricalRootsUpdate 676.00 ns/op 672.00 ns/op 1.01
mainnet_e81889 - altair processParticipationFlagUpdates 3.1430 us/op 1.9950 us/op 1.58
mainnet_e81889 - altair processSyncCommitteeUpdates 809.00 ns/op 614.00 ns/op 1.32
mainnet_e81889 - altair afterProcessEpoch 75.699 ms/op 93.713 ms/op 0.81
capella processEpoch - mainnet_e217614 1.2435 s/op 1.2355 s/op 1.01
mainnet_e217614 - capella beforeProcessEpoch 253.84 ms/op 252.41 ms/op 1.01
mainnet_e217614 - capella processJustificationAndFinalization 12.891 us/op 15.486 us/op 0.83
mainnet_e217614 - capella processInactivityUpdates 17.239 ms/op 17.725 ms/op 0.97
mainnet_e217614 - capella processRewardsAndPenalties 232.01 ms/op 271.42 ms/op 0.85
mainnet_e217614 - capella processRegistryUpdates 12.065 us/op 16.195 us/op 0.74
mainnet_e217614 - capella processSlashings 992.00 ns/op 484.00 ns/op 2.05
mainnet_e217614 - capella processEth1DataReset 871.00 ns/op 523.00 ns/op 1.67
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.2100 ms/op 12.292 ms/op 0.34
mainnet_e217614 - capella processSlashingsReset 4.6780 us/op 3.6700 us/op 1.27
mainnet_e217614 - capella processRandaoMixesReset 4.9010 us/op 4.9840 us/op 0.98
mainnet_e217614 - capella processHistoricalRootsUpdate 707.00 ns/op 813.00 ns/op 0.87
mainnet_e217614 - capella processParticipationFlagUpdates 3.7940 us/op 2.6840 us/op 1.41
mainnet_e217614 - capella afterProcessEpoch 257.95 ms/op 285.74 ms/op 0.90
phase0 processEpoch - mainnet_e58758 383.60 ms/op 408.64 ms/op 0.94
mainnet_e58758 - phase0 beforeProcessEpoch 122.63 ms/op 127.09 ms/op 0.96
mainnet_e58758 - phase0 processJustificationAndFinalization 13.951 us/op 16.549 us/op 0.84
mainnet_e58758 - phase0 processRewardsAndPenalties 28.373 ms/op 36.545 ms/op 0.78
mainnet_e58758 - phase0 processRegistryUpdates 7.4060 us/op 14.746 us/op 0.50
mainnet_e58758 - phase0 processSlashings 805.00 ns/op 405.00 ns/op 1.99
mainnet_e58758 - phase0 processEth1DataReset 771.00 ns/op 381.00 ns/op 2.02
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 837.07 us/op 1.6103 ms/op 0.52
mainnet_e58758 - phase0 processSlashingsReset 5.7910 us/op 3.4310 us/op 1.69
mainnet_e58758 - phase0 processRandaoMixesReset 8.8730 us/op 3.8090 us/op 2.33
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.0700 us/op 348.00 ns/op 3.07
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.2810 us/op 6.4150 us/op 0.67
mainnet_e58758 - phase0 afterProcessEpoch 70.458 ms/op 81.693 ms/op 0.86
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0917 ms/op 1.1681 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.3429 ms/op 2.7963 ms/op 0.48
altair processInactivityUpdates - 250000 normalcase 20.661 ms/op 22.361 ms/op 0.92
altair processInactivityUpdates - 250000 worstcase 18.930 ms/op 24.430 ms/op 0.77
phase0 processRegistryUpdates - 250000 normalcase 6.5890 us/op 9.6120 us/op 0.69
phase0 processRegistryUpdates - 250000 badcase_full_deposits 187.91 us/op 372.35 us/op 0.50
phase0 processRegistryUpdates - 250000 worstcase 0.5 108.63 ms/op 139.82 ms/op 0.78
altair processRewardsAndPenalties - 250000 normalcase 38.452 ms/op 41.751 ms/op 0.92
altair processRewardsAndPenalties - 250000 worstcase 50.132 ms/op 49.364 ms/op 1.02
phase0 getAttestationDeltas - 250000 normalcase 5.8839 ms/op 8.0626 ms/op 0.73
phase0 getAttestationDeltas - 250000 worstcase 6.0447 ms/op 8.3785 ms/op 0.72
phase0 processSlashings - 250000 worstcase 86.152 us/op 128.61 us/op 0.67
altair processSyncCommitteeUpdates - 250000 92.365 ms/op 133.55 ms/op 0.69
BeaconState.hashTreeRoot - No change 492.00 ns/op 412.00 ns/op 1.19
BeaconState.hashTreeRoot - 1 full validator 76.557 us/op 155.58 us/op 0.49
BeaconState.hashTreeRoot - 32 full validator 951.16 us/op 1.6675 ms/op 0.57
BeaconState.hashTreeRoot - 512 full validator 11.691 ms/op 17.111 ms/op 0.68
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 128.23 us/op 197.22 us/op 0.65
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5512 ms/op 1.9774 ms/op 0.78
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.061 ms/op 25.490 ms/op 0.79
BeaconState.hashTreeRoot - 1 balances 103.15 us/op 110.38 us/op 0.93
BeaconState.hashTreeRoot - 32 balances 776.41 us/op 1.1131 ms/op 0.70
BeaconState.hashTreeRoot - 512 balances 9.8924 ms/op 10.028 ms/op 0.99
BeaconState.hashTreeRoot - 250000 balances 128.49 ms/op 226.99 ms/op 0.57
aggregationBits - 2048 els - zipIndexesInBitList 19.572 us/op 26.597 us/op 0.74
byteArrayEquals 32 47.969 ns/op 56.084 ns/op 0.86
Buffer.compare 32 38.868 ns/op 47.447 ns/op 0.82
byteArrayEquals 1024 1.2615 us/op 1.6703 us/op 0.76
Buffer.compare 1024 47.691 ns/op 58.854 ns/op 0.81
byteArrayEquals 16384 19.615 us/op 27.142 us/op 0.72
Buffer.compare 16384 233.72 ns/op 236.00 ns/op 0.99
byteArrayEquals 123687377 153.33 ms/op 213.57 ms/op 0.72
Buffer.compare 123687377 4.1416 ms/op 9.7975 ms/op 0.42
byteArrayEquals 32 - diff last byte 47.415 ns/op 55.043 ns/op 0.86
Buffer.compare 32 - diff last byte 39.160 ns/op 52.732 ns/op 0.74
byteArrayEquals 1024 - diff last byte 1.1791 us/op 1.6612 us/op 0.71
Buffer.compare 1024 - diff last byte 42.566 ns/op 57.741 ns/op 0.74
byteArrayEquals 16384 - diff last byte 19.914 us/op 26.845 us/op 0.74
Buffer.compare 16384 - diff last byte 220.04 ns/op 240.57 ns/op 0.91
byteArrayEquals 123687377 - diff last byte 151.53 ms/op 208.96 ms/op 0.73
Buffer.compare 123687377 - diff last byte 3.7085 ms/op 11.597 ms/op 0.32
byteArrayEquals 32 - random bytes 4.8490 ns/op 5.9980 ns/op 0.81
Buffer.compare 32 - random bytes 41.517 ns/op 53.893 ns/op 0.77
byteArrayEquals 1024 - random bytes 4.8810 ns/op 5.9160 ns/op 0.83
Buffer.compare 1024 - random bytes 40.085 ns/op 52.503 ns/op 0.76
byteArrayEquals 16384 - random bytes 4.8910 ns/op 6.0450 ns/op 0.81
Buffer.compare 16384 - random bytes 39.934 ns/op 52.933 ns/op 0.75
byteArrayEquals 123687377 - random bytes 7.7400 ns/op 7.9100 ns/op 0.98
Buffer.compare 123687377 - random bytes 43.560 ns/op 58.570 ns/op 0.74
regular array get 100000 times 30.707 us/op 40.075 us/op 0.77
wrappedArray get 100000 times 30.709 us/op 37.941 us/op 0.81
arrayWithProxy get 100000 times 9.2036 ms/op 16.429 ms/op 0.56
ssz.Root.equals 39.256 ns/op 52.697 ns/op 0.74
byteArrayEquals 42.865 ns/op 51.450 ns/op 0.83
Buffer.compare 9.9000 ns/op 13.602 ns/op 0.73
shuffle list - 16384 els 5.5452 ms/op 7.3440 ms/op 0.76
shuffle list - 250000 els 81.337 ms/op 105.93 ms/op 0.77
processSlot - 1 slots 14.495 us/op 20.039 us/op 0.72
processSlot - 32 slots 3.0212 ms/op 4.3735 ms/op 0.69
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 42.831 ms/op 44.915 ms/op 0.95
getCommitteeAssignments - req 1 vs - 250000 vc 1.8773 ms/op 2.3623 ms/op 0.79
getCommitteeAssignments - req 100 vs - 250000 vc 3.4915 ms/op 4.5264 ms/op 0.77
getCommitteeAssignments - req 1000 vs - 250000 vc 3.7490 ms/op 4.9668 ms/op 0.75
findModifiedValidators - 10000 modified validators 235.93 ms/op 294.41 ms/op 0.80
findModifiedValidators - 1000 modified validators 169.43 ms/op 206.50 ms/op 0.82
findModifiedValidators - 100 modified validators 154.53 ms/op 214.97 ms/op 0.72
findModifiedValidators - 10 modified validators 146.89 ms/op 219.55 ms/op 0.67
findModifiedValidators - 1 modified validators 159.85 ms/op 183.69 ms/op 0.87
findModifiedValidators - no difference 157.00 ms/op 196.82 ms/op 0.80
compare ViewDUs 3.1625 s/op 3.6078 s/op 0.88
compare each validator Uint8Array 1.7039 s/op 1.4426 s/op 1.18
compare ViewDU to Uint8Array 851.00 ms/op 1.4072 s/op 0.60
migrate state 1000000 validators, 24 modified, 0 new 509.09 ms/op 728.37 ms/op 0.70
migrate state 1000000 validators, 1700 modified, 1000 new 786.02 ms/op 849.80 ms/op 0.92
migrate state 1000000 validators, 3400 modified, 2000 new 933.21 ms/op 1.0746 s/op 0.87
migrate state 1500000 validators, 24 modified, 0 new 554.38 ms/op 626.43 ms/op 0.88
migrate state 1500000 validators, 1700 modified, 1000 new 810.48 ms/op 876.69 ms/op 0.92
migrate state 1500000 validators, 3400 modified, 2000 new 921.94 ms/op 1.0674 s/op 0.86
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.3800 ns/op 4.6900 ns/op 1.36
state getBlockRootAtSlot - 250000 vs - 7PWei 895.57 ns/op 640.72 ns/op 1.40
computeProposers - vc 250000 5.8543 ms/op 8.2251 ms/op 0.71
computeEpochShuffling - vc 250000 84.212 ms/op 98.141 ms/op 0.86
getNextSyncCommittee - vc 250000 94.596 ms/op 126.05 ms/op 0.75
computeSigningRoot for AttestationData 16.826 us/op 17.651 us/op 0.95
hash AttestationData serialized data then Buffer.toString(base64) 1.1775 us/op 1.5922 us/op 0.74
toHexString serialized data 794.54 ns/op 979.90 ns/op 0.81
Buffer.toString(base64) 138.65 ns/op 199.72 ns/op 0.69

by benchmarkbot/action

@g11tech g11tech marked this pull request as ready for review June 1, 2024 15:37
@g11tech g11tech requested a review from a team as a code owner June 1, 2024 15:37
@twoeths
Copy link
Contributor

twoeths commented Jun 10, 2024

a note for reviewers, this PR also adds a new field "dataAvailabilityStatus" to forkchoice

@g11tech
feat: generalize blobs to data for extension ready for ils and/or dat…
7cec8ff 
…a columns

rename types and fix references to them

update forkchoice and availability flow

propagate changes to the codebase

lint and tsc

some fixes

cleanup

test run fixes

fix the tests

fix

apply feedback
@codecov Codecov
Copy link

codecov bot commented Jun 13, 2024

Codecov Report

Attention: Patch coverage is 42.28571% with 101 lines in your changes missing coverage. Please review.

Project coverage is 62.75%. Comparing base (79a008f) to head (7cec8ff).

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #6693      +/-   ##
============================================
- Coverage     62.75%   62.75%   -0.01%     
============================================
  Files           578      578              
  Lines         61280    61347      +67     
  Branches       2116     2116              
============================================
+ Hits          38459    38498      +39     
- Misses        22783    22811      +28     
  Partials         38       38

twoeths
twoeths approved these changes Jun 14, 2024
View reviewed changes
Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@g11tech g11tech merged commit d101913 into unstable Jun 14, 2024
19 of 20 checks passed
@g11tech g11tech deleted the blobstodata branch June 14, 2024 10:59
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.20.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@g11tech @twoeths @wemeetagain