test: improve sim tests for some remaining feedback #6654
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
| Benchmark suite | Current: d1fc151 | Previous: 669239b | Ratio |
|---|---|---|---|
| phase0 getAttestationDeltas - 250000 worstcase | 19.376 ms/op | 5.7044 ms/op | 3.40 |
Full benchmark results
| Benchmark suite | Current: d1fc151 | Previous: 669239b | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 875.54 us/op | 1.1323 ms/op | 0.77 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 107.61 us/op | 111.90 us/op | 0.96 |
| BLS verify - blst-native | 1.3494 ms/op | 1.1196 ms/op | 1.21 |
| BLS verifyMultipleSignatures 3 - blst-native | 2.8196 ms/op | 2.3493 ms/op | 1.20 |
| BLS verifyMultipleSignatures 8 - blst-native | 6.2012 ms/op | 5.2425 ms/op | 1.18 |
| BLS verifyMultipleSignatures 32 - blst-native | 23.484 ms/op | 19.424 ms/op | 1.21 |
| BLS verifyMultipleSignatures 64 - blst-native | 46.187 ms/op | 38.217 ms/op | 1.21 |
| BLS verifyMultipleSignatures 128 - blst-native | 90.492 ms/op | 75.469 ms/op | 1.20 |
| BLS deserializing 10000 signatures | 950.91 ms/op | 823.72 ms/op | 1.15 |
| BLS deserializing 100000 signatures | 9.2696 s/op | 8.4494 s/op | 1.10 |
| BLS verifyMultipleSignatures - same message - 3 - blst-native | 1.3972 ms/op | 1.1357 ms/op | 1.23 |
| BLS verifyMultipleSignatures - same message - 8 - blst-native | 1.6057 ms/op | 1.2905 ms/op | 1.24 |
| BLS verifyMultipleSignatures - same message - 32 - blst-native | 2.5016 ms/op | 2.0201 ms/op | 1.24 |
| BLS verifyMultipleSignatures - same message - 64 - blst-native | 3.5995 ms/op | 3.8050 ms/op | 0.95 |
| BLS verifyMultipleSignatures - same message - 128 - blst-native | 5.6984 ms/op | 4.8925 ms/op | 1.16 |
| BLS aggregatePubkeys 32 - blst-native | 28.652 us/op | 23.030 us/op | 1.24 |
| BLS aggregatePubkeys 128 - blst-native | 108.98 us/op | 88.903 us/op | 1.23 |
| notSeenSlots=1 numMissedVotes=1 numBadVotes=10 | 70.034 ms/op | 48.941 ms/op | 1.43 |
| notSeenSlots=1 numMissedVotes=0 numBadVotes=4 | 72.819 ms/op | 47.780 ms/op | 1.52 |
| notSeenSlots=2 numMissedVotes=1 numBadVotes=10 | 55.921 ms/op | 30.884 ms/op | 1.81 |
| getSlashingsAndExits - default max | 201.98 us/op | 159.62 us/op | 1.27 |
| getSlashingsAndExits - 2k | 593.95 us/op | 464.24 us/op | 1.28 |
| proposeBlockBody type=full, size=empty | 6.9657 ms/op | 5.2791 ms/op | 1.32 |
| isKnown best case - 1 super set check | 569.00 ns/op | 235.00 ns/op | 2.42 |
| isKnown normal case - 2 super set checks | 476.00 ns/op | 233.00 ns/op | 2.04 |
| isKnown worse case - 16 super set checks | 463.00 ns/op | 222.00 ns/op | 2.09 |
| InMemoryCheckpointStateCache - add get delete | 7.7350 us/op | 3.8380 us/op | 2.02 |
| validate api signedAggregateAndProof - struct | 2.9380 ms/op | 2.4351 ms/op | 1.21 |
| validate gossip signedAggregateAndProof - struct | 2.8784 ms/op | 2.3955 ms/op | 1.20 |
| validate gossip attestation - vc 640000 | 1.4002 ms/op | 1.1466 ms/op | 1.22 |
| batch validate gossip attestation - vc 640000 - chunk 32 | 169.75 us/op | 141.45 us/op | 1.20 |
| batch validate gossip attestation - vc 640000 - chunk 64 | 146.87 us/op | 129.61 us/op | 1.13 |
| batch validate gossip attestation - vc 640000 - chunk 128 | 140.92 us/op | 122.57 us/op | 1.15 |
| batch validate gossip attestation - vc 640000 - chunk 256 | 135.23 us/op | 114.46 us/op | 1.18 |
| pickEth1Vote - no votes | 1.3557 ms/op | 920.86 us/op | 1.47 |
| pickEth1Vote - max votes | 8.2638 ms/op | 13.692 ms/op | 0.60 |
| pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 16.332 ms/op | 20.103 ms/op | 0.81 |
| pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 25.521 ms/op | 28.073 ms/op | 0.91 |
| pickEth1Vote - Eth1Data fastSerialize value x2048 | 809.48 us/op | 521.40 us/op | 1.55 |
| pickEth1Vote - Eth1Data fastSerialize tree x2048 | 4.6036 ms/op | 7.8683 ms/op | 0.59 |
| bytes32 toHexString | 557.00 ns/op | 382.00 ns/op | 1.46 |
| bytes32 Buffer.toString(hex) | 312.00 ns/op | 201.00 ns/op | 1.55 |
| bytes32 Buffer.toString(hex) from Uint8Array | 529.00 ns/op | 404.00 ns/op | 1.31 |
| bytes32 Buffer.toString(hex) + 0x | 310.00 ns/op | 222.00 ns/op | 1.40 |
| Object access 1 prop | 0.17600 ns/op | 0.13000 ns/op | 1.35 |
| Map access 1 prop | 0.15600 ns/op | 0.10000 ns/op | 1.56 |
| Object get x1000 | 7.5010 ns/op | 4.7520 ns/op | 1.58 |
| Map get x1000 | 0.82500 ns/op | 0.71400 ns/op | 1.16 |
| Object set x1000 | 56.435 ns/op | 29.365 ns/op | 1.92 |
| Map set x1000 | 47.809 ns/op | 22.370 ns/op | 2.14 |
| Return object 10000 times | 0.25030 ns/op | 0.21760 ns/op | 1.15 |
| Throw Error 10000 times | 4.0204 us/op | 2.7258 us/op | 1.47 |
| fastMsgIdFn sha256 / 200 bytes | 3.3960 us/op | 1.9280 us/op | 1.76 |
| fastMsgIdFn h32 xxhash / 200 bytes | 352.00 ns/op | 257.00 ns/op | 1.37 |
| fastMsgIdFn h64 xxhash / 200 bytes | 397.00 ns/op | 261.00 ns/op | 1.52 |
| fastMsgIdFn sha256 / 1000 bytes | 11.691 us/op | 6.0260 us/op | 1.94 |
| fastMsgIdFn h32 xxhash / 1000 bytes | 458.00 ns/op | 361.00 ns/op | 1.27 |
| fastMsgIdFn h64 xxhash / 1000 bytes | 464.00 ns/op | 343.00 ns/op | 1.35 |
| fastMsgIdFn sha256 / 10000 bytes | 104.18 us/op | 51.002 us/op | 2.04 |
| fastMsgIdFn h32 xxhash / 10000 bytes | 2.0090 us/op | 1.7560 us/op | 1.14 |
| fastMsgIdFn h64 xxhash / 10000 bytes | 1.4120 us/op | 1.1420 us/op | 1.24 |
| send data - 1000 256B messages | 20.564 ms/op | 12.427 ms/op | 1.65 |
| send data - 1000 512B messages | 28.624 ms/op | 19.675 ms/op | 1.45 |
| send data - 1000 1024B messages | 53.731 ms/op | 22.871 ms/op | 2.35 |
| send data - 1000 1200B messages | 43.062 ms/op | 30.836 ms/op | 1.40 |
| send data - 1000 2048B messages | 52.224 ms/op | 41.642 ms/op | 1.25 |
| send data - 1000 4096B messages | 48.384 ms/op | 35.701 ms/op | 1.36 |
| send data - 1000 16384B messages | 114.78 ms/op | 103.06 ms/op | 1.11 |
| send data - 1000 65536B messages | 486.11 ms/op | 393.54 ms/op | 1.24 |
| enrSubnets - fastDeserialize 64 bits | 1.5300 us/op | 870.00 ns/op | 1.76 |
| enrSubnets - ssz BitVector 64 bits | 540.00 ns/op | 322.00 ns/op | 1.68 |
| enrSubnets - fastDeserialize 4 bits | 258.00 ns/op | 128.00 ns/op | 2.02 |
| enrSubnets - ssz BitVector 4 bits | 534.00 ns/op | 345.00 ns/op | 1.55 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 239.59 us/op | 139.42 us/op | 1.72 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 335.01 us/op | 179.13 us/op | 1.87 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 416.53 us/op | 287.25 us/op | 1.45 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 683.27 us/op | 427.23 us/op | 1.60 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 880.30 us/op | 511.67 us/op | 1.72 |
| array of 16000 items push then shift | 1.8187 us/op | 1.5020 us/op | 1.21 |
| LinkedList of 16000 items push then shift | 9.8970 ns/op | 6.2450 ns/op | 1.58 |
| array of 16000 items push then pop | 110.53 ns/op | 110.52 ns/op | 1.00 |
| LinkedList of 16000 items push then pop | 10.177 ns/op | 7.4380 ns/op | 1.37 |
| array of 24000 items push then shift | 2.7080 us/op | 1.8335 us/op | 1.48 |
| LinkedList of 24000 items push then shift | 9.9800 ns/op | 7.7000 ns/op | 1.30 |
| array of 24000 items push then pop | 186.14 ns/op | 144.05 ns/op | 1.29 |
| LinkedList of 24000 items push then pop | 9.6460 ns/op | 5.9510 ns/op | 1.62 |
| intersect bitArray bitLen 8 | 5.9380 ns/op | 4.8970 ns/op | 1.21 |
| intersect array and set length 8 | 80.791 ns/op | 49.848 ns/op | 1.62 |
| intersect bitArray bitLen 128 | 35.987 ns/op | 29.251 ns/op | 1.23 |
| intersect array and set length 128 | 1.0670 us/op | 713.42 ns/op | 1.50 |
| bitArray.getTrueBitIndexes() bitLen 128 | 1.8840 us/op | 1.2970 us/op | 1.45 |
| bitArray.getTrueBitIndexes() bitLen 248 | 2.9470 us/op | 2.2290 us/op | 1.32 |
| bitArray.getTrueBitIndexes() bitLen 512 | 6.3870 us/op | 3.7470 us/op | 1.70 |
| Buffer.concat 32 items | 1.1500 us/op | 919.00 ns/op | 1.25 |
| Uint8Array.set 32 items | 2.4800 us/op | 2.4550 us/op | 1.01 |
| Set add up to 64 items then delete first | 5.1612 us/op | 1.9929 us/op | 2.59 |
| OrderedSet add up to 64 items then delete first | 7.1038 us/op | 2.8534 us/op | 2.49 |
| Set add up to 64 items then delete last | 5.4597 us/op | 2.0530 us/op | 2.66 |
| OrderedSet add up to 64 items then delete last | 6.2923 us/op | 3.0679 us/op | 2.05 |
| Set add up to 64 items then delete middle | 5.2789 us/op | 2.0841 us/op | 2.53 |
| OrderedSet add up to 64 items then delete middle | 8.2187 us/op | 5.1237 us/op | 1.60 |
| Set add up to 128 items then delete first | 10.367 us/op | 4.7850 us/op | 2.17 |
| OrderedSet add up to 128 items then delete first | 13.682 us/op | 7.0097 us/op | 1.95 |
| Set add up to 128 items then delete last | 9.4476 us/op | 3.8744 us/op | 2.44 |
| OrderedSet add up to 128 items then delete last | 13.486 us/op | 5.8344 us/op | 2.31 |
| Set add up to 128 items then delete middle | 9.6241 us/op | 4.0111 us/op | 2.40 |
| OrderedSet add up to 128 items then delete middle | 18.676 us/op | 10.897 us/op | 1.71 |
| Set add up to 256 items then delete first | 20.343 us/op | 8.0310 us/op | 2.53 |
| OrderedSet add up to 256 items then delete first | 30.444 us/op | 14.054 us/op | 2.17 |
| Set add up to 256 items then delete last | 21.941 us/op | 9.4283 us/op | 2.33 |
| OrderedSet add up to 256 items then delete last | 27.160 us/op | 11.886 us/op | 2.29 |
| Set add up to 256 items then delete middle | 21.314 us/op | 9.4149 us/op | 2.26 |
| OrderedSet add up to 256 items then delete middle | 52.059 us/op | 35.413 us/op | 1.47 |
| transfer serialized Status (84 B) | 2.0000 us/op | 1.4800 us/op | 1.35 |
| copy serialized Status (84 B) | 1.5440 us/op | 1.0410 us/op | 1.48 |
| transfer serialized SignedVoluntaryExit (112 B) | 2.2780 us/op | 1.5620 us/op | 1.46 |
| copy serialized SignedVoluntaryExit (112 B) | 1.6390 us/op | 1.1130 us/op | 1.47 |
| transfer serialized ProposerSlashing (416 B) | 3.3890 us/op | 1.9260 us/op | 1.76 |
| copy serialized ProposerSlashing (416 B) | 3.2940 us/op | 2.9560 us/op | 1.11 |
| transfer serialized Attestation (485 B) | 3.5960 us/op | 2.9380 us/op | 1.22 |
| copy serialized Attestation (485 B) | 3.2740 us/op | 2.6380 us/op | 1.24 |
| transfer serialized AttesterSlashing (33232 B) | 3.0110 us/op | 2.4970 us/op | 1.21 |
| copy serialized AttesterSlashing (33232 B) | 8.1140 us/op | 8.6090 us/op | 0.94 |
| transfer serialized Small SignedBeaconBlock (128000 B) | 3.2070 us/op | 2.9360 us/op | 1.09 |
| copy serialized Small SignedBeaconBlock (128000 B) | 26.277 us/op | 13.610 us/op | 1.93 |
| transfer serialized Avg SignedBeaconBlock (200000 B) | 3.6710 us/op | 2.6510 us/op | 1.38 |
| copy serialized Avg SignedBeaconBlock (200000 B) | 40.701 us/op | 15.155 us/op | 2.69 |
| transfer serialized BlobsSidecar (524380 B) | 3.9220 us/op | 2.5370 us/op | 1.55 |
| copy serialized BlobsSidecar (524380 B) | 105.76 us/op | 138.36 us/op | 0.76 |
| transfer serialized Big SignedBeaconBlock (1000000 B) | 4.6360 us/op | 3.1880 us/op | 1.45 |
| copy serialized Big SignedBeaconBlock (1000000 B) | 199.65 us/op | 168.36 us/op | 1.19 |
| pass gossip attestations to forkchoice per slot | 4.5922 ms/op | 2.7351 ms/op | 1.68 |
| forkChoice updateHead vc 100000 bc 64 eq 0 | 704.42 us/op | 489.79 us/op | 1.44 |
| forkChoice updateHead vc 600000 bc 64 eq 0 | 5.7267 ms/op | 2.7861 ms/op | 2.06 |
| forkChoice updateHead vc 1000000 bc 64 eq 0 | 7.3945 ms/op | 4.7562 ms/op | 1.55 |
| forkChoice updateHead vc 600000 bc 320 eq 0 | 4.3306 ms/op | 2.7645 ms/op | 1.57 |
| forkChoice updateHead vc 600000 bc 1200 eq 0 | 4.5015 ms/op | 2.9176 ms/op | 1.54 |
| forkChoice updateHead vc 600000 bc 7200 eq 0 | 5.9987 ms/op | 4.0037 ms/op | 1.50 |
| forkChoice updateHead vc 600000 bc 64 eq 1000 | 11.601 ms/op | 10.121 ms/op | 1.15 |
| forkChoice updateHead vc 600000 bc 64 eq 10000 | 12.691 ms/op | 11.143 ms/op | 1.14 |
| forkChoice updateHead vc 600000 bc 64 eq 300000 | 17.615 ms/op | 12.090 ms/op | 1.46 |
| computeDeltas 500000 validators 300 proto nodes | 7.1578 ms/op | 3.4742 ms/op | 2.06 |
| computeDeltas 500000 validators 1200 proto nodes | 7.1279 ms/op | 3.4384 ms/op | 2.07 |
| computeDeltas 500000 validators 7200 proto nodes | 6.8696 ms/op | 3.3030 ms/op | 2.08 |
| computeDeltas 750000 validators 300 proto nodes | 10.067 ms/op | 5.1525 ms/op | 1.95 |
| computeDeltas 750000 validators 1200 proto nodes | 10.357 ms/op | 5.2449 ms/op | 1.97 |
| computeDeltas 750000 validators 7200 proto nodes | 10.173 ms/op | 5.0659 ms/op | 2.01 |
| computeDeltas 1400000 validators 300 proto nodes | 19.518 ms/op | 10.141 ms/op | 1.92 |
| computeDeltas 1400000 validators 1200 proto nodes | 19.669 ms/op | 9.8342 ms/op | 2.00 |
| computeDeltas 1400000 validators 7200 proto nodes | 19.362 ms/op | 9.6649 ms/op | 2.00 |
| computeDeltas 2100000 validators 300 proto nodes | 27.428 ms/op | 15.607 ms/op | 1.76 |
| computeDeltas 2100000 validators 1200 proto nodes | 29.735 ms/op | 15.311 ms/op | 1.94 |
| computeDeltas 2100000 validators 7200 proto nodes | 27.840 ms/op | 14.403 ms/op | 1.93 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 2.3572 ms/op | 1.6522 ms/op | 1.43 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 3.3307 ms/op | 2.9627 ms/op | 1.12 |
| altair processAttestation - setStatus - 1/6 committees join | 195.69 us/op | 120.58 us/op | 1.62 |
| altair processAttestation - setStatus - 1/3 committees join | 360.13 us/op | 241.47 us/op | 1.49 |
| altair processAttestation - setStatus - 1/2 committees join | 487.94 us/op | 288.28 us/op | 1.69 |
| altair processAttestation - setStatus - 2/3 committees join | 651.50 us/op | 378.03 us/op | 1.72 |
| altair processAttestation - setStatus - 4/5 committees join | 819.19 us/op | 561.54 us/op | 1.46 |
| altair processAttestation - setStatus - 100% committees join | 987.22 us/op | 594.66 us/op | 1.66 |
| altair processBlock - 250000 vs - 7PWei normalcase | 8.0171 ms/op | 7.6770 ms/op | 1.04 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 35.951 ms/op | 34.930 ms/op | 1.03 |
| altair processBlock - 250000 vs - 7PWei worstcase | 45.264 ms/op | 36.133 ms/op | 1.25 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 95.812 ms/op | 95.826 ms/op | 1.00 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 2.9811 ms/op | 2.1381 ms/op | 1.39 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 30.600 ms/op | 28.775 ms/op | 1.06 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 576.66 us/op | 280.08 us/op | 2.06 |
| getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 | 7.7150 us/op | 14.373 us/op | 0.54 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 | 68.847 us/op | 32.972 us/op | 2.09 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 | 24.318 us/op | 17.727 us/op | 1.37 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 | 11.533 us/op | 9.0110 us/op | 1.28 |
| getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 | 219.85 us/op | 148.76 us/op | 1.48 |
| getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 | 1.5667 ms/op | 1.0563 ms/op | 1.48 |
| getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 2.5206 ms/op | 1.8290 ms/op | 1.38 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 2.2495 ms/op | 1.3441 ms/op | 1.67 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 3.5738 ms/op | 2.6521 ms/op | 1.35 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.9447 ms/op | 1.8722 ms/op | 1.57 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 5.7831 ms/op | 4.4778 ms/op | 1.29 |
| Tree 40 250000 create | 419.00 ms/op | 346.30 ms/op | 1.21 |
| Tree 40 250000 get(125000) | 217.82 ns/op | 123.45 ns/op | 1.76 |
| Tree 40 250000 set(125000) | 1.2120 us/op | 855.09 ns/op | 1.42 |
| Tree 40 250000 toArray() | 22.080 ms/op | 23.259 ms/op | 0.95 |
| Tree 40 250000 iterate all - toArray() + loop | 23.737 ms/op | 22.786 ms/op | 1.04 |
| Tree 40 250000 iterate all - get(i) | 75.726 ms/op | 50.373 ms/op | 1.50 |
| MutableVector 250000 create | 19.335 ms/op | 7.3393 ms/op | 2.63 |
| MutableVector 250000 get(125000) | 7.0290 ns/op | 5.6410 ns/op | 1.25 |
| MutableVector 250000 set(125000) | 299.41 ns/op | 219.81 ns/op | 1.36 |
| MutableVector 250000 toArray() | 3.9607 ms/op | 2.3133 ms/op | 1.71 |
| MutableVector 250000 iterate all - toArray() + loop | 3.9429 ms/op | 2.5731 ms/op | 1.53 |
| MutableVector 250000 iterate all - get(i) | 1.5877 ms/op | 1.3498 ms/op | 1.18 |
| Array 250000 create | 3.5099 ms/op | 2.3406 ms/op | 1.50 |
| Array 250000 clone - spread | 1.4063 ms/op | 1.3174 ms/op | 1.07 |
| Array 250000 get(125000) | 1.1750 ns/op | 1.0400 ns/op | 1.13 |
| Array 250000 set(125000) | 4.4860 ns/op | 1.2480 ns/op | 3.59 |
| Array 250000 iterate all - loop | 169.48 us/op | 158.06 us/op | 1.07 |
| effectiveBalanceIncrements clone Uint8Array 300000 | 38.545 us/op | 19.311 us/op | 2.00 |
| effectiveBalanceIncrements clone MutableVector 300000 | 407.00 ns/op | 397.00 ns/op | 1.03 |
| effectiveBalanceIncrements rw all Uint8Array 300000 | 200.95 us/op | 186.25 us/op | 1.08 |
| effectiveBalanceIncrements rw all MutableVector 300000 | 96.194 ms/op | 71.802 ms/op | 1.34 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 123.47 ms/op | 82.578 ms/op | 1.50 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 56.127 ms/op | 61.178 ms/op | 0.92 |
| altair processEpoch - mainnet_e81889 | 547.40 ms/op | 382.65 ms/op | 1.43 |
| mainnet_e81889 - altair beforeProcessEpoch | 87.998 ms/op | 77.872 ms/op | 1.13 |
| mainnet_e81889 - altair processJustificationAndFinalization | 17.227 us/op | 19.237 us/op | 0.90 |
| mainnet_e81889 - altair processInactivityUpdates | 7.6905 ms/op | 5.1091 ms/op | 1.51 |
| mainnet_e81889 - altair processRewardsAndPenalties | 63.875 ms/op | 73.555 ms/op | 0.87 |
| mainnet_e81889 - altair processRegistryUpdates | 2.4760 us/op | 2.7590 us/op | 0.90 |
| mainnet_e81889 - altair processSlashings | 620.00 ns/op | 692.00 ns/op | 0.90 |
| mainnet_e81889 - altair processEth1DataReset | 593.00 ns/op | 670.00 ns/op | 0.89 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.7063 ms/op | 1.5013 ms/op | 1.14 |
| mainnet_e81889 - altair processSlashingsReset | 4.5900 us/op | 3.2230 us/op | 1.42 |
| mainnet_e81889 - altair processRandaoMixesReset | 9.6090 us/op | 4.0420 us/op | 2.38 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 686.00 ns/op | 606.00 ns/op | 1.13 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 2.3830 us/op | 1.2010 us/op | 1.98 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 768.00 ns/op | 588.00 ns/op | 1.31 |
| mainnet_e81889 - altair afterProcessEpoch | 128.10 ms/op | 84.536 ms/op | 1.52 |
| capella processEpoch - mainnet_e217614 | 2.2150 s/op | 1.8646 s/op | 1.19 |
| mainnet_e217614 - capella beforeProcessEpoch | 530.36 ms/op | 408.19 ms/op | 1.30 |
| mainnet_e217614 - capella processJustificationAndFinalization | 17.929 us/op | 8.2570 us/op | 2.17 |
| mainnet_e217614 - capella processInactivityUpdates | 22.082 ms/op | 18.469 ms/op | 1.20 |
| mainnet_e217614 - capella processRewardsAndPenalties | 673.67 ms/op | 419.50 ms/op | 1.61 |
| mainnet_e217614 - capella processRegistryUpdates | 38.116 us/op | 23.545 us/op | 1.62 |
| mainnet_e217614 - capella processSlashings | 1.1770 us/op | 438.00 ns/op | 2.69 |
| mainnet_e217614 - capella processEth1DataReset | 1.1840 us/op | 296.00 ns/op | 4.00 |
| mainnet_e217614 - capella processEffectiveBalanceUpdates | 4.7465 ms/op | 3.1207 ms/op | 1.52 |
| mainnet_e217614 - capella processSlashingsReset | 7.9920 us/op | 2.0430 us/op | 3.91 |
| mainnet_e217614 - capella processRandaoMixesReset | 6.7370 us/op | 3.0940 us/op | 2.18 |
| mainnet_e217614 - capella processHistoricalRootsUpdate | 1.5530 us/op | 454.00 ns/op | 3.42 |
| mainnet_e217614 - capella processParticipationFlagUpdates | 4.4120 us/op | 1.2140 us/op | 3.63 |
| mainnet_e217614 - capella afterProcessEpoch | 398.19 ms/op | 202.73 ms/op | 1.96 |
| phase0 processEpoch - mainnet_e58758 | 562.63 ms/op | 405.16 ms/op | 1.39 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 181.83 ms/op | 105.55 ms/op | 1.72 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 18.873 us/op | 9.5230 us/op | 1.98 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 84.162 ms/op | 58.536 ms/op | 1.44 |
| mainnet_e58758 - phase0 processRegistryUpdates | 13.481 us/op | 6.5720 us/op | 2.05 |
| mainnet_e58758 - phase0 processSlashings | 601.00 ns/op | 359.00 ns/op | 1.67 |
| mainnet_e58758 - phase0 processEth1DataReset | 715.00 ns/op | 342.00 ns/op | 2.09 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 1.3999 ms/op | 1.5273 ms/op | 0.92 |
| mainnet_e58758 - phase0 processSlashingsReset | 4.1710 us/op | 3.1190 us/op | 1.34 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 4.9910 us/op | 2.6350 us/op | 1.89 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 804.00 ns/op | 493.00 ns/op | 1.63 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 6.0970 us/op | 3.2140 us/op | 1.90 |
| mainnet_e58758 - phase0 afterProcessEpoch | 108.81 ms/op | 64.044 ms/op | 1.70 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.4499 ms/op | 1.0185 ms/op | 1.42 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 1.8732 ms/op | 1.0837 ms/op | 1.73 |
| altair processInactivityUpdates - 250000 normalcase | 31.433 ms/op | 21.041 ms/op | 1.49 |
| altair processInactivityUpdates - 250000 worstcase | 36.239 ms/op | 24.106 ms/op | 1.50 |
| phase0 processRegistryUpdates - 250000 normalcase | 15.174 us/op | 6.7000 us/op | 2.26 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 492.06 us/op | 482.62 us/op | 1.02 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 139.44 ms/op | 132.77 ms/op | 1.05 |
| altair processRewardsAndPenalties - 250000 normalcase | 79.941 ms/op | 63.744 ms/op | 1.25 |
| altair processRewardsAndPenalties - 250000 worstcase | 74.045 ms/op | 62.377 ms/op | 1.19 |
| phase0 getAttestationDeltas - 250000 normalcase | 14.248 ms/op | 5.7884 ms/op | 2.46 |
| phase0 getAttestationDeltas - 250000 worstcase | 19.376 ms/op | 5.7044 ms/op | 3.40 |
| phase0 processSlashings - 250000 worstcase | 127.67 us/op | 78.709 us/op | 1.62 |
| altair processSyncCommitteeUpdates - 250000 | 176.82 ms/op | 110.37 ms/op | 1.60 |
| BeaconState.hashTreeRoot - No change | 648.00 ns/op | 267.00 ns/op | 2.43 |
| BeaconState.hashTreeRoot - 1 full validator | 147.08 us/op | 136.80 us/op | 1.08 |
| BeaconState.hashTreeRoot - 32 full validator | 1.5639 ms/op | 1.5123 ms/op | 1.03 |
| BeaconState.hashTreeRoot - 512 full validator | 16.990 ms/op | 16.324 ms/op | 1.04 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 172.95 us/op | 144.95 us/op | 1.19 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 2.3315 ms/op | 2.0147 ms/op | 1.16 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 31.789 ms/op | 32.985 ms/op | 0.96 |
| BeaconState.hashTreeRoot - 1 balances | 146.61 us/op | 143.11 us/op | 1.02 |
| BeaconState.hashTreeRoot - 32 balances | 1.3115 ms/op | 1.2595 ms/op | 1.04 |
| BeaconState.hashTreeRoot - 512 balances | 13.339 ms/op | 11.131 ms/op | 1.20 |
| BeaconState.hashTreeRoot - 250000 balances | 236.86 ms/op | 152.33 ms/op | 1.55 |
| aggregationBits - 2048 els - zipIndexesInBitList | 41.135 us/op | 19.257 us/op | 2.14 |
| byteArrayEquals 32 | 78.449 ns/op | 63.656 ns/op | 1.23 |
| Buffer.compare 32 | 59.286 ns/op | 38.507 ns/op | 1.54 |
| byteArrayEquals 1024 | 2.1918 us/op | 1.7210 us/op | 1.27 |
| Buffer.compare 1024 | 75.553 ns/op | 43.417 ns/op | 1.74 |
| byteArrayEquals 16384 | 35.104 us/op | 27.511 us/op | 1.28 |
| Buffer.compare 16384 | 283.90 ns/op | 229.25 ns/op | 1.24 |
| byteArrayEquals 123687377 | 268.29 ms/op | 189.02 ms/op | 1.42 |
| Buffer.compare 123687377 | 9.9790 ms/op | 4.8832 ms/op | 2.04 |
| byteArrayEquals 32 - diff last byte | 79.850 ns/op | 58.765 ns/op | 1.36 |
| Buffer.compare 32 - diff last byte | 62.062 ns/op | 39.213 ns/op | 1.58 |
| byteArrayEquals 1024 - diff last byte | 2.1731 us/op | 1.7524 us/op | 1.24 |
| Buffer.compare 1024 - diff last byte | 76.693 ns/op | 43.737 ns/op | 1.75 |
| byteArrayEquals 16384 - diff last byte | 34.883 us/op | 27.840 us/op | 1.25 |
| Buffer.compare 16384 - diff last byte | 279.42 ns/op | 220.47 ns/op | 1.27 |
| byteArrayEquals 123687377 - diff last byte | 261.51 ms/op | 207.19 ms/op | 1.26 |
| Buffer.compare 123687377 - diff last byte | 10.223 ms/op | 4.4224 ms/op | 2.31 |
| byteArrayEquals 32 - random bytes | 7.1030 ns/op | 4.5760 ns/op | 1.55 |
| Buffer.compare 32 - random bytes | 63.331 ns/op | 47.298 ns/op | 1.34 |
| byteArrayEquals 1024 - random bytes | 6.2000 ns/op | 4.5920 ns/op | 1.35 |
| Buffer.compare 1024 - random bytes | 62.675 ns/op | 47.624 ns/op | 1.32 |
| byteArrayEquals 16384 - random bytes | 6.4490 ns/op | 4.7710 ns/op | 1.35 |
| Buffer.compare 16384 - random bytes | 63.177 ns/op | 46.098 ns/op | 1.37 |
| byteArrayEquals 123687377 - random bytes | 9.2900 ns/op | 7.1700 ns/op | 1.30 |
| Buffer.compare 123687377 - random bytes | 74.130 ns/op | 48.150 ns/op | 1.54 |
| regular array get 100000 times | 45.978 us/op | 40.895 us/op | 1.12 |
| wrappedArray get 100000 times | 47.835 us/op | 41.151 us/op | 1.16 |
| arrayWithProxy get 100000 times | 17.640 ms/op | 11.329 ms/op | 1.56 |
| ssz.Root.equals | 57.486 ns/op | 55.401 ns/op | 1.04 |
| byteArrayEquals | 58.307 ns/op | 54.005 ns/op | 1.08 |
| Buffer.compare | 12.744 ns/op | 9.6400 ns/op | 1.32 |
| shuffle list - 16384 els | 9.0012 ms/op | 5.6118 ms/op | 1.60 |
| shuffle list - 250000 els | 133.81 ms/op | 83.585 ms/op | 1.60 |
| processSlot - 1 slots | 19.833 us/op | 11.284 us/op | 1.76 |
| processSlot - 32 slots | 4.7799 ms/op | 2.1886 ms/op | 2.18 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 64.710 ms/op | 47.723 ms/op | 1.36 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 2.7845 ms/op | 2.4390 ms/op | 1.14 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 4.0478 ms/op | 3.4977 ms/op | 1.16 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 4.4100 ms/op | 3.8150 ms/op | 1.16 |
| findModifiedValidators - 10000 modified validators | 417.82 ms/op | 288.80 ms/op | 1.45 |
| findModifiedValidators - 1000 modified validators | 240.17 ms/op | 158.39 ms/op | 1.52 |
| findModifiedValidators - 100 modified validators | 200.33 ms/op | 159.90 ms/op | 1.25 |
| findModifiedValidators - 10 modified validators | 215.86 ms/op | 156.50 ms/op | 1.38 |
| findModifiedValidators - 1 modified validators | 180.53 ms/op | 142.32 ms/op | 1.27 |
| findModifiedValidators - no difference | 209.72 ms/op | 161.24 ms/op | 1.30 |
| compare ViewDUs | 5.0225 s/op | 3.7193 s/op | 1.35 |
| compare each validator Uint8Array | 1.2745 s/op | 1.7513 s/op | 0.73 |
| compare ViewDU to Uint8Array | 1.3035 s/op | 823.04 ms/op | 1.58 |
| migrate state 1000000 validators, 24 modified, 0 new | 835.36 ms/op | 671.93 ms/op | 1.24 |
| migrate state 1000000 validators, 1700 modified, 1000 new | 1.2805 s/op | 896.71 ms/op | 1.43 |
| migrate state 1000000 validators, 3400 modified, 2000 new | 1.4554 s/op | 1.0667 s/op | 1.36 |
| migrate state 1500000 validators, 24 modified, 0 new | 838.77 ms/op | 729.76 ms/op | 1.15 |
| migrate state 1500000 validators, 1700 modified, 1000 new | 1.2910 s/op | 1.0584 s/op | 1.22 |
| migrate state 1500000 validators, 3400 modified, 2000 new | 1.5086 s/op | 1.2998 s/op | 1.16 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 5.3000 ns/op | 4.8300 ns/op | 1.10 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 1.1129 us/op | 735.44 ns/op | 1.51 |
| computeProposers - vc 250000 | 11.094 ms/op | 8.0411 ms/op | 1.38 |
| computeEpochShuffling - vc 250000 | 131.49 ms/op | 88.795 ms/op | 1.48 |
| getNextSyncCommittee - vc 250000 | 189.38 ms/op | 147.51 ms/op | 1.28 |
| computeSigningRoot for AttestationData | 30.391 us/op | 25.499 us/op | 1.19 |
| hash AttestationData serialized data then Buffer.toString(base64) | 2.4286 us/op | 1.4732 us/op | 1.65 |
| toHexString serialized data | 1.4059 us/op | 929.93 ns/op | 1.51 |
| Buffer.toString(base64) | 255.41 ns/op | 190.56 ns/op | 1.34 |
by benchmarkbot/action
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## unstable #6654 +/- ##
============================================
+ Coverage 61.69% 61.81% +0.12%
============================================
Files 556 556
Lines 58820 59050 +230
Branches 1887 1898 +11
============================================
+ Hits 36287 36502 +215
- Misses 22492 22505 +13
- Partials 41 43 +2 |
|
🎉 This PR is included in v1.18.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Improve the sim tests assertions.
Description
Steps to test or reproduce