Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update forkchoice using latestValidHash updates from execution engine #4182

Merged
merged 20 commits into from
Sep 2, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jun 22, 2022

Motivation
Post merge, engine can give optimistic syncing status on fcU heads which can (should be) eventually resolved into valid/invalid.
This need to be propagated to forkchoice and remove the consideration of invalids and their descendant from the forkchoice,

This PR implements the fcU updates coming from from the execution via latestValidHash

TODO testcases:

  • for propagate valid
    - [ ] verify is_merge_block if the mergeblock has not yet been validated moved to a separate issue as doesn't addressing this may be a bit more evolved, though EL already should have run terminal validations and hence de duping work isn't really necessary.

  • for propagate invalid

    • with latest hash found in forkchoice
    • with latesthash not found in forkchoice
  • real life optimistic sync testing against ELs

Closes #3508

Testcases summary:

  executionStatus / normal updates
    ✔ preValidation forkchoice setup should be correct
    ✔ correcly invalidate 3C and validate 2C only
    ✔ Validate 3B, 2B, 1A
    ✔ Invalidate 3A, 2A with 2A loosing its bestChild, bestDescendant

  executionStatus / invalidate all postmerge chain
    ✔ preValidation forkchoice setup should be correct
    ✔ all post merge blocks should be invalidated except Cs
    ✔ pre merge block should be the FC head

  executionStatus / poision forkchoice if we invalidate previous valid
    ✔ preValidation forkchoice setup should be correct
    ✔ Validate 3B, 2B, 1A
    ✔ protoarray should be poisioned with a buggy LVH response

  executionStatus / poision forkchoice if we validate previous invalid
    ✔ preValidation forkchoice setup should be correct
    ✔ Inalidate 3B, 2B, 1A
    ✔ protoarray should be poisioned with a buggy LVH response


  13 passing (8ms)

@github-actions
Copy link
Contributor

github-actions bot commented Jun 22, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: aefea00 Previous: 04030be Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.2030 ms/op 2.2266 ms/op 0.99
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 73.013 us/op 76.347 us/op 0.96
BLS verify - blst-native 1.8551 ms/op 1.8563 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8016 ms/op 3.8108 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.1849 ms/op 8.1886 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.681 ms/op 29.681 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.345 us/op 39.172 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 152.76 us/op 152.93 us/op 1.00
getAttestationsForBlock 179.59 ms/op 175.09 ms/op 1.03
isKnown best case - 1 super set check 424.00 ns/op 464.00 ns/op 0.91
isKnown normal case - 2 super set checks 410.00 ns/op 452.00 ns/op 0.91
isKnown worse case - 16 super set checks 416.00 ns/op 454.00 ns/op 0.92
CheckpointStateCache - add get delete 8.6340 us/op 9.2810 us/op 0.93
validate gossip signedAggregateAndProof - struct 4.2754 ms/op 4.3161 ms/op 0.99
validate gossip attestation - struct 2.0325 ms/op 2.0346 ms/op 1.00
pickEth1Vote - no votes 2.2507 ms/op 2.2160 ms/op 1.02
pickEth1Vote - max votes 21.046 ms/op 21.322 ms/op 0.99
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.380 ms/op 11.413 ms/op 1.00
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.487 ms/op 21.712 ms/op 0.99
pickEth1Vote - Eth1Data fastSerialize value x2048 1.7022 ms/op 1.6348 ms/op 1.04
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.332 ms/op 14.284 ms/op 1.00
bytes32 toHexString 1.1970 us/op 1.1900 us/op 1.01
bytes32 Buffer.toString(hex) 755.00 ns/op 752.00 ns/op 1.00
bytes32 Buffer.toString(hex) from Uint8Array 995.00 ns/op 1.0140 us/op 0.98
bytes32 Buffer.toString(hex) + 0x 729.00 ns/op 739.00 ns/op 0.99
Object access 1 prop 0.36600 ns/op 0.39700 ns/op 0.92
Map access 1 prop 0.29100 ns/op 0.29900 ns/op 0.97
Object get x1000 17.921 ns/op 17.582 ns/op 1.02
Map get x1000 1.1790 ns/op 0.98400 ns/op 1.20
Object set x1000 119.85 ns/op 128.47 ns/op 0.93
Map set x1000 70.979 ns/op 79.286 ns/op 0.90
Return object 10000 times 0.37180 ns/op 0.37110 ns/op 1.00
Throw Error 10000 times 5.9688 us/op 5.8484 us/op 1.02
enrSubnets - fastDeserialize 64 bits 2.8740 us/op 2.9640 us/op 0.97
enrSubnets - ssz BitVector 64 bits 749.00 ns/op 769.00 ns/op 0.97
enrSubnets - fastDeserialize 4 bits 408.00 ns/op 417.00 ns/op 0.98
enrSubnets - ssz BitVector 4 bits 726.00 ns/op 784.00 ns/op 0.93
prioritizePeers score -10:0 att 32-0.1 sync 2-0 98.295 us/op 99.257 us/op 0.99
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 124.57 us/op 122.59 us/op 1.02
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 213.00 us/op 226.11 us/op 0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 462.46 us/op 477.46 us/op 0.97
prioritizePeers score 0:0 att 64-1 sync 4-1 464.07 us/op 458.28 us/op 1.01
RateTracker 1000000 limit, 1 obj count per request 221.08 ns/op 211.73 ns/op 1.04
RateTracker 1000000 limit, 2 obj count per request 167.93 ns/op 164.24 ns/op 1.02
RateTracker 1000000 limit, 4 obj count per request 135.09 ns/op 140.75 ns/op 0.96
RateTracker 1000000 limit, 8 obj count per request 121.66 ns/op 129.11 ns/op 0.94
RateTracker with prune 4.3660 us/op 5.1610 us/op 0.85
array of 16000 items push then shift 3.2119 us/op 3.1913 us/op 1.01
LinkedList of 16000 items push then shift 19.253 ns/op 17.654 ns/op 1.09
array of 16000 items push then pop 256.63 ns/op 231.37 ns/op 1.11
LinkedList of 16000 items push then pop 17.659 ns/op 16.621 ns/op 1.06
array of 24000 items push then shift 4.5601 us/op 4.5578 us/op 1.00
LinkedList of 24000 items push then shift 23.291 ns/op 21.374 ns/op 1.09
array of 24000 items push then pop 207.73 ns/op 213.57 ns/op 0.97
LinkedList of 24000 items push then pop 19.957 ns/op 18.387 ns/op 1.09
intersect bitArray bitLen 8 11.767 ns/op 11.760 ns/op 1.00
intersect array and set length 8 175.26 ns/op 188.92 ns/op 0.93
intersect bitArray bitLen 128 61.909 ns/op 61.986 ns/op 1.00
intersect array and set length 128 2.3290 us/op 2.3463 us/op 0.99
Buffer.concat 32 items 2.2090 ns/op 2.2160 ns/op 1.00
pass gossip attestations to forkchoice per slot 6.5283 ms/op 3.2001 ms/op 2.04
computeDeltas 3.6523 ms/op 3.2569 ms/op 1.12
computeProposerBoostScoreFromBalances 907.77 us/op 909.38 us/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 4.4462 ms/op 4.3969 ms/op 1.01
altair processAttestation - 250000 vs - 7PWei worstcase 6.0691 ms/op 6.5472 ms/op 0.93
altair processAttestation - setStatus - 1/6 committees join 210.57 us/op 214.06 us/op 0.98
altair processAttestation - setStatus - 1/3 committees join 399.90 us/op 410.95 us/op 0.97
altair processAttestation - setStatus - 1/2 committees join 558.76 us/op 560.89 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 723.49 us/op 721.89 us/op 1.00
altair processAttestation - setStatus - 4/5 committees join 1.0003 ms/op 1.0052 ms/op 1.00
altair processAttestation - setStatus - 100% committees join 1.1821 ms/op 1.1961 ms/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 29.303 ms/op 29.209 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase hashState 40.494 ms/op 45.214 ms/op 0.90
altair processBlock - 250000 vs - 7PWei worstcase 84.085 ms/op 75.369 ms/op 1.12
altair processBlock - 250000 vs - 7PWei worstcase hashState 99.875 ms/op 112.21 ms/op 0.89
phase0 processBlock - 250000 vs - 7PWei normalcase 3.7669 ms/op 3.7949 ms/op 0.99
phase0 processBlock - 250000 vs - 7PWei worstcase 47.625 ms/op 47.033 ms/op 1.01
altair processEth1Data - 250000 vs - 7PWei normalcase 698.91 us/op 945.01 us/op 0.74
Tree 40 250000 create 766.00 ms/op 979.39 ms/op 0.78
Tree 40 250000 get(125000) 293.39 ns/op 304.72 ns/op 0.96
Tree 40 250000 set(125000) 2.5174 us/op 3.2001 us/op 0.79
Tree 40 250000 toArray() 38.972 ms/op 34.337 ms/op 1.14
Tree 40 250000 iterate all - toArray() + loop 39.462 ms/op 33.902 ms/op 1.16
Tree 40 250000 iterate all - get(i) 118.42 ms/op 115.22 ms/op 1.03
MutableVector 250000 create 15.089 ms/op 19.768 ms/op 0.76
MutableVector 250000 get(125000) 14.737 ns/op 14.752 ns/op 1.00
MutableVector 250000 set(125000) 659.11 ns/op 855.65 ns/op 0.77
MutableVector 250000 toArray() 7.6453 ms/op 8.0785 ms/op 0.95
MutableVector 250000 iterate all - toArray() + loop 7.7520 ms/op 8.2752 ms/op 0.94
MutableVector 250000 iterate all - get(i) 3.2797 ms/op 3.2804 ms/op 1.00
Array 250000 create 6.8338 ms/op 7.5383 ms/op 0.91
Array 250000 clone - spread 3.7026 ms/op 4.1528 ms/op 0.89
Array 250000 get(125000) 1.5480 ns/op 1.8360 ns/op 0.84
Array 250000 set(125000) 1.5620 ns/op 1.9310 ns/op 0.81
Array 250000 iterate all - loop 167.87 us/op 167.89 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 85.432 us/op 115.97 us/op 0.74
effectiveBalanceIncrements clone MutableVector 300000 707.00 ns/op 1.5130 us/op 0.47
effectiveBalanceIncrements rw all Uint8Array 300000 252.39 us/op 252.55 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 189.68 ms/op 262.32 ms/op 0.72
phase0 afterProcessEpoch - 250000 vs - 7PWei 199.48 ms/op 185.21 ms/op 1.08
phase0 beforeProcessEpoch - 250000 vs - 7PWei 77.447 ms/op 115.92 ms/op 0.67
altair processEpoch - mainnet_e81889 602.06 ms/op 589.09 ms/op 1.02
mainnet_e81889 - altair beforeProcessEpoch 185.70 ms/op 170.74 ms/op 1.09
mainnet_e81889 - altair processJustificationAndFinalization 25.863 us/op 22.263 us/op 1.16
mainnet_e81889 - altair processInactivityUpdates 11.581 ms/op 10.761 ms/op 1.08
mainnet_e81889 - altair processRewardsAndPenalties 93.860 ms/op 95.569 ms/op 0.98
mainnet_e81889 - altair processRegistryUpdates 7.0570 us/op 3.4610 us/op 2.04
mainnet_e81889 - altair processSlashings 1.5960 us/op 784.00 ns/op 2.04
mainnet_e81889 - altair processEth1DataReset 1.4750 us/op 767.00 ns/op 1.92
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3357 ms/op 2.4399 ms/op 0.96
mainnet_e81889 - altair processSlashingsReset 10.572 us/op 5.4870 us/op 1.93
mainnet_e81889 - altair processRandaoMixesReset 9.7340 us/op 5.6940 us/op 1.71
mainnet_e81889 - altair processHistoricalRootsUpdate 1.6050 us/op 790.00 ns/op 2.03
mainnet_e81889 - altair processParticipationFlagUpdates 5.0120 us/op 2.9610 us/op 1.69
mainnet_e81889 - altair processSyncCommitteeUpdates 1.1140 us/op 664.00 ns/op 1.68
mainnet_e81889 - altair afterProcessEpoch 198.65 ms/op 194.92 ms/op 1.02
phase0 processEpoch - mainnet_e58758 550.28 ms/op 549.14 ms/op 1.00
mainnet_e58758 - phase0 beforeProcessEpoch 245.86 ms/op 244.07 ms/op 1.01
mainnet_e58758 - phase0 processJustificationAndFinalization 26.830 us/op 21.054 us/op 1.27
mainnet_e58758 - phase0 processRewardsAndPenalties 139.92 ms/op 145.04 ms/op 0.96
mainnet_e58758 - phase0 processRegistryUpdates 17.517 us/op 9.4770 us/op 1.85
mainnet_e58758 - phase0 processSlashings 1.3220 us/op 721.00 ns/op 1.83
mainnet_e58758 - phase0 processEth1DataReset 1.3460 us/op 706.00 ns/op 1.91
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2056 ms/op 2.3048 ms/op 0.96
mainnet_e58758 - phase0 processSlashingsReset 8.3910 us/op 4.6230 us/op 1.82
mainnet_e58758 - phase0 processRandaoMixesReset 9.8200 us/op 5.9380 us/op 1.65
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.6320 us/op 799.00 ns/op 2.04
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.9590 us/op 4.6450 us/op 1.71
mainnet_e58758 - phase0 afterProcessEpoch 162.11 ms/op 159.07 ms/op 1.02
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6776 ms/op 2.5747 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.5602 ms/op 3.4980 ms/op 1.02
altair processInactivityUpdates - 250000 normalcase 43.827 ms/op 42.897 ms/op 1.02
altair processInactivityUpdates - 250000 worstcase 52.554 ms/op 51.949 ms/op 1.01
phase0 processRegistryUpdates - 250000 normalcase 10.708 us/op 7.5210 us/op 1.42
phase0 processRegistryUpdates - 250000 badcase_full_deposits 421.93 us/op 407.10 us/op 1.04
phase0 processRegistryUpdates - 250000 worstcase 0.5 219.12 ms/op 229.84 ms/op 0.95
altair processRewardsAndPenalties - 250000 normalcase 128.92 ms/op 130.16 ms/op 0.99
altair processRewardsAndPenalties - 250000 worstcase 92.601 ms/op 107.32 ms/op 0.86
phase0 getAttestationDeltas - 250000 normalcase 13.918 ms/op 15.833 ms/op 0.88
phase0 getAttestationDeltas - 250000 worstcase 13.055 ms/op 16.274 ms/op 0.80
phase0 processSlashings - 250000 worstcase 5.4457 ms/op 5.5535 ms/op 0.98
altair processSyncCommitteeUpdates - 250000 286.71 ms/op 299.79 ms/op 0.96
BeaconState.hashTreeRoot - No change 516.00 ns/op 530.00 ns/op 0.97
BeaconState.hashTreeRoot - 1 full validator 65.211 us/op 63.689 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 627.75 us/op 621.18 us/op 1.01
BeaconState.hashTreeRoot - 512 full validator 6.5363 ms/op 6.9903 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 84.541 us/op 91.897 us/op 0.92
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2036 ms/op 1.1893 ms/op 1.01
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 15.821 ms/op 16.707 ms/op 0.95
BeaconState.hashTreeRoot - 1 balances 60.894 us/op 62.546 us/op 0.97
BeaconState.hashTreeRoot - 32 balances 468.22 us/op 575.68 us/op 0.81
BeaconState.hashTreeRoot - 512 balances 6.7501 ms/op 6.0956 ms/op 1.11
BeaconState.hashTreeRoot - 250000 balances 90.951 ms/op 95.032 ms/op 0.96
aggregationBits - 2048 els - zipIndexesInBitList 33.008 us/op 44.835 us/op 0.74
regular array get 100000 times 67.434 us/op 67.428 us/op 1.00
wrappedArray get 100000 times 67.363 us/op 67.416 us/op 1.00
arrayWithProxy get 100000 times 28.879 ms/op 28.739 ms/op 1.00
ssz.Root.equals 480.00 ns/op 564.00 ns/op 0.85
byteArrayEquals 478.00 ns/op 549.00 ns/op 0.87
shuffle list - 16384 els 11.098 ms/op 12.183 ms/op 0.91
shuffle list - 250000 els 166.47 ms/op 162.44 ms/op 1.02
processSlot - 1 slots 12.350 us/op 12.811 us/op 0.96
processSlot - 32 slots 1.8106 ms/op 1.7465 ms/op 1.04
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 410.98 us/op 391.41 us/op 1.05
getCommitteeAssignments - req 1 vs - 250000 vc 5.3468 ms/op 5.3386 ms/op 1.00
getCommitteeAssignments - req 100 vs - 250000 vc 7.3128 ms/op 7.3858 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7941 ms/op 7.8177 ms/op 1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.9700 ns/op 11.360 ns/op 0.88
state getBlockRootAtSlot - 250000 vs - 7PWei 1.1898 us/op 1.2787 us/op 0.93
computeProposers - vc 250000 16.717 ms/op 17.481 ms/op 0.96
computeEpochShuffling - vc 250000 169.84 ms/op 166.22 ms/op 1.02
getNextSyncCommittee - vc 250000 273.50 ms/op 288.60 ms/op 0.95

by benchmarkbot/action

@g11tech g11tech mentioned this pull request Jun 25, 2022
22 tasks
@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch 2 times, most recently from ff89835 to b5d6d0d Compare July 4, 2022 09:57
@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch 2 times, most recently from 2e6beb1 to c9a28af Compare July 5, 2022 18:13
}
}

propagateInValidExecutionStatusByIndex(invalidateFromIndex: number, latestValidHashIndex: number): void {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make the 1st param name consistent to the consumer? this is called invalidateFromIndex while the consumer calls invalidateTillBlockIndex

also it's nice to have some descriptions in the method header, or draw some ascii texts representing forkchoice branches

/** invalidateAll is a flag which indicates detection of consensus failure */
let invalidateAll = false;
let invalidateIndex: number | undefined = invalidateFromIndex;
while (invalidateIndex !== undefined && invalidateFromIndex > latestValidHashIndex) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be

while (invalidateIndex !== undefined && invalidateIndex > latestValidHashIndex) {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose adding some unit tests for those new methods should help

Copy link
Contributor Author

@g11tech g11tech Jul 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed! thanks for spotting it @tuyennhv ! yes will add a test case around this as well.

@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch from 7373342 to adedd33 Compare July 6, 2022 17:59

if (execResponse.executionStatus === ExecutionStatus.Valid) {
const {latestValidExecHash} = execResponse;
let latestValidHashIndex = this.nodes.length - 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the default latest valid hash index should not be the last node, maybe make it undefined or null?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the input, i have initialized to -1, keeping in line with "not found" assignments of the indices. if you still feel strongly for undefined or null to represent not found, do respond 🙂

const {latestValidExecHash} = execResponse;
let latestValidHashIndex = this.nodes.length - 1;

for (; latestValidHashIndex >= 0; latestValidHashIndex--) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps extract this to an array util so that it's reusable and testable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its not really generic, and is covered by the validation test cases.

@@ -193,17 +206,23 @@ export async function verifyBlockStateTransition(
switch (execResult.status) {
case ExecutePayloadStatus.VALID:
executionStatus = ExecutionStatus.Valid;
chain.forkChoice.validateLatestHash(execResult.latestValidHash, null);
chain.forkChoice.validateLatestHash({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech my understanding is latestValidHash is the current execution payload which we have not imported into forkchoice, is it an issue?

Copy link
Contributor Author

@g11tech g11tech Jul 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tuyennhv its not an issue here, since the latestValidHash should be the parent of the current payload, as current payload should not have become part of the chain as it has not been fcU-ed yet. But I will validate this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@g11tech what I see is go-ethereum returns the current execution payload as latestValidHash most of the time so we should handle it https://github.com/ethereum/go-ethereum/blob/e108d36575fd93e6088259cdabeed6dddf76458c/eth/catalyst/api.go#L324

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

modified, just directly passing the parent's exechash now, so no matter what EL responds as LVH (block or its parent), we pass the correct one

@g11tech g11tech marked this pull request as ready for review July 16, 2022 12:16
@g11tech g11tech requested a review from a team as a code owner July 16, 2022 12:16
@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch 2 times, most recently from 1f5bbb5 to e893be4 Compare July 18, 2022 12:24
latestValidExecHash: parentBlock.executionPayloadBlockHash,
invalidateTillBlockHash: null,
});
}
break; // OK

case ExecutePayloadStatus.INVALID: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should there be executionStatus = ExecutionStatus.Invalid here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are just throwing below after validateLatestHash invalid status propagation, so its not getting used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also added some typesafety to it:

-  let executionStatus: ExecutionStatus;
+  let executionStatus: MaybeValidExecutionStatus;

@twoeths
Copy link
Contributor

twoeths commented Aug 1, 2022

@g11tech take a look at ethereum/consensus-specs#2954

@g11tech
Copy link
Contributor Author

g11tech commented Aug 1, 2022

@g11tech take a look at ethereum/consensus-specs#2954

yes, the PR is the result of the discussions we already had on discord interop channel.
And this PR already incorporates this (if latestValidHash is not found in forkChoice, only the block is invalidated)

node.executionStatus = ExecutionStatus.Valid;
break;

default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty default clause?

@@ -482,6 +747,7 @@ export class ProtoArray {
* head.
*/
nodeIsViableForHead(node: ProtoNode): boolean {
if (node.executionStatus === ExecutionStatus.Invalid) return false;
Copy link
Contributor

@dapplion dapplion Aug 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (node.executionStatus === ExecutionStatus.Invalid) return false;
if (node.executionStatus === ExecutionStatus.Invalid) {
return false;
}

// Only invalidate if this is post merge, and either parent is invalid or the
// concensus has failed
if (node.executionStatus !== ExecutionStatus.PreMerge && parent?.executionStatus === ExecutionStatus.Invalid) {
if (node.executionStatus === ExecutionStatus.Valid) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this two if statements be merged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simplified the conditions, hopefully its more clear now

@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch from f7a1937 to e40a529 Compare August 6, 2022 11:40
@g11tech g11tech enabled auto-merge (squash) August 12, 2022 18:20
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️ meta-approving! Looks really good, thanks for the simplifications.

Leaving un-approved until we cut 0.42 release. Left some minor comments, if you want to address before then

packages/fork-choice/src/protoArray/protoArray.ts Outdated Show resolved Hide resolved
@@ -17,6 +21,7 @@ export class ProtoArray {
finalizedRoot: RootHex;
nodes: ProtoNode[] = [];
indices = new Map<RootHex, number>();
lvhError?: LVHExecError;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should include the mechanism to kill beacon process from the beacon package. Just check every slot or at some predicatable spot

packages/fork-choice/src/protoArray/protoArray.ts Outdated Show resolved Hide resolved
packages/fork-choice/src/protoArray/protoArray.ts Outdated Show resolved Hide resolved
private propagateValidExecutionStatusByIndex(validNodeIndex: number): void {
let nodeIndex: number | undefined = validNodeIndex;
// propagate till we keep encountering syncing status
while (nodeIndex !== undefined) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This loop should also stop when the parent is already valid as a cases in addition to going until invalid, premerge, or thru all ancestors.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the loop break condition to !== syncing 👍

g11tech and others added 17 commits September 2, 2022 17:03
 * extend forkchoice execution type

 * handle the invalid status in protoarray

 * add the pre execution status validation testcase

 * relocate validateLatestHash to protoarray for better testing

 * add case for invalidating a branch

 * add invalidations and validations

 * add a bit more desc to test case title

 * fix the invalidate index bug

 * refac the reverse lookup and throw if forkchoice becomes invalidated

 * relax the LVH conditions in invalid responses

 * only invalidate post merge

 * modify the invalidations to allow for various LVH scenarios

 * add invalidation case

 * simplyfy lvh search and be more forgiving when not found

 * add a fail hard case

 * correct the lvh passed in valid response

 * refactor how to extract lvh in valid scenario

 * replace invalidate all with proper error handling to be more verbose and swallow less

 * add test for invalid to valid

 * add type safety to executionStatus

 * linting if for better visual clarity

 * fix tests

 * refac the invalidation condition in propagate invalid lvh

 * remove empty default clause

 * simplify segment exec response

 * error logging

 * always throw if there is segment error
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
@g11tech g11tech force-pushed the g11tech/latest-valid-hash-update branch from e4d6d3f to 6b75d90 Compare September 2, 2022 11:44
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@g11tech g11tech merged commit 5f3509e into unstable Sep 2, 2022
@g11tech g11tech deleted the g11tech/latest-valid-hash-update branch September 2, 2022 14:55
@philknows philknows added this to the v1.1.0 milestone Sep 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimistic Sync: Part 2 - Prune/Validate/Update fork-choice on lastValidHash response
5 participants