Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix remote signing #4502

Merged
merged 41 commits into from Sep 14, 2022
Merged

Fix remote signing #4502

merged 41 commits into from Sep 14, 2022

Conversation

dadepo
Copy link
Contributor

@dadepo dadepo commented Sep 1, 2022

Motivation

Using remote signer currently does not work. This is because the web3signer URL is different from what lodestar expects.

PS: currently draft PR as the branch still need to be tested on goerli test net.

Description

Update implementation to use correct web3signer endpoints.

Closes #4424

Steps to test or reproduce

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 731c938 Previous: 0886d75 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.4513 ms/op 2.2080 ms/op 1.11
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 78.189 us/op 73.089 us/op 1.07
BLS verify - blst-native 1.8572 ms/op 1.8569 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8126 ms/op 3.8002 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.1977 ms/op 8.1861 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.684 ms/op 29.683 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.197 us/op 39.090 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 152.63 us/op 152.95 us/op 1.00
getAttestationsForBlock 174.10 ms/op 173.56 ms/op 1.00
isKnown best case - 1 super set check 426.00 ns/op 478.00 ns/op 0.89
isKnown normal case - 2 super set checks 425.00 ns/op 469.00 ns/op 0.91
isKnown worse case - 16 super set checks 412.00 ns/op 466.00 ns/op 0.88
CheckpointStateCache - add get delete 9.1510 us/op 8.8750 us/op 1.03
validate gossip signedAggregateAndProof - struct 4.2738 ms/op 4.2548 ms/op 1.00
validate gossip attestation - struct 2.0363 ms/op 2.0216 ms/op 1.01
pickEth1Vote - no votes 2.2726 ms/op 2.1828 ms/op 1.04
pickEth1Vote - max votes 21.119 ms/op 19.826 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.506 ms/op 11.523 ms/op 1.00
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.795 ms/op 20.639 ms/op 1.06
pickEth1Vote - Eth1Data fastSerialize value x2048 1.6595 ms/op 1.6196 ms/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 13.690 ms/op 12.810 ms/op 1.07
bytes32 toHexString 1.1220 us/op 1.0390 us/op 1.08
bytes32 Buffer.toString(hex) 740.00 ns/op 709.00 ns/op 1.04
bytes32 Buffer.toString(hex) from Uint8Array 947.00 ns/op 933.00 ns/op 1.02
bytes32 Buffer.toString(hex) + 0x 732.00 ns/op 715.00 ns/op 1.02
Object access 1 prop 0.39700 ns/op 0.36700 ns/op 1.08
Map access 1 prop 0.29300 ns/op 0.29600 ns/op 0.99
Object get x1000 17.871 ns/op 18.815 ns/op 0.95
Map get x1000 0.98300 ns/op 0.97000 ns/op 1.01
Object set x1000 123.03 ns/op 122.12 ns/op 1.01
Map set x1000 73.571 ns/op 73.522 ns/op 1.00
Return object 10000 times 0.36690 ns/op 0.37880 ns/op 0.97
Throw Error 10000 times 5.9692 us/op 5.9895 us/op 1.00
enrSubnets - fastDeserialize 64 bits 2.6640 us/op 2.6430 us/op 1.01
enrSubnets - ssz BitVector 64 bits 757.00 ns/op 752.00 ns/op 1.01
enrSubnets - fastDeserialize 4 bits 414.00 ns/op 386.00 ns/op 1.07
enrSubnets - ssz BitVector 4 bits 761.00 ns/op 741.00 ns/op 1.03
prioritizePeers score -10:0 att 32-0.1 sync 2-0 98.020 us/op 94.739 us/op 1.03
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 124.07 us/op 122.59 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 227.13 us/op 218.53 us/op 1.04
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 485.61 us/op 474.63 us/op 1.02
prioritizePeers score 0:0 att 64-1 sync 4-1 465.43 us/op 458.24 us/op 1.02
RateTracker 1000000 limit, 1 obj count per request 190.60 ns/op 204.92 ns/op 0.93
RateTracker 1000000 limit, 2 obj count per request 146.82 ns/op 156.90 ns/op 0.94
RateTracker 1000000 limit, 4 obj count per request 119.92 ns/op 131.83 ns/op 0.91
RateTracker 1000000 limit, 8 obj count per request 108.69 ns/op 122.18 ns/op 0.89
RateTracker with prune 4.5670 us/op 4.4160 us/op 1.03
array of 16000 items push then shift 3.1939 us/op 3.1895 us/op 1.00
LinkedList of 16000 items push then shift 19.486 ns/op 17.296 ns/op 1.13
array of 16000 items push then pop 249.35 ns/op 230.81 ns/op 1.08
LinkedList of 16000 items push then pop 17.831 ns/op 15.876 ns/op 1.12
array of 24000 items push then shift 4.5643 us/op 4.5541 us/op 1.00
LinkedList of 24000 items push then shift 23.328 ns/op 20.670 ns/op 1.13
array of 24000 items push then pop 212.69 ns/op 207.34 ns/op 1.03
LinkedList of 24000 items push then pop 20.190 ns/op 17.852 ns/op 1.13
intersect bitArray bitLen 8 11.734 ns/op 11.455 ns/op 1.02
intersect array and set length 8 168.22 ns/op 164.06 ns/op 1.03
intersect bitArray bitLen 128 62.021 ns/op 71.970 ns/op 0.86
intersect array and set length 128 2.3157 us/op 2.2891 us/op 1.01
Buffer.concat 32 items 2.2640 ns/op 1.8640 ns/op 1.21
pass gossip attestations to forkchoice per slot 3.2158 ms/op 4.9492 ms/op 0.65
computeDeltas 3.6997 ms/op 3.5756 ms/op 1.03
computeProposerBoostScoreFromBalances 907.99 us/op 924.91 us/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 4.0762 ms/op 3.8940 ms/op 1.05
altair processAttestation - 250000 vs - 7PWei worstcase 7.4142 ms/op 6.0183 ms/op 1.23
altair processAttestation - setStatus - 1/6 committees join 216.24 us/op 216.04 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 415.81 us/op 410.27 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 585.91 us/op 576.59 us/op 1.02
altair processAttestation - setStatus - 2/3 committees join 758.26 us/op 741.85 us/op 1.02
altair processAttestation - setStatus - 4/5 committees join 1.0450 ms/op 1.0151 ms/op 1.03
altair processAttestation - setStatus - 100% committees join 1.2470 ms/op 1.2202 ms/op 1.02
altair processBlock - 250000 vs - 7PWei normalcase 28.806 ms/op 27.318 ms/op 1.05
altair processBlock - 250000 vs - 7PWei normalcase hashState 39.766 ms/op 38.945 ms/op 1.02
altair processBlock - 250000 vs - 7PWei worstcase 91.653 ms/op 82.597 ms/op 1.11
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.727 ms/op 108.22 ms/op 0.91
phase0 processBlock - 250000 vs - 7PWei normalcase 3.8355 ms/op 3.5492 ms/op 1.08
phase0 processBlock - 250000 vs - 7PWei worstcase 47.322 ms/op 46.588 ms/op 1.02
altair processEth1Data - 250000 vs - 7PWei normalcase 921.68 us/op 825.50 us/op 1.12
Tree 40 250000 create 783.06 ms/op 807.38 ms/op 0.97
Tree 40 250000 get(125000) 287.82 ns/op 287.92 ns/op 1.00
Tree 40 250000 set(125000) 2.7079 us/op 2.4843 us/op 1.09
Tree 40 250000 toArray() 33.632 ms/op 32.744 ms/op 1.03
Tree 40 250000 iterate all - toArray() + loop 34.194 ms/op 32.685 ms/op 1.05
Tree 40 250000 iterate all - get(i) 113.25 ms/op 111.93 ms/op 1.01
MutableVector 250000 create 16.572 ms/op 15.350 ms/op 1.08
MutableVector 250000 get(125000) 14.764 ns/op 14.750 ns/op 1.00
MutableVector 250000 set(125000) 684.02 ns/op 622.15 ns/op 1.10
MutableVector 250000 toArray() 7.7118 ms/op 7.4729 ms/op 1.03
MutableVector 250000 iterate all - toArray() + loop 7.9664 ms/op 7.8016 ms/op 1.02
MutableVector 250000 iterate all - get(i) 3.4409 ms/op 3.2800 ms/op 1.05
Array 250000 create 7.2503 ms/op 6.9745 ms/op 1.04
Array 250000 clone - spread 3.7751 ms/op 3.6724 ms/op 1.03
Array 250000 get(125000) 1.5280 ns/op 1.5640 ns/op 0.98
Array 250000 set(125000) 1.5570 ns/op 1.5480 ns/op 1.01
Array 250000 iterate all - loop 170.37 us/op 170.29 us/op 1.00
effectiveBalanceIncrements clone Uint8Array 300000 91.418 us/op 96.103 us/op 0.95
effectiveBalanceIncrements clone MutableVector 300000 1.0930 us/op 1.1580 us/op 0.94
effectiveBalanceIncrements rw all Uint8Array 300000 252.43 us/op 252.57 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 232.70 ms/op 208.44 ms/op 1.12
phase0 afterProcessEpoch - 250000 vs - 7PWei 187.55 ms/op 186.68 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 117.38 ms/op 100.95 ms/op 1.16
altair processEpoch - mainnet_e81889 603.22 ms/op 579.64 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 148.66 ms/op 145.63 ms/op 1.02
mainnet_e81889 - altair processJustificationAndFinalization 24.389 us/op 28.397 us/op 0.86
mainnet_e81889 - altair processInactivityUpdates 12.097 ms/op 11.001 ms/op 1.10
mainnet_e81889 - altair processRewardsAndPenalties 94.623 ms/op 90.940 ms/op 1.04
mainnet_e81889 - altair processRegistryUpdates 3.7530 us/op 4.3450 us/op 0.86
mainnet_e81889 - altair processSlashings 923.00 ns/op 1.1190 us/op 0.82
mainnet_e81889 - altair processEth1DataReset 835.00 ns/op 1.1390 us/op 0.73
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.4200 ms/op 2.4403 ms/op 0.99
mainnet_e81889 - altair processSlashingsReset 5.6420 us/op 7.6580 us/op 0.74
mainnet_e81889 - altair processRandaoMixesReset 6.3390 us/op 7.7890 us/op 0.81
mainnet_e81889 - altair processHistoricalRootsUpdate 910.00 ns/op 1.1200 us/op 0.81
mainnet_e81889 - altair processParticipationFlagUpdates 2.9150 us/op 4.1720 us/op 0.70
mainnet_e81889 - altair processSyncCommitteeUpdates 739.00 ns/op 985.00 ns/op 0.75
mainnet_e81889 - altair afterProcessEpoch 197.41 ms/op 195.34 ms/op 1.01
phase0 processEpoch - mainnet_e58758 560.32 ms/op 528.50 ms/op 1.06
mainnet_e58758 - phase0 beforeProcessEpoch 258.57 ms/op 229.07 ms/op 1.13
mainnet_e58758 - phase0 processJustificationAndFinalization 20.976 us/op 20.196 us/op 1.04
mainnet_e58758 - phase0 processRewardsAndPenalties 151.71 ms/op 139.84 ms/op 1.08
mainnet_e58758 - phase0 processRegistryUpdates 10.074 us/op 12.469 us/op 0.81
mainnet_e58758 - phase0 processSlashings 799.00 ns/op 975.00 ns/op 0.82
mainnet_e58758 - phase0 processEth1DataReset 801.00 ns/op 1.1000 us/op 0.73
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0870 ms/op 2.1165 ms/op 0.99
mainnet_e58758 - phase0 processSlashingsReset 5.5000 us/op 6.7510 us/op 0.81
mainnet_e58758 - phase0 processRandaoMixesReset 5.5130 us/op 7.9080 us/op 0.70
mainnet_e58758 - phase0 processHistoricalRootsUpdate 880.00 ns/op 905.00 ns/op 0.97
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.6630 us/op 5.1400 us/op 0.91
mainnet_e58758 - phase0 afterProcessEpoch 162.03 ms/op 160.76 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.5447 ms/op 2.5448 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.4340 ms/op 3.4758 ms/op 0.99
altair processInactivityUpdates - 250000 normalcase 41.774 ms/op 42.350 ms/op 0.99
altair processInactivityUpdates - 250000 worstcase 56.056 ms/op 49.646 ms/op 1.13
phase0 processRegistryUpdates - 250000 normalcase 7.9550 us/op 11.292 us/op 0.70
phase0 processRegistryUpdates - 250000 badcase_full_deposits 406.95 us/op 416.86 us/op 0.98
phase0 processRegistryUpdates - 250000 worstcase 0.5 226.69 ms/op 215.37 ms/op 1.05
altair processRewardsAndPenalties - 250000 normalcase 129.71 ms/op 119.77 ms/op 1.08
altair processRewardsAndPenalties - 250000 worstcase 88.775 ms/op 84.881 ms/op 1.05
phase0 getAttestationDeltas - 250000 normalcase 14.593 ms/op 11.119 ms/op 1.31
phase0 getAttestationDeltas - 250000 worstcase 11.796 ms/op 12.518 ms/op 0.94
phase0 processSlashings - 250000 worstcase 5.5047 ms/op 5.3945 ms/op 1.02
altair processSyncCommitteeUpdates - 250000 291.81 ms/op 282.14 ms/op 1.03
BeaconState.hashTreeRoot - No change 475.00 ns/op 491.00 ns/op 0.97
BeaconState.hashTreeRoot - 1 full validator 64.193 us/op 54.800 us/op 1.17
BeaconState.hashTreeRoot - 32 full validator 646.70 us/op 638.25 us/op 1.01
BeaconState.hashTreeRoot - 512 full validator 7.8680 ms/op 5.9398 ms/op 1.32
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 77.448 us/op 78.346 us/op 0.99
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1815 ms/op 1.1399 ms/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.521 ms/op 15.779 ms/op 1.05
BeaconState.hashTreeRoot - 1 balances 62.801 us/op 61.750 us/op 1.02
BeaconState.hashTreeRoot - 32 balances 549.98 us/op 545.37 us/op 1.01
BeaconState.hashTreeRoot - 512 balances 6.0358 ms/op 6.2782 ms/op 0.96
BeaconState.hashTreeRoot - 250000 balances 98.442 ms/op 84.737 ms/op 1.16
aggregationBits - 2048 els - zipIndexesInBitList 41.250 us/op 33.316 us/op 1.24
regular array get 100000 times 67.389 us/op 67.374 us/op 1.00
wrappedArray get 100000 times 67.442 us/op 67.381 us/op 1.00
arrayWithProxy get 100000 times 29.033 ms/op 28.887 ms/op 1.01
ssz.Root.equals 555.00 ns/op 511.00 ns/op 1.09
byteArrayEquals 554.00 ns/op 493.00 ns/op 1.12
shuffle list - 16384 els 11.355 ms/op 11.301 ms/op 1.00
shuffle list - 250000 els 165.57 ms/op 165.80 ms/op 1.00
processSlot - 1 slots 12.538 us/op 13.121 us/op 0.96
processSlot - 32 slots 1.7724 ms/op 1.8208 ms/op 0.97
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 385.70 us/op 388.91 us/op 0.99
getCommitteeAssignments - req 1 vs - 250000 vc 5.2872 ms/op 5.3350 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 7.2967 ms/op 7.3276 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 7.8147 ms/op 7.7624 ms/op 1.01
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 10.140 ns/op 9.8600 ns/op 1.03
state getBlockRootAtSlot - 250000 vs - 7PWei 1.3044 us/op 1.1634 us/op 1.12
computeProposers - vc 250000 17.571 ms/op 16.601 ms/op 1.06
computeEpochShuffling - vc 250000 168.62 ms/op 169.14 ms/op 1.00
getNextSyncCommittee - vc 250000 288.56 ms/op 273.52 ms/op 1.05

by benchmarkbot/action

};
}

async signAttestationSelectionProof(pubkey: BLSPubkeyMaybeHex, slot: Slot): Promise<BLSSignature> {
const domain = this.config.getDomain(slot, DOMAIN_SELECTION_PROOF);
const signingRoot = computeSigningRoot(ssz.Slot, slot, domain);

return await this.getSignature(pubkey, signingRoot);
const signableMessage: SignableMessage = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the web3signer is not enabled, all this objects are being created for nothing right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to only create the Singable messages when web3signer is enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dapplion Your recent changes now makes it such that the objects are being created even when web3signer is not enabled. Is that intentional/okay now?

@dapplion
Copy link
Contributor

Please add an e2e test that run the web3signer docker container and request signatures for all available objects. I would really need to have programmatic certainty that this works before merging.

block: ssz.altair.BeaconBlock.toJson(data.block as altair.BeaconBlock),
};
} else {
throw new Error(`version ${data.version} not supported by remote signer`);
Copy link
Contributor

@dapplion dapplion Sep 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about the bellatrix fork? Using this pattern doesn't scale and will be painful to mantain

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I basically followed the pattern used in lighthouse here. Had an older version so bellatrix was not included. Added it now. As regards observation about maintenance. I'll revisit this after setting up all the tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yet to do the final sanity check but it occurred to me, that maybe the maintenance won't be too hard. Because:

  1. Having another fork is a coordinated effort that will be planned out and include coordinated code changes
  2. Even if lodestar can handle new forks without code changes, it should not, because it also depends on making sure that the Web3Signer has been updated to also handle the new fork. If not, lodestar will pass the call to web3signer but it will fail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right I wasnt aware of how weird the web3signer request types are 😅 made such that any fork after bellatrix with use the bellatrix path

@dapplion dapplion marked this pull request as ready for review September 14, 2022 12:50
@dapplion dapplion requested a review from a team as a code owner September 14, 2022 12:50
return getRemoteSigners(args);
} else {
return indexes.map((index) => ({type: SignerType.Local, secretKey: interopSecretKey(index)}));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this? I don't follow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To consume the interop indexes from the web3signer you should load the interop keystores in the web3signer and then use .fetch

@wemeetagain wemeetagain merged commit 9254e72 into unstable Sep 14, 2022
@wemeetagain wemeetagain deleted the dadepo/fix-remote-signer-endpoint branch September 14, 2022 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Retrieving PubKeys from Remote Signer Fails
3 participants