-
Notifications
You must be signed in to change notification settings - Fork 36.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Native Descriptor Wallets using DescriptorScriptPubKeyMan #16528
Native Descriptor Wallets using DescriptorScriptPubKeyMan #16528
Conversation
acbd19c
to
6f8afc0
Compare
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Strong Concept ACK, this should be much nicer now it is preceded by the rework. |
ef59fb7
to
90a631f
Compare
ef63001
to
755c4d1
Compare
Approach ACK. It looks pretty straight forward and thanks to The Box feels cleaner than the previous attempt. I suggest that, after a bit more progress on #16341, we review this in parallel. That ensures that we don't mess up the division of labour between Also concept ACK on using BIP44/49/84 for new descriptor wallets. That may need some discussion, because it means individual addresses are no longer hardened. Some issues I found perusing the commits:
|
Concept ACK. @achow101 I just noticed that both the existing https://github.com/bitcoin/bitcoin/blob/master/src/rpc/misc.cpp#L215 |
755c4d1
to
d777554
Compare
I guess so. Latest pushed should do that. |
693c8cc
to
84fb7d7
Compare
655ea88
to
a208faa
Compare
a208faa
to
f581c8c
Compare
f8edc40
to
69ad25c
Compare
69ad25c
to
98b7838
Compare
Due to your latest change upstream, this now complains: |
98b7838
to
d80de7d
Compare
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [20/43] : bitcoin/bitcoin@e014886 Depends on D8417 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, PiRK, majcosta Reviewed By: #bitcoin_abc, PiRK, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8418
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [21/43] : bitcoin/bitcoin@58c7651 Depends on D8417 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Subscribers: majcosta Differential Revision: https://reviews.bitcoinabc.org/D8419
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [22/43] : bitcoin/bitcoin@bfdd073 Depends on D8419 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Subscribers: majcosta Differential Revision: https://reviews.bitcoinabc.org/D8421
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [23/43] : bitcoin/bitcoin@a775f7c Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8426
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [24/43] : bitcoin/bitcoin@f866957 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8427
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [25/43] : bitcoin/bitcoin@586b57a Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8428
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [26/43] :bitcoin/bitcoin@f1ca5fe Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8445
Summary: Internally, a GetSigningProvider function is introduced which allows for some private keys to be optionally included. This can be called with a script as the argument (i.e. a scriptPubKey from our wallet when we are signing) or with a pubkey. In order to know what index to expand the private keys for that pubkey, we need to also cache all of the pubkeys involved when we expand the descriptor. So SetCache and TopUp are updated to do this too. Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [27/43] : bitcoin/bitcoin@d50c8dd Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8446
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [28/43] : bitcoin/bitcoin@bde7c9f Depends on D8446 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8447
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [29/43] : bitcoin/bitcoin@84b4978 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8448
Summary: FillPSBT will add our own scripts to the PSBT if those inputs are ours. If an input also lists pubkeys that we happen to know the private keys for, we will sign those inputs too. Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [30/43] : bitcoin/bitcoin@72a9540 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8451
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [31/43] : bitcoin/bitcoin@8b9603b Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien, majcosta Reviewed By: #bitcoin_abc, Fabien, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8453
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [31/43] : bitcoin/bitcoin@b713baa Depends on D8453 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D8454
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [33/43] : bitcoin/bitcoin@82ae02b Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D8455
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [34/43] : bitcoin/bitcoin@1cb42b2 Depends on D8455 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D8457
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [35/43] : bitcoin/bitcoin@ce24a94 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D8458
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [40/43] : bitcoin/bitcoin@886e0d7 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, jasonbcox Reviewed By: #bitcoin_abc, jasonbcox Differential Revision: https://reviews.bitcoinabc.org/D8466
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [41/43] : bitcoin/bitcoin@cf06062 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, jasonbcox Reviewed By: #bitcoin_abc, jasonbcox Differential Revision: https://reviews.bitcoinabc.org/D8468
Summary: When a CWallet doesn't have a ScriptPubKeyMan for the requested type in GetNewDestination, give a meaningful error. Also handle this in Qt which did not do anything with errors. Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [39/43] : bitcoin/bitcoin@3c19fdd Test Plan: ninja all check-all Reviewers: #bitcoin_abc, jasonbcox Reviewed By: #bitcoin_abc, jasonbcox Differential Revision: https://reviews.bitcoinabc.org/D8467
Summary: Co-authored-by: Andrew Chow <achow101-github@achow101.com> Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [36/43] : bitcoin/bitcoin@f193ea8 Add the wallet_util changes from [[bitcoin/bitcoin@223588b | 223588b]] because the patch is broken without it. Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Subscribers: majcosta Differential Revision: https://reviews.bitcoinabc.org/D8475
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [37/43] : bitcoin/bitcoin@1346e14 Depends on D8475 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Subscribers: majcosta Differential Revision: https://reviews.bitcoinabc.org/D8487
Summary: RPCOverloadWrapper overloads some deprecated or disabled RPCs with an implementation using other RPCs to avoid having a ton of code churn around replacing those RPCs. Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [42/43] : bitcoin/bitcoin@869f7ab Depends on D8487 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D8498
Summary: Backport of Core [[bitcoin/bitcoin#16528 | PR16528]] [38/43] : bitcoin/bitcoin@388ba94 Test Plan: ninja all check-all Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D8488
…mbers are left uninitialized after construction Summary: > This is a small folllow-up to [[bitcoin/bitcoin#16528 | PR16528]] ("Native Descriptor Wallets using DescriptorScriptPubKeyMan") > Before this change bool m_internal was left uninitialized when using the DescriptorScriptPubKeyMan(WalletStorage&, WalletDescriptor&) ctor. > The same goes for the now initialized integers which were left uninitialized when using the WalletDescriptor() ctor. wallet: Make sure no DescriptorScriptPubKeyMan members are uninitialized after construction bitcoin/bitcoin@ff046ae wallet: Make sure no WalletDescriptor members are uninitialized after construction bitcoin/bitcoin@2a78098 This is a backport of Core [[bitcoin/bitcoin#18782 | PR18782]] Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, majcosta Reviewed By: #bitcoin_abc, majcosta Differential Revision: https://reviews.bitcoinabc.org/D9073
Summary: > Adds a --descriptors option globally to the test framework. This will > make the test create and use descriptor wallets. However some tests may > not work with this. > > Some tests are modified to work with --descriptors and run with that > option in test_runer: > * wallet_encryption.py > * wallet_keypool.py > * wallet_keypool_topup.py > * wallet_labels.py > * wallet_avoidreuse.py This is a backport of [[bitcoin/bitcoin#16528 | core#16528]] [43a/43] bitcoin/bitcoin@223588b I'm splitting this commit into multiple commits to make review easier. This first part contains everything that works more or less out of the box and does not require significant changes to work with our codebase. All the deviations from the original PR in this revision are explained by out of order backports and lack of segwit and bech32. For instance, changes to key.py were already included in D8475 & D9934. Deviations in `RPCOverloadWrapper.createwallet` are explained by D10185 and D9101. I excluded specifically 3 tests that require changes to transaction amounts and fees when descriptors are used. With our codebase, descriptor wallets generate different fees, because in some situations the legacy wallet generates legacy p2pkh outputs and descriptor wallets generate `scriptPubKey` / `p2sh` outputs. I will address these in one or two separate diffs. These excluded tests are: - rpc_psbt.py - wallet_basic.py - wallet_keypool.py Test Plan: `ninja check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D10213
Summary: > Tests that a fully signed transaction given to > `signrawtransactionwithwallet` is both unchanged and marked as complete. > This tests for a regression in Bitcoin Core 0.20 where the transaction would not be > marked as complete. We do not have the regression in Bitcoin ABC and it seems unlikely that it will happen, as it was bissected to [[bitcoin/bitcoin#16528 | core#16528]] which we already finished backporting, and had to be fixed only on Core 0.20 (not master). But just in case, it is better to have the test to avoid any bad surprise. This is a backport of [[bitcoin/bitcoin#20562 | core#20562]] Test Plan: `ninja check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D10894
Summary: > Adds a --descriptors option globally to the test framework. This will > make the test create and use descriptor wallets. However some tests may > not work with this. > > Some tests are modified to work with --descriptors and run with that > option in test_runer: > * wallet_encryption.py > * wallet_keypool.py > * wallet_keypool_topup.py > * wallet_labels.py > * wallet_avoidreuse.py This is a backport of [[bitcoin/bitcoin#16528 | core#16528]] [43a/43] bitcoin/bitcoin@223588b I'm splitting this commit into multiple commits to make review easier. This first part contains everything that works more or less out of the box and does not require significant changes to work with our codebase. All the deviations from the original PR in this revision are explained by out of order backports and lack of segwit and bech32. For instance, changes to key.py were already included in D8475 & D9934. Deviations in `RPCOverloadWrapper.createwallet` are explained by D10185 and D9101. I excluded specifically 3 tests that require changes to transaction amounts and fees when descriptors are used. With our codebase, descriptor wallets generate different fees, because in some situations the legacy wallet generates legacy p2pkh outputs and descriptor wallets generate `scriptPubKey` / `p2sh` outputs. I will address these in one or two separate diffs. These excluded tests are: - rpc_psbt.py - wallet_basic.py - wallet_keypool.py Test Plan: `ninja check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D10213
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using
createwallet
.Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new
importdescriptors
RPC.Native descriptor wallets use the
ScriptPubKeyMan
interface introduced in #16341 to add aDescriptorScriptPubKeyMan
. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore,DescriptorScriptPubKeyMan
does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet withdisable_private_keys
needs to be used for watchonly things.A
--descriptor
option was added to some tests (wallet_basic.py
,wallet_encryption.py
,wallet_keypool.py
,wallet_keypool_topup.py
, andwallet_labels.py
) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (importprivkey
,importpubkey
,importaddress
,importmulti
,addmultisigaddress
,dumpprivkey
,dumpwallet
,importwallet
, andsethdseed
).