Basic Taproot signing support for descriptor wallets

[sipa]

Builds on top of #22051, adding signing support after derivation support.

Nothing is changed in descriptor features. Signing works for key path and script path spending, through the normal sending functions, and PSBT-based RPCs. However, PSBT usability is rather low as no extensions have been defined to convey Taproot-specific information, so all script information must be known to the signing wallet.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• Add support for inferring tr() descriptors #22166 (Add support for inferring tr() descriptors by sipa)
• Implement BIP-119 Validation (CheckTemplateVerify) #21702 (Implement BIP-119 Validation (CheckTemplateVerify) by JeremyRubin)
• rpc, gui: bumpfee signer support #21576 (rpc: bumpfee signer support by Sjors)
• Implement BIP 370 PSBTv2 #21283 (Implement BIP 370 PSBTv2 by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[sipa]

Rebased on top of #21246, renamed "inner key" everywhere in this PR to "internal key" as well, and added very basic PSBT support (you can have a watchonly tr(KEY) wallet now, and get it signed by an offline version that has the private key).

[Sjors]

Concept ACK 🎉

[roconnor-blockstream]

TBH, I find your "must be specified in depth-first search order" somewhat confusing. As an alternative syntax I suggest simply using some notation for a binary or. For example.

• tr(KEY): only taproot output key known (or is it an internal pubkey with common NUMS tweak??)
• tr(KEY,A): internal pubkey with a single script A at level 0.
• tr(KEY,[A|B]): internal pubkey with a two scripts, A and B at level 1.
• tr(KEY.[[A|B]|C]): internal pubkey with a two scripts, A and B at level 2, and a third script at level 1.
• tr(KEY.[A|[B|C]]): same as above except for A is at level 1 and C is at level 2 instead.
• tr(KEY.[[B|C]|A]): equal to the above descriptor.
• tr(KEY.[[A|B]|[C|D]]): internal pubkey with a four scripts, all at level 2.
• tr(KEY.[[A|C]|[B|D]]): same four scripts as above, again all at level 2 but now associated differently which makes for a different tweak.
• tr(KEY.[A|[[B|C]|D]]): same four scripts as above, but no longer all at the same level.

My claimed advantages are the implicitness of the depth, and clearer notation indicating how the scripts are associated amongst each other.

The notation above is just an example and don't care too much about the specific syntax.

---

Other things to include in the list of features to be added later, a MerkleRoot descriptor for fixed but unknown branches.

[sipa]

@roconnor-blockstream Yeah, I considered something like that, and am open to it. Both notations are equivalent in that you can fairly easily convert in both directions between them (counting bracing level in yours corresponds to the depth specified explicitly in mine, with leaves ultimately listed in the same order). The advantage I see to the current one is that I think it's a bit more readable, avoiding the need to go count braces to see relative depth of leaves. Yours is perhaps a bit easier to write (but maybe that's not something we really expect humans to do much for nontrivial cases).

There are many more variations possible. One extreme would be permitting arbitrary weights for every node, and have the descriptor expansion convert it to a Huffman tree. I'd prefer not to do this because it's adding information that can't be inferred back from the script tree it is converted to.

[roconnor-blockstream]

Right. My concern with explicit levels is that it is so easy to violate the invariant that the sum of the weights has to add up to 1, and you cannot really splice together subexpressions to compose a descriptor from fragments without going through and globally adjusting all the levels to be sane.

[sipa]

Rebased.

[achow101]

re-ACK 458a345

[Sjors]

I'll try to review this today. Having this feature in the signet / testnet wallet makes it easier for people to test taproot before it activates, and potentially find problems (bugs, many eyes, shallow, etc). So it seems worth merging even slightly after the feature freeze.

[michaelfolkson]

Concept ACK, Approach ACK. Updated syntax choice makes sense to me under the readability assumption that use cases that need more than say 2 levels will be rarer. Ignoring readability, the updated syntax choice seems superior due to arguments given.

Agree with @Sjors on importance of getting this in for 22.0. Two weeks for any bug fixes from today should be ok and protections in place (e.g. #22156) regardless to prevent users from spending mainnet Taproot outputs pre-activation (November).

[Sjors]

Very light testing using this PR rebased onto #22260: I'm still able to spend from my tr(xpub...) and tr(xpub1, pk(xpub2)) signet descriptor wallets in the GUI.

[fjahr]

Code review ACK 458a345

I don't consider any of my comments blocking. Will test over the next days.

[fjahr]

nit: this could go before the block with sigdata.taproot_key_path_sig.size() == 0 to return early and then that other block would not need that other if statement

[sipa]

I tried this for a bit, but I don't find the result cleaner.

[fjahr]

nit: m_tr_spenddata?

[sipa]

It's a struct, not a class, so the developer notes don't require that (it's inconsistently applied through the codebase, in some cases using m_ prefixes for struct members too, but I don't think that's enough reason to do so here).

[Sjors]

ACK 458a345

[S3RK]

two comments

[hebasto]

On macOS Mojave 10.14.6 (18G9216) system clang fails to compile this line due to the lack of std::set::merge support.

$ clang --version
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

See: https://stackoverflow.com/questions/50444908/clang-6-not-supporting-unordered-mapmerge

Reported by @S3RK.

[maflcko]

I presume this would also make it harder to compile on https://packages.debian.org/stretch/gcc

[hebasto]

Since #20413 we support gcc 7+ only, right?

[maflcko]

In theory there is https://packages.debian.org/stretch/clang-7, but I don't think anyone was able to compile master with system packages on stretch recently.

[maflcko]

Confirmed (for fun) that the issue exists with libc++7.

# make V=1
Making all in src
make[1]: Entering directory '/bitcoin/src'
make[2]: Entering directory '/bitcoin/src'
make[3]: Entering directory '/bitcoin'
make[3]: Leaving directory '/bitcoin'
/usr/bin/ccache clang++-7 -stdlib=libc++ -std=c++17 -DHAVE_CONFIG_H -I. -I../src/config   -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -I. -I./secp256k1/include  -DBOOST_SP_USE_STD_ATOMIC -DBOOST_AC_USE_STD_ATOMIC -I/usr/include -I./leveldb/include -I./leveldb/helpers/memenv -I./univalue/include -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS -DPROVIDE_FUZZ_MAIN_FUNCTION -fdebug-prefix-map=/bitcoin/src=. -Wstack-protector -fstack-protector-all -Wall -Wextra -Wgnu -Wformat -Wformat-security -Wvla -Wshadow-field -Wswitch -Wthread-safety -Wrange-loop-analysis -Wredundant-decls -Wunused-variable -Wunused-member-function -Wdate-time -Wconditional-uninitialized -Wsign-compare -Woverloaded-virtual -Wunreachable-code-loop-increment -Wno-unused-parameter -Wno-self-assign -Wno-unused-local-typedef -Wno-implicit-fallthrough    -fPIE -g -O2 -MT script/libbitcoin_common_a-standard.o -MD -MP -MF script/.deps/libbitcoin_common_a-standard.Tpo -c -o script/libbitcoin_common_a-standard.o `test -f 'script/standard.cpp' || echo './'`script/standard.cpp
script/standard.cpp:410:22: error: no member named 'merge' in 'std::__1::set<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >, ShortestVectorFirstComparator, std::__1::allocator<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > >'
        scripts[key].merge(std::move(control_blocks));
        ~~~~~~~~~~~~ ^
1 error generated.
Makefile:7756: recipe for target 'script/libbitcoin_common_a-standard.o' failed
make[2]: *** [script/libbitcoin_common_a-standard.o] Error 1
make[2]: Leaving directory '/bitcoin/src'
Makefile:15489: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/bitcoin/src'
Makefile:820: recipe for target 'all-recursive' failed
make: *** [all-recursive] Error 1

[hebasto]

See #22339.