Basic Miniscript support in output descriptors

[sipa]

This pull request introduces support for miniscript in Bitcoin Core.

The bulk of the code is in the 3 commits that add the miniscript module, including conversion from/to CScript, converting to and parsing from its engineer-readable string notation, property analysis and ops limit/stack size limit that are necessary to assess the security of arbitrary scripts.

A number of tests are included, including tests against known scripts, and against randomly generated scripts.

The final commit integrates the miniscript module into descriptors. This is only rudimentary, as it is not yet integrated in the signing code. I'm including it here to give something accessible to play with, but if desirable I can move that to a later PR as well.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #17056 (descriptors: Introduce sortedmulti descriptor by achow101)
• #16889 (Add some general std::vector utility functions by sipa)
• #16887 (Abstract out some of the descriptor Span-parsing helpers by sipa)
• #16807 (Let validateaddress locate error in Bech32 address by meshcollider)
• #16710 (build: Enable -Wsuggest-override if available by hebasto)
• #16440 (BIP-322: Generic signed message format by kallewoof)
• #15590 (Descriptor: add GetAddressType() and IsSegWit() by Sjors)
• #13751 (Utils and libraries: Drops the boost/algorithm/string/split.hpp dependency by l2a5b1)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[practicalswift]

Concept ACK

Thanks for the great work on miniscript and the C++ implementation!

This version seems to deviate somewhat to the version in the upstream repo which makes it unclear to me which of the issues I found during my review that have been addressed:

• Avoid termination due to unhandled exception when parsing invalid arguments to after(...), older(...), thresh(...) and thresh_m(...)
• Avoid assertion failure on after(k) or older(k) where k < 1 or k >= 0x80000000UL
• Avoid running out of stack space by limiting recursion depth in Parse(...)
• Policies with too high nesting depth are not rejected: It is possible to create small inputs that cause extreme memory usage (in practice: OOM kill or std::bad_alloc)

Could you clarify? :-)

[sipa]

@practicalswift Thanks for reminding me, I had forgotten about those (and wasn't really looking at the miniscript repo while working on this PR). Specifically:

• The overflow/underflow issue and the OOM issue don't apply here, as they're in the compiler code which I'm not PR'ing here.
• The unhandled exception I've fixed independently because stoul and friends are locale dependent (both here and upstream).
• The stack depth we need to fix still, I've commented on the issue.

Also, this PR should be perfectly in sync with the upstream repo, so if you want to PR fixes, they can go there preferably, and I'll incorporate them here.

[practicalswift]

I'm afraid a carefully constructed script is able to trigger a heap out-of-bounds read in Node::CalcOps (called indirectly from miniscript::FromScript).

Node::CalcOps appears to be reachable via RPC calls listunspent, scantxoutset and getaddressinfo (see call graph below).

Code:

bitcoin/src/script/miniscript.h

Lines 451 to 462 in 037e55a

	case NodeType::THRESH: {
	uint32_t stat = 0;
	auto sats = Vector(internal::MaxInt<uint32_t>(0));
	for (const auto& sub : subs) {
	stat += sub->ops.stat + 1;
	auto next_sats = Vector(sats[0] + sub->ops.dsat);
	for (size_t j = 1; j < sats.size(); ++j) next_sats.push_back(Choose(sats[j] + sub->ops.dsat, sats[j - 1] + sub->ops.sat));
	next_sats.push_back(sats[sats.size() - 1] + sub->ops.sat);
	sats = std::move(next_sats);
	}
	return {stat, sats[k], sats[0]};
	}

Note that k is not necessarily within bounds.

Call graph:

• listunspent/scantxoutset/getaddressinfo → InferDescriptor → InferScript → miniscript::FromScript → DecodeMulti → DecodeSingle → MakeNodeRef → Node ctor → Node::CalcOps

(Also posted to the original repo as sipa/miniscript#12)

[practicalswift]

The smallest Bitcoin script I've been able to construct that triggers this heap out-of-bounds read is OP_0 OP_2 OP_EQUAL (00 52 87).

[practicalswift]

Found another somewhat related issue: an assertion failure in ComputeType is hit when processing the script OP_0 OP_0 OP_EQUAL (00 00 87).

bitcoin/script/miniscript.cpp:54: 
    miniscript::Type miniscript::internal::ComputeType(…):
    Assertion `k > 1 && k < n_subs' failed.

It is AFAICT reachable via the same code paths as the Node::CalcOps discussed above.

(Also posted to the original repo as sipa/miniscript#13).

[promag]

Concept ACK.

You could submit the refactors as separate pulls one by one. For instance, one for fc46918 which I think is acceptable on its own.

[DrahtBot]

Needs rebase

[practicalswift]

@sipa Would you mind cherry-picking in the miniscript::FromScript(...) fuzzer from #17129 in to this PR? That would allow for closing #17129 which is entirely dependent on the merge of this one anyway :)

[practicalswift]

@fanquake Could you add "Waiting for author"? :)

[MarcoFalke]

"Needs rebase" implies "Waiting for author"