Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic Miniscript support in output descriptors #16800

Open
wants to merge 7 commits into
base: master
from

Conversation

@sipa
Copy link
Member

commented Sep 4, 2019

This pull request introduces support for miniscript in Bitcoin Core.

The bulk of the code is in the 3 commits that add the miniscript module, including conversion from/to CScript, converting to and parsing from its engineer-readable string notation, property analysis and ops limit/stack size limit that are necessary to assess the security of arbitrary scripts.

A number of tests are included, including tests against known scripts, and against randomly generated scripts.

The final commit integrates the miniscript module into descriptors. This is only rudimentary, as it is not yet integrated in the signing code. I'm including it here to give something accessible to play with, but if desirable I can move that to a later PR as well.

@sipa sipa force-pushed the sipa:201908_miniscript branch 3 times, most recently from deb2092 to 892acd4 Sep 4, 2019
@DrahtBot

This comment has been minimized.

Copy link
Contributor

commented Sep 4, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17056 (descriptors: Introduce sortedmulti descriptor by achow101)
  • #16889 (Add some general std::vector utility functions by sipa)
  • #16887 (Abstract out some of the descriptor Span-parsing helpers by sipa)
  • #16807 (Let validateaddress locate error in Bech32 address by meshcollider)
  • #16710 (build: Enable -Wsuggest-override if available by hebasto)
  • #16440 (BIP-322: Generic signed message format by kallewoof)
  • #15590 (Descriptor: add GetAddressType() and IsSegWit() by Sjors)
  • #13751 (Utils and libraries: Drops the boost/algorithm/string/split.hpp dependency by l2a5b1)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@practicalswift

This comment has been minimized.

src/script/miniscript.h Outdated Show resolved Hide resolved
src/test/miniscript_tests.cpp Outdated Show resolved Hide resolved
src/script/miniscript.h Outdated Show resolved Hide resolved
@sipa sipa force-pushed the sipa:201908_miniscript branch 2 times, most recently from d0804c7 to 7912f91 Sep 4, 2019
@sipa

This comment has been minimized.

Copy link
Member Author

commented Sep 4, 2019

@practicalswift Thanks for reminding me, I had forgotten about those (and wasn't really looking at the miniscript repo while working on this PR). Specifically:

  • The overflow/underflow issue and the OOM issue don't apply here, as they're in the compiler code which I'm not PR'ing here.
  • The unhandled exception I've fixed independently because stoul and friends are locale dependent (both here and upstream).
  • The stack depth we need to fix still, I've commented on the issue.

Also, this PR should be perfectly in sync with the upstream repo, so if you want to PR fixes, they can go there preferably, and I'll incorporate them here.

@sipa sipa force-pushed the sipa:201908_miniscript branch 5 times, most recently from 0e511de to 046a0b9 Sep 4, 2019
src/script/miniscript.h Outdated Show resolved Hide resolved
@sipa sipa force-pushed the sipa:201908_miniscript branch from 046a0b9 to 4890d97 Sep 5, 2019
sipa added 2 commits Jul 30, 2019
Added are:

* Vector(arg1,arg2,arg3,...) constructs a vector with the specified
  arguments as elements. The vector's type is derived from the
  arguments. If some of the arguments are rvalue references, they
  will be moved into place rather than copied (which can't be achieved
  using list initialization).

* Cat(vector1,vector2) returns a concatenation of the two vectors,
  efficiently moving elements when relevant.

Vector generalizes (and replaces) the Singleton function in
src/descriptor.cpp, and Cat replaces the Cat function in bech32.cpp
@sipa sipa force-pushed the sipa:201908_miniscript branch from 4890d97 to 5fae3f1 Sep 5, 2019
@sipa sipa force-pushed the sipa:201908_miniscript branch from 5fae3f1 to 037e55a Sep 5, 2019
@practicalswift

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

I'm afraid a carefully constructed script is able to trigger a heap out-of-bounds read in Node::CalcOps (called indirectly from miniscript::FromScript).

Node::CalcOps appears to be reachable via RPC calls listunspent, scantxoutset and getaddressinfo (see call graph below).

Code:

case NodeType::THRESH: {
uint32_t stat = 0;
auto sats = Vector(internal::MaxInt<uint32_t>(0));
for (const auto& sub : subs) {
stat += sub->ops.stat + 1;
auto next_sats = Vector(sats[0] + sub->ops.dsat);
for (size_t j = 1; j < sats.size(); ++j) next_sats.push_back(Choose(sats[j] + sub->ops.dsat, sats[j - 1] + sub->ops.sat));
next_sats.push_back(sats[sats.size() - 1] + sub->ops.sat);
sats = std::move(next_sats);
}
return {stat, sats[k], sats[0]};
}

Note that k is not necessarily within bounds.

Call graph:

  • listunspent/scantxoutset/getaddressinfoInferDescriptorInferScriptminiscript::FromScriptDecodeMultiDecodeSingleMakeNodeRefNode ctorNode::CalcOps

(Also posted to the original repo as sipa/miniscript#12)

@practicalswift

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

The smallest Bitcoin script I've been able to construct that triggers this heap out-of-bounds read is OP_0 OP_2 OP_EQUAL (00 52 87).

@practicalswift

This comment has been minimized.

Copy link
Member

commented Sep 12, 2019

Found another somewhat related issue: an assertion failure in ComputeType is hit when processing the script OP_0 OP_0 OP_EQUAL (00 00 87).

bitcoin/script/miniscript.cpp:54: 
    miniscript::Type miniscript::internal::ComputeType(…):
    Assertion `k > 1 && k < n_subs' failed.

It is AFAICT reachable via the same code paths as the Node::CalcOps discussed above.

(Also posted to the original repo as sipa/miniscript#13).

@promag

This comment has been minimized.

Copy link
Member

commented Sep 15, 2019

Concept ACK.

You could submit the refactors as separate pulls one by one. For instance, one for fc46918 which I think is acceptable on its own.

@DrahtBot

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2019

Needs rebase
MarcoFalke added a commit that referenced this pull request Oct 10, 2019
bb36372 test: add unit tests for Span-parsing helpers (Sebastian Falbesoner)
5e69aee Add documenting comments to spanparsing.h (Pieter Wuille)
230d43f Abstract out some of the descriptor Span-parsing helpers (Pieter Wuille)

Pull request description:

  As suggested here: #16800 (comment).

  This moves the Span parsing functions out of the descriptor module, making them more easily usable for other parsers (in particular, in preparation for miniscript parsing).

ACKs for top commit:
  MarcoFalke:
    ACK bb36372

Tree-SHA512: b5c5c11a9bc3f0a1c2c4cfa22755654ecfb8d4b69da0dc1fb9f04e1556dc0f6ffd87ad153600963279ac465d587d7971b53d240ced802d12693682411ac73deb
sidhujag added a commit to syscoin/syscoin that referenced this pull request Oct 11, 2019
… helpers

bb36372 test: add unit tests for Span-parsing helpers (Sebastian Falbesoner)
5e69aee Add documenting comments to spanparsing.h (Pieter Wuille)
230d43f Abstract out some of the descriptor Span-parsing helpers (Pieter Wuille)

Pull request description:

  As suggested here: bitcoin#16800 (comment).

  This moves the Span parsing functions out of the descriptor module, making them more easily usable for other parsers (in particular, in preparation for miniscript parsing).

ACKs for top commit:
  MarcoFalke:
    ACK bb36372

Tree-SHA512: b5c5c11a9bc3f0a1c2c4cfa22755654ecfb8d4b69da0dc1fb9f04e1556dc0f6ffd87ad153600963279ac465d587d7971b53d240ced802d12693682411ac73deb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.