Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

O(1) OP_IF/NOTIF/ELSE/ENDIF script implementation #16902

Open
wants to merge 3 commits into
base: master
from

Conversation

@sipa
Copy link
Member

sipa commented Sep 18, 2019

While investigating what mechanisms are possible to maximize the per-opcode verification cost of scripts, I noticed that the logic for determining whether a particular opcode is to be executed is O(n) in the nesting depth. This issue was also pointed out by Sergio Demian Lerner in https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/, and this PR implements a variant of the O(1) algorithm suggested there.

This is not a problem currently, because even with a nesting depth of 100 (the maximum possible right now due to the 201 ops limit), the slowdown caused by this on my machine is around 70 ns per opcode (or 0.25 s per block) at worst, far lower than what is possible with other opcodes.

This PR mostly serves as a proof of concept that it's possible to avoid it, which may be relevant in discussions around increasing the opcode limits in future script versions. Without it, the execution time of scripts can grow quadratically with the nesting depth, which very quickly becomes unreasonable.

This improves upon #14245 by completely removing the vfExec vector.

@sipa sipa force-pushed the sipa:201909_o1nestedifs branch from 0502341 to 55b7ae8 Sep 18, 2019
@DrahtBot

This comment has been minimized.

Copy link
Contributor

DrahtBot commented Sep 18, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #13062 (Make script interpreter independent from storage type CScript by sipa)
  • #10729 (Wrap EvalScript in a ScriptExecution class by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ajtowns

This comment has been minimized.

Copy link
Contributor

ajtowns commented Sep 18, 2019

It might be easier to review if the operations on vfExec are pulled out into a separate class first, so that the optimisation just means updating the class from:

class FalseCounter {
private:
    std::vector<bool> flags;
public:
    bool all_true() { return !count(flags.begin(), flags.end(), false); }
    void push_back(bool f) { flags.push_back(f); }
    void pop_back() { flags.pop_back(); }
    bool empty() { return flags.empty(); }
    void toggle_top() { flags.back() = !flags.back(); }
};

to

class FalseCounter {
private:
    int depth = 0;
    int first_false = 0;
public:
    bool all_true() { return first_false == 0; }
    void push_back(bool f) {
        ++depth;
        if (first_false == 0 && !f) first_false = depth;
    }
    void pop_back() {
        if (first_false == depth) first_false = 0;
        --depth;
    }
    bool empty() { return depth == 0; }
    void toggle_top() {
        if (first_false == 0) {
            first_false = depth;
        } else if (first_false == depth) {
            first_false = 0;
        } else {
            // no action needed as change is unobservable
        }
    }
};

without changing the usage of the class. I think it'd be plausible to do a coq proof that those implementations are logically equivalent / implement the same API (at least given !empty() preconditions for pop_back() and toggle_top()).

cf https://github.com/ajtowns/bitcoin/commits/201909-vfexec-optimise

@sipa

This comment has been minimized.

Copy link
Member Author

sipa commented Sep 18, 2019

@ajtowns If there's interest in including this patch at some point we should pick up that approach.

@practicalswift

This comment has been minimized.

Copy link
Member

practicalswift commented Sep 18, 2019

Somewhat related issue: sipa/miniscript#7 ("Policies with too high nesting depth are not rejected: It is possible to create small inputs that cause extreme memory usage (in practice: OOM kill or std::bad_alloc)")

Context: Nested tresh tend to be very OP_IF/OP_ELSE/OP_ENDIF intensive (or_i) when compiled to script :)

@sipa sipa force-pushed the sipa:201909_o1nestedifs branch from 55b7ae8 to 16155cc Nov 4, 2019
@sipa sipa changed the title [POC] O(1) OP_IF/NOTIF/ELSE/ENDIF script implementation O(1) OP_IF/NOTIF/ELSE/ENDIF script implementation Nov 4, 2019
@sipa sipa force-pushed the sipa:201909_o1nestedifs branch from 16155cc to 7ff02f9 Nov 4, 2019
@sipa

This comment has been minimized.

Copy link
Member Author

sipa commented Nov 4, 2019

I've modified the code to follow @ajtowns's approach more closely, though with fewer commits, and extra comments.

@sipa sipa force-pushed the sipa:201909_o1nestedifs branch 2 times, most recently from 132225a to d4721cd Nov 4, 2019
src/script/interpreter.cpp Outdated Show resolved Hide resolved
src/script/interpreter.cpp Outdated Show resolved Hide resolved
@sipa sipa force-pushed the sipa:201909_o1nestedifs branch 3 times, most recently from 3d09358 to 59c0b4e Nov 6, 2019
sipa and others added 3 commits Sep 18, 2019
Includes comments added by Pieter Wuille.
This optimization was first suggested by Sergio Demian Lerner in
https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/.
The implementation follows the suggested approach there, but with a slightly
simpler representation.
@sipa sipa force-pushed the sipa:201909_o1nestedifs branch from 59c0b4e to e6e622e Nov 7, 2019
nkohen added a commit to nkohen/bitcoin-s-core that referenced this pull request Nov 7, 2019
Christewart added a commit to bitcoin-s/bitcoin-s that referenced this pull request Nov 8, 2019
* Redid conditional interpreting without binary trees

* Fixed ControlOperationsInterpreterTest

* Implemented O(1) conditional handling as proposed here bitcoin/bitcoin#16902 because Ben Carman pointed it out

* Added some docs

* Responded to code review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.