O(1) OP_IF/NOTIF/ELSE/ENDIF script implementation

[sipa]

While investigating what mechanisms are possible to maximize the per-opcode verification cost of scripts, I noticed that the logic for determining whether a particular opcode is to be executed is O(n) in the nesting depth. This issue was also pointed out by Sergio Demian Lerner in https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/, and this PR implements a variant of the O(1) algorithm suggested there.

This is not a problem currently, because even with a nesting depth of 100 (the maximum possible right now due to the 201 ops limit), the slowdown caused by this on my machine is around 70 ns per opcode (or 0.25 s per block) at worst, far lower than what is possible with other opcodes.

This PR mostly serves as a proof of concept that it's possible to avoid it, which may be relevant in discussions around increasing the opcode limits in future script versions. Without it, the execution time of scripts can grow quadratically with the nesting depth, which very quickly becomes unreasonable.

This improves upon #14245 by completely removing the vfExec vector.

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #13062 (Make script interpreter independent from storage type CScript by sipa)
• #10729 (Wrap EvalScript in a ScriptExecution class by luke-jr)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[ajtowns]

It might be easier to review if the operations on vfExec are pulled out into a separate class first, so that the optimisation just means updating the class from:

class FalseCounter {
private:
    std::vector<bool> flags;
public:
    bool all_true() { return !count(flags.begin(), flags.end(), false); }
    void push_back(bool f) { flags.push_back(f); }
    void pop_back() { flags.pop_back(); }
    bool empty() { return flags.empty(); }
    void toggle_top() { flags.back() = !flags.back(); }
};

to

class FalseCounter {
private:
    int depth = 0;
    int first_false = 0;
public:
    bool all_true() { return first_false == 0; }
    void push_back(bool f) {
        ++depth;
        if (first_false == 0 && !f) first_false = depth;
    }
    void pop_back() {
        if (first_false == depth) first_false = 0;
        --depth;
    }
    bool empty() { return depth == 0; }
    void toggle_top() {
        if (first_false == 0) {
            first_false = depth;
        } else if (first_false == depth) {
            first_false = 0;
        } else {
            // no action needed as change is unobservable
        }
    }
};

without changing the usage of the class. I think it'd be plausible to do a coq proof that those implementations are logically equivalent / implement the same API (at least given !empty() preconditions for pop_back() and toggle_top()).

cf https://github.com/ajtowns/bitcoin/commits/201909-vfexec-optimise

[sipa]

@ajtowns If there's interest in including this patch at some point we should pick up that approach.

[practicalswift]

Somewhat related issue: sipa/miniscript#7 ("Policies with too high nesting depth are not rejected: It is possible to create small inputs that cause extreme memory usage (in practice: OOM kill or std::bad_alloc)")

Context: Nested tresh tend to be very OP_IF/OP_ELSE/OP_ENDIF intensive (or_i) when compiled to script :)

[sipa]

I've modified the code to follow @ajtowns's approach more closely, though with fewer commits, and extra comments.