Add scriptPubKey enforced sendescrow and redeemescrow API calls

[groffer]

This implements an initial feature set for "escrow" transactions, as mentioned on various threads on the forum. It allows coins to be controlled by threshold voting among multiple parties.

[gavinandresen]

Should be possible to implement this much more cleanly using the MULTISIG opcodes.

Also, what do escrow transactions look like in the GUI?

[groffer]

Should be possible to implement this much more cleanly using the MULTISIG
opcodes.

I overlooked that MULTISIG allows for less signatures than keys. I will
go ahead and rewrite the script.

Also, what do escrow transactions look like in the GUI?

My plan is to make the display look reasonable, but not provide a GUI for
initiating escrow. Escrow requires communicating with another party. The
API allows for an external escrow UI to be implemented and sounds to me
like a better solution than hardcoding into the client. What are your
thoughts?

[davout]

I personnally don't feel this should go into the main client implementation, but in some sort of extension instead

[gmaxwell]

Davout, care to explain why?

Without support in the main client implementation we couldn't expect these transactions to be relayed or mined by most miners. Also an external implementation would require apis for importing/exporting transactions, and wouldn't e.g. have access to the wallet keys and address book. Seems ugly to me.

The api/cli presented in this patch is nice (well, the redeem needs to also support sendmany redemptions). Obviously a GUI for this would take a bit more work— and might reasonably wait until there are some examples of the functionality being used to inform the design.

[TheBlueMatt]

I would like to see some rethinking of IsStandard, but IMO that is quite a way down the road. I would like to see the codebase generally cleaned a ton before that ever happens.

[sacarlson]

I modified zamgo's https://github.com/sacarlson/bitcoin-webskin to support and help document sendescrow and redeemescrow and also incorporated the changes into MultiCoin https://github.com/sacarlson/MultiCoin so that people can start learning how to start using escrow services in the furture. MultiCoin's escrow features have now been tested on testnet and weedsnet and will also be supported on beertokensnet. I hope soon we will start to see escrow on bitcoin mainnet. I will also support any changes in the escrow API format if needed.

[davout]

@gmaxwell just my personal opinion, it doesn't matter that much, and consensus seems to go the other way :)
@TheBlueMatt +1

[groffer]

Changed:

• Rebased on latest head
• Renamed all instances of "escrow" to "multisign" because this is not really escrow in the traditional sense
• Switched to CHECKMULTISIG which did not clean things up that much because of the need to check hash160s
• Added unit tests
• Fixed some unit testing issues with the makefile. No need to compile with GUI, add all the object files and ensure that we get the correct main().

@sacarlson - looks very interesting

[groffer]

Oh, I found a bug in CHECKMULTISIG. It drops one too many items from the stack, so I had to push a dummy value to work around that.

[sacarlson]

Todays tests with me and groffer on his bitcoin branch https://github.com/groffer/bitcoin for multisign transactions were a success on commit d260515 to see detailed IRC session of tests see http://paste.ubuntu.com/637841/ I also merged this with my bitcoin branch MultiCoin for further testing and updated my branch of bitcoin-webskin to include the new features.

[gmaxwell]

It would be nice if redeem worked even when you had non of the required keys, so someone could request payment from an escrow which they can't sign for by forming a txn and sending it around to people to sign.

This would also make it easier for escrow parties who keep their escrow keys offline, — as they could start the escrow release process from a host with the blockchain but gets signed by offline hosts.

[groffer]

@gmaxwell - good point, and actually it already works this way. If you have none of the keys, it will create a tx without any signatures. To increase visibility into the signature collection process, the latest commit adds the list of addresses that have signed the tx to the output of the RPC call.

The latest commit also refactors the script Solver function so it can be used to check more complex scripts that include repeated segments. multisign now uses this facility to perform exact script matching. This prevents an attacker from fooling the user into thinking a tx is protected when it is put fully under the control of the attacker by a maliciously crafted script.

[groffer]

Also added some unit tests for script.cpp.

[gavinandresen]

Can you write up a description of what the values in the TxIn are, and what operations are being done to them in the TxOut to redeem? My puny brain is having trouble figuring out what the ROLL SIZE NOT OVER HASH.. is accomplishing.

Looking at: http://blockexplorer.com/testnet/tx/a17b21f52859ed326d1395d8a56d5c7389f5fc83c17b9140a71d7cb86fdf0f5f#i96368

... the TxIn is:
0 pubkey1 pubkey2 2 sig1 sig2 0

... and the TxOut is:
3 OP_ROLL OP_DUP
2 OP_GREATERTHANOREQUAL OP_VERIFY

3 OP_ROLL OP_SIZE OP_NOT OP_OVER OP_HASH160
80677c5392220db736455533477d0bc2fba65502 OP_EQUAL
OP_BOOLOR OP_VERIFY

3 OP_ROLL OP_SIZE OP_NOT OP_OVER OP_HASH160
02d7aa2e76d9066fb2b3c41ff8839a5c81bdca19 OP_EQUAL
OP_BOOLOR OP_VERIFY

3 OP_ROLL OP_SIZE OP_NOT OP_OVER OP_HASH160
10039ce4fdb5d4ee56148fe3935b9bfbbe4ecc89 OP_EQUAL OP_BOOLOR OP_VERIFY

3 OP_CHECKMULTISIG

[groffer]

There's a bit more operand stack detail in comments on line 1575, and a writeup below.

This is a 2 party out of 3 spend, with one party not participating.

The scriptSig (TxIn) is (reading from right to left): OP_0 (to work around a bug in OP_CHECKMULTISIG), then two participating signatures, then the number of signatures, followed by the two participating pubkeys, then an empty operand holding place for the third pubkey that is not participating in this spend.

The scriptPubKey (TxOut) is (reading from left to right):

3 OP_ROLL OP_DUP // move the number of signatures to the top of the stack, and duplicate it
2 OP_GREATERTHANOREQUAL OP_VERIFY // Make sure the number of signatures is greater than the voting threshold

// In the section below we want to check that the pubkey is either present and hashes to the address we expect
// OR that it is missing and an empty operand holds its place
3 OP_ROLL // move the next pubkey to the top of the stack
OP_SIZE OP_NOT // check if the pubkey is just a placeholder
OP_OVER OP_HASH160 80677c5 OP_EQUAL // check if the pubkey hashes as expected
OP_BOOLOR OP_VERIFY // the pubkey should either be a placeholder or hash as expected


// Same for second participant
3 OP_ROLL OP_SIZE OP_NOT OP_OVER OP_HASH160 02d7aa2 OP_EQUAL OP_BOOLOR OP_VERIFY

// Same for third participant
3 OP_ROLL OP_SIZE OP_NOT OP_OVER OP_HASH160 10039ce OP_EQUAL OP_BOOLOR OP_VERIFY

3 OP_CHECKMULTISIG  // Check the signatures

Note: we do a OP_SIZE OP_NOT to check if a pubkey is a placeholder. If we did OP_0 OP_NE, it would fail because pubkeys cannot be cast to bignums.

I'll be happy to cover this in more detail where needed.

[gavinandresen]

Unless I'm grossly misunderstanding CHECKMULTISIG, the simplest form of a 2-of-3 CHECKMULTISIG would be:

TxIn: sig1 sig2
TxOut: 2 pub1 pub2 pub3 3 CHECKMULTISIG

I've been working on schemes to hash the public keys so that people can use shorter bitcoin addresses...

[gavinandresen]

groffer: appreciate comments on: https://gist.github.com/39158239e36f6af69d6f

[sipa]

An alternative way for handling N-out-of-M multisig with addresses, without CHECKMULTISIG at all:

scriptSig:

• for each pubkey/signature that is provided: [signature] [pubkey]
• for each pubkey/signature that is missing: OP_0 OP_0

So for 2-out-of-3, with the second one missing:

• [signature3] [pubkey3]
• OP_0 OP_0
• [signature1] [pubkey1]

scriptPubKey:

• for each of the M addresses:
  • OP_HASH160 [hash160] OP_EQUAL, (check if pubkey matches address)
    • OP_IF OP_CHECKSIG (put validness of sig on top of stack)
  • OP_ELSE OP_DROP OP_0 (skip signature, and put 0 on top of stack)
  • OP_ENDIF
  • for all but the first address: OP_FROMALTSTACK OP_ADD
  • for all but the last address; OP_TOALTSTACK
    (count number of valids on number on altstack)
• finally, at the end:
  • n OP_GREATERTHANOREQUAL

This means output scripts of 12_M-1 bytes (+ M_20 bytes for the addresses), and input scripts of 2*M bytes (+ N times the size of a signature + pubkey).

[gavinandresen]

groffer: is sipa's suggestion close to what you started with?

I'm thinking about whether or not I like the ability to create a multisig address if all you know is the hashes of the public keys. If I'm understanding, use case is:

• I know 3 bitcoin addresses, create a (say) 2-of-3 address
• People send to the 2-of-3 address
• Sometime later, 2 full public keys and signatures are gathered, and the coins are spent.

Advantage is the signature and public key gathering can happen all at once.

[gavinandresen]

I think this works (somebody check my work) if "we" decide that redeem-with-only-m-full-public-keys is important:

TxIn: s1 s2 2 p1 OP_0 p3
TxOut:
3DUP -- duplicate public keys so we can check hashes
HASH160 ... EQUAL TOALTSTACK
HASH160 ... EQUAL FROMALTSTACK ADD TOALTSTACK
HASH160 ... EQUAL FROMALTSTACK ADD 2 GREATERTHANOREQUAL VERIFY
3 CHECKMULTISIG

... and is smaller than sipa's suggestion.

(although I'm not sure what requirements CHECKMULTISIG puts on public keys, might have to replace the OP_0 with a properly-formatted public key).

[groffer]

@sipa - that's actually very similar to what I had before switching to OP_CHECKMULTISIG in dc2dfba (search for OP_TUCK). I can revert to that if it's preferred.

@gavinandresen - yes, otherwise you have to distribute the pubkeys ahead of time.

[groffer]

(sorry for the skew - my response was before I saw last comment from @gavinandresen)

Yes, CHECKMULTISIG is okay with a malformed pubkey (CheckSig returns false, and the loop moves on to the next pubkey).

But the last CHECKMULTISIG solution above can be redeemed with just one correct signature, as coblee explains in the gist.

[groffer]

I would love to add the case of (a AND b) OR c as mentioned in the gist, and for that matter a generalized disjunctions () OR () OR () .

This will enable emergency backup signer. It will also enable contracts based on broadcast information. For example with sender and receiver keys Ks, Kr, a coin with scriptSig (Ks AND K1) OR (Kr AND K2) could be redeemed by sender if private key for K1 is broadcast or by receiver if private key for K2 is broadcast.

Should I wait with disjunctions until after this pull is decided, or should I add now?

The other question is whether to go with CHECKMULTISIG. Right now I have padding to get around the GetSigOpCount check. I just noticed that CheckBlock uses that, so we won't really be able to change GetSigOpCount anytime soon (and my change to reduce the MULTISIG count from 20 to 5 has to be reverted). Should I revert to the original CHECKSIG implementation?

[gmaxwell]

On Wed, Aug 24, 2011 at 3:18 PM, groffer
reply@reply.github.com
wrote:

Do you mean just the IsStandard part?  I can do that.  We should make a decision on a couple of questions:

• Use coblee CHECKSIG solution?
• Add disjunctions?

I'd rather do this change once instead of twice. Unless the
disjunction script gives people security fears I'd rather it be
supported.

[sipa]

As long as each hash160 is used only once, you can write each boolean expression in reverse-polish notation, and use coblee's technique to evaluate it.

For example: a1 OR (a2 AND a3) OR COUNT(a4,a5,a6)>1
In RPN: a1 a2 a3 AND OR a4 a5 ADD a6 ADD 1 GREATER OR

For each address element, you need to know the number n of elements on the stack that belong to the RPN processing - all those beneath are pubkey/signature arguments.

Here is the same example in RPN notation with stack depth annotated:
(0) a1 (1) a2 (2) a3 (3) AND (2) OR (1) a4 (2) a5 (3) ADD (2) a6 (3) ADD (2) 1 (3) GREATER (2) OR (1)

The first step is to bring the pubkey and signature that are in configuration: [sig] [pubkey] [n elements] into configuration [n elements] [pubkey] [signature] [pubkey]:

• n=0: TUCK
• n=1: OVER 2SWAP
• n=2: 2SWAP TUCK
• n=3: 3 PICK 2ROT
• n=4: 2ROT TUCK
• other: [n] ROLL [n] ROLL TUCK
  Afterwards follows: CHECKSIG SWAP HASH160 [hash160] EQUAL BOOLAND

Other operations in the RPN form are simply passed through as-is (they manipulate the RPN stack, which is now on top).

That means the above example becomes:

• (0)a1: TUCK CHECKSIG SWAP HASH160 [a1] EQUAL BOOLAND
• (1)a2: OVER 2SWAP CHECKSIG SWAP HASH160 [a2] EQUAL BOOLAND
• (2)a3: 2SWAP TUCK CHECKSIG SWAP HASH160 [a3] EQUAL BOOLAND
• AND: BOOLAND
• OR: BOOLOR
• (1)a4: OVER 2SWAP CHECKSIG SWAP HASH160 [a4] EQUAL BOOLAND
• (2)a5: 2SWAP TUCK CHECKSIG SWAP HASH160 [a5] EQUAL BOOLAND
• ADD: ADD
• (2)a6: 2SWAP TUCK CHECKSIG SWAP HASH160 [a6] EQUAL BOOLAND
• ADD: ADD
• 1: 1
• GREATER: GREATERTHAN
• OR: BOOLOR

This scheme allows any expression over signatures, given in RPN notation to be compiled to bitcoin scripts.

It can even be extended to support pubkey-based matching instead of address-based matching. Instead of bringing two elements to the front, one suffices:

• n=0:
• n=1: SWAP
• other: [n] ROLL
  Followed by a simple: [pubkey] CHECKSIG

If an address or pubkey is present more than once in the expression, put its evaluation logic (together with the corresponding expected position of input arguments) in front. Each time it is used in the RPN processing, use [m] PICK to retrieve its evaluation, except the last time, when you use [m] ROLL.

[groffer]

I would love to implement the more general solution from @sipa if @gavinandresen is on board with that.

[gavinandresen]

What does the code for IsStandard() look like with sipa's generalized case?
How does higher-level code figure out what the heck kind of transaction it is dealing with when there's a tangle of TUCKS and GREATERTHAN?

I am NOT on board with implementing sipa's general solution-- can we please start with the cases we know are useful right now, and talk about generalizing when we've got some experience with the simpler multi-sign cases?

[groffer]

To implement sipa's suggestion we would need RPL -> Script and Script -> RPL conversion functions. Higher level code would generate and look for transactions with specific RPL templates.

To start with, IsStandard would accept RPL code within specific limits (recursion depth and operators). The next step would be to write API calls that deals simple multisign RPL code cases. For example: count(a1, a2, .. , an) > m .

From a user perspective, there would probably not be a big difference initially, just a difference in IsStandard.

[sipa]

The first step is probably adding some extra specific standard cases to the solver, like 1-of-2, 1-of-3, 2-of-3, a-and-(b-or-c) (anything else that is known to be useful?). This is relatively easy to test, and as IsStandard simpy checks for solutions to an unknown keystore, IsStandard would support them immediately.

To generalize IsStandard, you indeed just need to split the script into components. Each is either

• an address check with matching stack depth
• a pubkey check with matching stack depth
• an [m] PICK or [m] ROLL with m<n (n = stack depth)
• a supported RPN operator (small number, ADD, BOOLAND, BOOLOR, comparison operators)
  When the entire script can be decomposed into these components, and the final stack depth is 1, the script is valid/standard.

To actually solve such scripts in general, you'll indeed want to convert to RPN again, but that can be postponed.

PS: shame on me; i was talking about reverse polish notation, but somehow started using the abbreviation "RPL"...

[groffer]

I think it is easier to check the correctness of a relatively general solution than to have a growing set of special cases. We already have a simple general solution in the current code that does general COUNT(a1..an) > m and is pretty easy to validate.

I will switch to back to a CHECKSIG solution since it is simpler, eliminates the need to pad (for SigOpCount workaround) and more flexible for the future.

I could also do a general RPN IsStandard - a rudimentary componentized solver is already in the code and as @sipa says, there are only a few cases to consider for each component.

[gmaxwell]

At the very minimum, even if we don't go for full RPN support now the transaction encoding for the subset we do support should be in the RPN encoding ordering so that extending the support in the future doesn't result in yet another transaction type.

[groffer]

@gmaxwell good point.

[gavinandresen]

The right place for this discussion is the bitcoin-dev mailing list. See Mike Hearn's concerns, and see if you can convince him that supporting arbitrary combinations of keys will be easier.

[sipa]

As far as I understand it, that was about putting arbitrary such expressions in addresses, not about having them pass IsStandard(). Anyway, continuing the discussion there...

[gavinandresen]

Anybody mind if I close this? BIPs 11 12 and 13 (and my OP_EVAL pull request) are the replacement.