Segregated Witness Support - WIP #520

rubensayshi · 2016-01-15T12:41:09Z

WORK IN PROGRESS

getting shit to work before making it pretty; almost done with getting shit to work!

depends on: #540

first time spending a p2wpkh (6a539927494f3d6f07078dd1c70a28e0c4910662593072305529eaa66257f59d) and p2wpkh as p2sh (4704de796b8ea819f262eb3399fad85b838327497639da2ceafbd04a08a06be3) with bitcoinjs-lib :D

ToDo

Usage

Signing Native P2WPKH

transactionBuilder.sign(0, keyPair, null, null, /* segWit = */ true, /* txIn.value = */ parseInt(0.9998 * 1e8)))

Signing P2WPKH nested in P2SH

var pubKeyHash = bcrypto.hash160(keyPair.getPublicKeyBuffer())
var redeemScript = bscript.segWitPubKeyHashOutput(pubKeyHash)

transactionBuilder.sign(0, keyPair, redeemScript, null, /* segWit = */ true, /* txIn.value = */ parseInt(0.9998 * 1e8)))

References

BIP141 (general segwit) https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki
BIP142 (segwit address) https://github.com/bitcoin/bips/blob/master/bip-0142.mediawiki
BIP143 (signature verification with new signature hash) https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki
BIP144 (peer services) https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki
Staging branch https://github.com/sipa/bitcoin/tree/segwit

…s/bitcoinjs-lib#520)

fanatid · 2016-01-23T11:51:13Z

src/transaction.js

@rubensayshi it isn't python 😆

hehe yea noticed that yday, been switching between projects too much :P

dcousens · 2016-01-23T14:27:21Z

@rubensayshi when is the next time you can be available on slack? It'd be good to talk over how this [and future changes] will integrate with TransactionBuilder and Transaction.

rubensayshi · 2016-01-24T12:03:53Z

@dcousens I've got slack on my phone so you can always poke me but I usually prefer keeping my weekends code-free, though this weekend is an exception and working on this PR atm :D
poke me whenever you have time this week, I'm available all week (GMT+1)

rubensayshi · 2016-01-24T12:32:54Z

src/types.js

is this okay to do ?

rubensayshi · 2016-01-24T12:52:28Z

src/address.js

not sure if this should go here or in fromBase58Check, considering the length checks are in fromBase58Check, this code has gotten a bit akward now ...

consider one of the test vectors for fromBase58Check we had of an address with hash length of 22, it no longer throws a 'is too long' exception and because we're unaware of network in fromBase58Check we can't really check if it's sane either ...

what was the original reason for seperating fromBase58Check and putting the exceptions there?

also is fromBase58Check considered public API? is it painful to change things?

rubensayshi · 2016-06-28T08:11:31Z

@dcousens I was about to ask you about this, because I noticed the refactoring of a few things ...

let me know if there's anything I can do!

btcdrak · 2016-06-30T06:54:06Z

It would be great to get this finished and merged so it can be used on testnet. Segwit is merged to bitcoin/master and active on Bitcoin testnet. It means people can start working on integration in earnest.

dcousens · 2016-06-30T07:06:02Z

@btcdrak I'll see if I can't work towards a 10th July PR date.

dcousens · 2016-07-14T09:25:00Z

src/transaction.js

+  }
+}
+
+Transaction.prototype.hashForSignatureV2 = function (inIndex, prevOutScript, amount, hashType) {


Why is this necessary?

Not sure if this was cleared up - segwit has a different signature hashing scheme which commits the input amount in the signature, amongst other changes. It has a reusable midstate for all inputs in a transaction which may be cached, so avoids the O(n^2) hashing issue.

dcousens · 2016-07-14T09:29:27Z

Lots of debugging console.log lefts in here

dcousens · 2016-07-14T09:30:28Z

src/bufferwriter.js

@@ -0,0 +1,55 @@
+var bufferutils = require('./bufferutils')


I'd rather opt for something like varstruct than go this route if possible

dcousens · 2016-07-14T09:32:45Z

Ideally it'd be great to do this in 4 PRs.

scripts (#602)
Transaction
TransactionBuilder
address

dcousens · 2016-07-14T09:33:22Z

Cache hashPrevouts, hashSequence and hashOutputs for all signatures in a transaction (since that was one of the main reasons for changing the signatureHash xD)

Why are we doing such a change in this PR?

btcdrak · 2016-10-04T21:16:33Z

bump

btcdrak · 2016-10-04T21:17:58Z

Note for OP, BIP142 was deferred

btcdrak · 2016-10-04T21:25:48Z

The witness program size was extended from 32 to max 40 bytes https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Witness_program

dcousens · 2016-10-04T23:35:28Z

@btcdrak support for SegWit scripts was already added a small while ago. #602
This PR is just serving as a reminder to implement the transaction serialisation logic.

See https://github.com/bitcoinjs/bitcoinjs-lib/blob/master/src/script.js#L209-L223 and https://github.com/bitcoinjs/bitcoinjs-lib/blob/master/src/script.js#L343-L354 and https://github.com/bitcoinjs/bitcoinjs-lib/blob/master/src/script.js#L382

btcdrak · 2016-10-05T06:48:59Z

Also need to disable uncompressed keys which are disallowed under segwit.

dcousens · 2016-10-05T08:14:02Z

Also need to disable uncompressed keys which are disallowed under segwit.

Is that in the BIP?

btcdrak · 2016-10-05T15:22:48Z

@dcousens See BIP143 https://github.com/bitcoin/bips/pull/459/files#diff-d6a595a37a5d316054e79e53e9c2382fR128

afk11 · 2016-10-07T12:19:30Z

I might find some time for this coming up, the release including segwit activation is coming up pretty close. There are some breaks in the API, probably passing the sigVersion in places, and the input amount now needs to be known during signing.

Mostly, sigVersion can be determined in situ, and by including the amount, the full TxOut is now a requirement for signing (previously it was just the scriptPubKey). To address the upcoming BC break, I'd suggest accepting a full txout in the signing API (eg, hashForSignatureV2) mostly to avoid cluttering things with extra parameters.

dcousens · 2016-10-07T12:56:24Z

and the input amount now needs to be known during signing.

That's the first I've seen of that, but again, my attention has been diverted? Any references?

@afk11 do you have a medium with which we can collaborate on that effort more closely?

afk11 · 2016-10-07T13:08:31Z

That's the first I've seen of that, but again, my attention has been diverted? Any references?

I think the 'motivation' is scenarios like a Trezor being fed the wrong UTXO amount, since it blindly signs. If we commit the amount in the sig-hash, full nodes would reject a signature where the wrong amount was used.

https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#specification

I'm starting to think for a signing API, the requirements (bar keypairs) are the TxOut, redeemScript, witnessScript, and hashType. Looking at it further, I'd love to remove the prevOutScript from addInput, and consolidate it into a txout parameter that replaces amount in TransactionBuilder.sign as seen in this PR.

Inferences

I notice for P2SH the library infers prevOutScript based on the redeemScript. It could probably do this based with segwit scripts too, although I prefer requiring the full TxOut from users always, and only optionally taking a redeemScript / witnessScript.

Core doesn't infer things, it works from the scriptPubKey and decides what it needs next. (Solver in script/standard.cpp) While checking p2sh?/witness?/bare-scriptPubKey? it determines "is it a standard type, and what sighash sigVersion do I use?". P2SH / P2WSH are treated specially, and involve checking elsewhere to see that the hashes match and another call to Solver.

Probably not a major issue, but I like there being a chain of hashes indicating which witnessScript to use (the reverse of infering the scriptPubKey) because then your wallet works based strictly on what scriptPubKey's do you have the required data for.

@afk11 do you have a medium with which we can collaborate on that effort?

Core's slack could work, otherwise happy to use email / XMPP at me$(at)$(IRL name)$(dot)io

dcousens · 2016-10-07T13:13:08Z

I'd love to remove the prevOutScript from addInput, and consolidate it into a txout parameter that replaces amount in TransactionBuilder.sign as seen in this PR.

Interestingly, I'd prefer to move more to addInput and keep sign as minimal as possible, be good to hear your opinions on why the other way.

afk11 · 2016-10-07T14:11:57Z

Ah ok :) I suppose the main reason would be to keep signing atomic - given a transaction, signing only takes so much (nIn, PrivateKeyInterface key, TransactionOutputInterface txOut, ScriptInterface redeemScript = null, ScriptInterface witnessScript = null, sigHashType = SigHashInterface::ALL)

Whether we pass all the arguments at signing, or all the arguments at addInput probably doesn't matter, but at least keep it pure, since I don't see any other method to set the prevOutScript (except for directly modifying internal values). I guess my disagreement stems from (i) it being split across two functions and, (ii) we're saving state anticipating it will only be used in that later function, so why not provide it only when necessary?

Also believe the scriptPubKey should be the thing you check a redeemScript against to test suitability when signing, rather than inferring a scriptPubKey from an arbitrary redeemScript and signing with it. I really like how Core work out what to sign with given only a scriptPubKey and it's key/script store: https://github.com/bitcoin/bitcoin/blob/master/src/script/sign.cpp#L139

< /opinion>

dcousens · 2016-10-07T15:01:39Z

As discussed on slack, we should set up an example somewhere that idiomatically shows
P2PKH, P2SH, P2WPKH and P2WSH examples in succession, with all the necessary pieces of information.
Obviously the TransactionBuilder API isn't ready for the task, but from that point we could play with the API prospectively, seeing what fits nicely and what doesn't.

dcousens · 2016-10-13T12:53:21Z

Updated milestone to 3.1.0

dcousens · 2016-11-04T01:13:17Z

Closing in favour of the newer PRs.
Captured some of the checklist with #686 and #685

dcousens assigned rubensayshi Jan 15, 2016

dcousens added the feature label Jan 15, 2016

dcousens added this to the 2.2.0 milestone Jan 15, 2016

rubensayshi force-pushed the segwit branch 6 times, most recently from d548668 to 636999b Compare January 15, 2016 16:01

greenaddress pushed a commit to greenaddress/GreenAddressWebFiles that referenced this pull request Jan 21, 2016

SegWit transactions spending support (lots of copypaste from bitcoinj…

69b7021

…s/bitcoinjs-lib#520)

greenaddress pushed a commit to greenaddress/WalletCrx that referenced this pull request Jan 21, 2016

SegWit transactions spending support (lots of copypaste from bitcoinj…

2f2f4a0

…s/bitcoinjs-lib#520)

greenaddress pushed a commit to greenaddress/WalletCordova that referenced this pull request Jan 21, 2016

SegWit transactions spending support (lots of copypaste from bitcoinj…

b270e5b

…s/bitcoinjs-lib#520)

rubensayshi force-pushed the segwit branch 9 times, most recently from ca8ae32 to 2ac407e Compare January 22, 2016 22:37

fanatid reviewed Jan 23, 2016
View reviewed changes

rubensayshi force-pushed the segwit branch from 2ac407e to c2a4f70 Compare January 23, 2016 12:53

rubensayshi force-pushed the segwit branch from c2a4f70 to 6c742f3 Compare January 24, 2016 12:00

rubensayshi reviewed Jan 24, 2016
View reviewed changes

src/types.js Outdated

Copy link

Contributor Author

rubensayshi Jan 24, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this okay to do ?

rubensayshi force-pushed the segwit branch 2 times, most recently from 5ef44ce to 9099951 Compare January 24, 2016 12:48

rubensayshi reviewed Jan 24, 2016
View reviewed changes

dcousens changed the title ~~[WIP] Segregated Witness Support~~ Segregated Witness Support - WIP Jul 11, 2016

dcousens mentioned this pull request Jul 11, 2016

SegWit - witness script templates #602

Merged

dcousens reviewed Jul 14, 2016
View reviewed changes

dcousens modified the milestones: 3.1.0, 3.0.0 Oct 13, 2016

dcousens closed this Nov 4, 2016

Segregated Witness Support - WIP #520

Segregated Witness Support - WIP #520

Uh oh!

Conversation

rubensayshi commented Jan 15, 2016 • edited by dcousens Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WORK IN PROGRESS

ToDo

Usage

References

Uh oh!

fanatid Jan 23, 2016

Choose a reason for hiding this comment

Uh oh!

rubensayshi Jan 23, 2016

Choose a reason for hiding this comment

Uh oh!

dcousens commented Jan 23, 2016

Uh oh!

rubensayshi commented Jan 24, 2016

Uh oh!

rubensayshi Jan 24, 2016

Choose a reason for hiding this comment

Uh oh!

rubensayshi Jan 24, 2016

Choose a reason for hiding this comment

Uh oh!

rubensayshi commented Jun 28, 2016

Uh oh!

btcdrak commented Jun 30, 2016

Uh oh!

dcousens commented Jun 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcousens Jul 14, 2016

Choose a reason for hiding this comment

Uh oh!

afk11 Oct 7, 2016

Choose a reason for hiding this comment

Uh oh!

dcousens commented Jul 14, 2016

Uh oh!

dcousens Jul 14, 2016

Choose a reason for hiding this comment

Uh oh!

dcousens commented Jul 14, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcousens commented Jul 14, 2016

Uh oh!

btcdrak commented Oct 4, 2016

Uh oh!

btcdrak commented Oct 4, 2016

Uh oh!

btcdrak commented Oct 4, 2016

Uh oh!

dcousens commented Oct 4, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

btcdrak commented Oct 5, 2016

Uh oh!

dcousens commented Oct 5, 2016

Uh oh!

btcdrak commented Oct 5, 2016

Uh oh!

afk11 commented Oct 7, 2016

Uh oh!

dcousens commented Oct 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

afk11 commented Oct 7, 2016

Inferences

Uh oh!

dcousens commented Oct 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

afk11 commented Oct 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dcousens commented Oct 7, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

rubensayshi commented Jan 15, 2016 •

edited by dcousens

Loading

dcousens commented Jun 30, 2016 •

edited

Loading

dcousens commented Jul 14, 2016 •

edited

Loading

dcousens commented Oct 4, 2016 •

edited

Loading

dcousens commented Oct 7, 2016 •

edited

Loading

dcousens commented Oct 7, 2016 •

edited

Loading

afk11 commented Oct 7, 2016 •

edited

Loading

dcousens commented Oct 7, 2016 •

edited

Loading