Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for descriptors to utxoupdatepsbt #15427

Merged
merged 4 commits into from Jul 2, 2019

Conversation

@sipa
Copy link
Member

@sipa sipa commented Feb 17, 2019

This adds a descriptors argument to the utxoupdatepsbt RPC. This means:

  • Input and output scripts and keys will be filled in when known.
  • P2SH-witness inputs will be filled in from the UTXO set when a descriptor is provided that shows they're spending segwit outputs.

This also moves some (newly) shared code to separate functions: UpdatePSBTOutput (an analogue to SignPSBTInput), IsSegWitOutput, and EvalDescriptorStringOrObject (implementing the string or object notation parsing used in scantxoutset).

@sipa
Copy link
Member Author

@sipa sipa commented Feb 17, 2019

@achow101 @gwillen You may be interested in this.

Loading

@sipa sipa force-pushed the 201902_utxoupdatepsbtdesc branch from ad6facf to e90a5c7 Feb 17, 2019
@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Feb 18, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #16240 (JSONRPCRequest-aware RPCHelpMan by kallewoof)
  • #16227 (Refactor CWallet's inheritance chain by achow101)
  • #15996 (rpc: Deprecate totalfee argument in bumpfee by instagibbs)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Loading

Copy link
Member

@achow101 achow101 left a comment

Conecpt ACK

Loading

src/rpc/rawtransaction.cpp Outdated Show resolved Hide resolved
Loading
Copy link
Member

@promag promag left a comment

Concept ACK, beside comment below, abstracted code looks good. Could include release note.

Loading

src/script/sign.cpp Show resolved Hide resolved
Loading
@sipa sipa force-pushed the 201902_utxoupdatepsbtdesc branch from e90a5c7 to e767de4 Feb 24, 2019
@sipa
Copy link
Member Author

@sipa sipa commented Feb 24, 2019

Rebased.

Loading

@sipa sipa force-pushed the 201902_utxoupdatepsbtdesc branch from e767de4 to cdabfae Mar 2, 2019
@sipa
Copy link
Member Author

@sipa sipa commented Mar 2, 2019

Rebased after merge of #15497.

Loading

@meshcollider
Copy link
Member

@meshcollider meshcollider commented Apr 14, 2019

Concept ACK

Loading

@laanwj laanwj added this to Blockers in High-priority for review Apr 18, 2019
Copy link
Member

@jnewbery jnewbery left a comment

utACk cdabfae

I left a bunch of comments, mostly around comments.

Loading

src/psbt.h Outdated Show resolved Hide resolved
Loading
@@ -210,6 +210,20 @@ bool PSBTInputSigned(PSBTInput& input)
return !input.final_script_sig.empty() || !input.final_script_witness.IsNull();
}

void UpdatePSBTOutput(const SigningProvider& provider, PartiallySignedTransaction& psbt, int index)
{
Copy link
Member

@jnewbery jnewbery May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is move-only, but I found this functionality pretty inscrutable, and think it could do with some additional commenting. Things that would be helpful:

  • explain why the caller is instantiating a HidingSigningProvider with nosign set to true
  • explain what the magic 0 and 1 values are in the call to ProduceSignature()
  • explain that ProduceSignature() isn't really signing (or if it does, then that signature gets thrown away)

Loading

Copy link
Member Author

@sipa sipa May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a bunch of comments. Better?

Loading

src/rpc/rawtransaction.cpp Outdated Show resolved Hide resolved
Loading
input.witness_utxo = coin.out;
}

// Update the inputs using descriptor data.
Copy link
Member

@jnewbery jnewbery May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment could be expanded to say what is being updated (redeem_script, witness_script and hd_keypaths are being filled)

Loading

SignPSBTInput(HidingSigningProvider(&provider, /* nosign */ true, /* nobip32derivs */ false), psbtx, i, /* sighash_type */ 1);
}

// Update the outputs
Copy link
Member

@jnewbery jnewbery May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment could be expanded to say what is being updated (redeem_script, witness_script and hd_keypaths are being filled)

Loading

Copy link
Member Author

@sipa sipa May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather not be specific here, as it may change over time.

Loading

input.witness_utxo = coin.out;
}

// Update the inputs using descriptor data.
SignPSBTInput(HidingSigningProvider(&provider, /* nosign */ true, /* nobip32derivs */ false), psbtx, i, /* sighash_type */ 1);
Copy link
Member

@jnewbery jnewbery May 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found using SignPSBTInput() a bit jarring here, since nothing is actually being signed. Can you add a comment to say that SignPSBTInput() won't actually sign when provided with a HidingSigningProvider where nosign is true?

Loading

Copy link
Member Author

@sipa sipa May 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some comments.

Loading

src/rpc/rawtransaction.cpp Outdated Show resolved Hide resolved
Loading
test/functional/rpc_psbt.py Outdated Show resolved Hide resolved
Loading
@sipa sipa force-pushed the 201902_utxoupdatepsbtdesc branch from cdabfae to 535b649 May 9, 2019
src/rpc/rawtransaction.cpp Outdated Show resolved Hide resolved
Loading
sipa added 4 commits May 10, 2019
This is not a pure refactor; additional functionality is added in
IsSegWitOutput which lets it recurse into P2SH when a
SigningProvider is provided that knows about the inner script.
This adds a descriptors argument to the utxoupdatepsbt RPC. This means:
* Input and output scripts and keys will be filled in when known
* P2SH-witness outputs will be filled in from the UTXO set when a descriptor
  is provided to show they're segwit outputs.
@sipa sipa force-pushed the 201902_utxoupdatepsbtdesc branch from 535b649 to 26fe9b9 May 10, 2019
@sipa
Copy link
Member Author

@sipa sipa commented May 10, 2019

Rebased.

Loading

@jnewbery
Copy link
Member

@jnewbery jnewbery commented May 10, 2019

utACK 26fe9b9

Loading

Copy link
Member

@promag promag left a comment

A couple of comments.

Loading

// Parse descriptors, if any.
FlatSigningProvider provider;
if (!request.params[1].isNull()) {
auto descs = request.params[1].get_array();
Copy link
Member

@promag promag May 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use range loop

for (const UniValue& scanobject : request.params[1].get_array().getValues()) {

Like here.

Loading

MutableTransactionSignatureCreator creator(psbtx.tx.get_ptr(), 0, out.nValue, 1);
ProduceSignature(HidingSigningProvider(pwallet, true, !bip32derivs), creator, out.scriptPubKey, sigdata);
psbt_out.FromSignatureData(sigdata);
UpdatePSBTOutput(HidingSigningProvider(pwallet, true, !bip32derivs), psbtx, i);
Copy link
Member

@promag promag May 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to instantiate HidingSigningProvider before the loop?

Loading

test_psbt_input_keys(decoded['inputs'][2], [])

# Try again, now while providing descriptors, making P2SH-segwit work, and causing bip32_derivs and redeem_script to be filled in
descs = [self.nodes[1].getaddressinfo(addr)['desc'] for addr in [addr1,addr2,addr3]]
Copy link
Member

@promag promag May 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, spaces after , - [addr1, addr2, addr3].

Loading

@@ -215,6 +215,25 @@ bool PSBTInputSigned(const PSBTInput& input)
return !input.final_script_sig.empty() || !input.final_script_witness.IsNull();
}

void UpdatePSBTOutput(const SigningProvider& provider, PartiallySignedTransaction& psbt, int index)
Copy link
Member

@promag promag May 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could name this UpdatePSBT() and iterate outputs here? - which is what both callers do.

Loading

Copy link
Member

@Empact Empact May 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, seems this could be appropriate as PartiallySignedTransaction::UpdateOutput

Loading

@laanwj
Copy link
Member

@laanwj laanwj commented May 20, 2019

utACK 26fe9b9 (will hold merging until response to promag's comments)

Loading

Copy link
Member

@ariard ariard left a comment

@sipa I'm trying the RPC with the following command:

bitcoin-cli utxoupdatepsbt 'cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA=='  '[{"desc":"sh(wpkh([bd50871a/0h/0h/0h]03895c66337b38699bfafff1084ad35bc347fac4f4e5e5fe5eb7dd81155280db53))"}]'

With psbt built from createpsbt and a desc from listunspent but got a RPC_TYPE_ERROR Expected type array, got string. Is there somehting really basic I'm missing or a real parsing issue ? (given than descriptors hasn't been added as argNames)

Loading

@@ -232,4 +232,7 @@ void UpdateInput(CTxIn& input, const SignatureData& data);
* Solvability is unrelated to whether we consider this output to be ours. */
bool IsSolvable(const SigningProvider& provider, const CScript& script);

/** Check whether a scriptPubKey is known to be segwit. */
Copy link
Member

@ariard ariard May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "native segwit or nested-segwit"?

Loading

{"desc", RPCArg::Type::STR, RPCArg::Optional::NO, "An output descriptor"},
{"range", RPCArg::Type::RANGE, "1000", "Up to what index HD chains should be explored (either end or [begin,end])"},
}},
}},
},
RPCResult {
" \"psbt\" (string) The base64-encoded partially signed transaction with inputs updated\n"
Copy link
Member

@ariard ariard May 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "with inputs and outputs updated"

Loading

@@ -1490,12 +1490,19 @@ UniValue converttopsbt(const JSONRPCRequest& request)

UniValue utxoupdatepsbt(const JSONRPCRequest& request)
{
if (request.fHelp || request.params.size() != 1) {
if (request.fHelp || request.params.size() < 1 || request.params.size() > 2) {
Copy link
Member

@promag promag Jun 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

26fe9b9

nit, could use the "new" way:

    if (request.fHelp || !help.IsValidNumArgs(request.params.size())) {
        throw std::runtime_error(help.ToString());
    }

Loading

@moneyball
Copy link
Contributor

@moneyball moneyball commented Jun 6, 2019

It seems like next steps for this PR are for @sipa to respond to @promag and @ariard comments, then probably back to @laanwj for merge consideration.

Loading

@promag
Copy link
Member

@promag promag commented Jun 11, 2019

My comments aren't blocking FWIW.

Loading

@promag
Copy link
Member

@promag promag commented Jun 12, 2019

@sipa my suggestions are in fb3540a, and rebased here d7a8cf2.

Loading

@promag
Copy link
Member

@promag promag commented Jun 12, 2019

ACK 26fe9b9, checked refactors and tests look comprehensive. Still missing a release note but can be added later.

Loading

@laanwj laanwj merged commit 26fe9b9 into bitcoin:master Jul 2, 2019
2 checks passed
Loading
@laanwj
Copy link
Member

@laanwj laanwj commented Jul 2, 2019

Going to merge this, I think the comments are minor enough to be addressable afterward and don't really need to block the merge.

Loading

laanwj added a commit that referenced this issue Jul 2, 2019
26fe9b9 Add support for descriptors to utxoupdatepsbt (Pieter Wuille)
3135c1a Abstract out UpdatePSBTOutput from FillPSBT (Pieter Wuille)
fb90ec3 Abstract out EvalDescriptorStringOrObject from scantxoutset (Pieter Wuille)
eaf4f88 Abstract out IsSegWitOutput from utxoupdatepsbt (Pieter Wuille)

Pull request description:

  This adds a descriptors argument to the `utxoupdatepsbt` RPC. This means:
  * Input and output scripts and keys will be filled in when known.
  * P2SH-witness inputs will be filled in from the UTXO set when a descriptor is provided that shows they're spending segwit outputs.

  This also moves some (newly) shared code to separate functions: `UpdatePSBTOutput` (an analogue to `SignPSBTInput`), `IsSegWitOutput`, and `EvalDescriptorStringOrObject` (implementing the string or object notation parsing used in `scantxoutset`).

ACKs for top commit:
  jnewbery:
    utACK 26fe9b9
  laanwj:
    utACK 26fe9b9 (will hold merging until response to promag's comments)
  promag:
    ACK 26fe9b9, checked refactors and tests look comprehensive. Still missing a release note but can be added later.

Tree-SHA512: 1d833b7351b59d6c5ded6da399ff371a8a2a6ad04c0a8f90e6e46105dc737fa6f2740b1e5340280d59e01f42896c40b720c042f44417e38dfbee6477b894b245
@jnewbery
Copy link
Member

@jnewbery jnewbery commented Jul 2, 2019

It's a shame that this got merged without a bunch of review comments being addressed. It seems to me that this: #15427 (review) is a bug, and that the new functionality won't work with bitcoin-cli or using named arguments.

Loading

@jnewbery
Copy link
Member

@jnewbery jnewbery commented Jul 2, 2019

Fixed here: #16326

Loading

@MarcoFalke
Copy link
Member

@MarcoFalke MarcoFalke commented Jul 2, 2019

This had 3 ACKs. master is always expected to have bugs and if they are trivially fixed in a follow-up, why not?

Loading

sidhujag added a commit to syscoin/syscoin that referenced this issue Jul 3, 2019
26fe9b9 Add support for descriptors to utxoupdatepsbt (Pieter Wuille)
3135c1a Abstract out UpdatePSBTOutput from FillPSBT (Pieter Wuille)
fb90ec3 Abstract out EvalDescriptorStringOrObject from scantxoutset (Pieter Wuille)
eaf4f88 Abstract out IsSegWitOutput from utxoupdatepsbt (Pieter Wuille)

Pull request description:

  This adds a descriptors argument to the `utxoupdatepsbt` RPC. This means:
  * Input and output scripts and keys will be filled in when known.
  * P2SH-witness inputs will be filled in from the UTXO set when a descriptor is provided that shows they're spending segwit outputs.

  This also moves some (newly) shared code to separate functions: `UpdatePSBTOutput` (an analogue to `SignPSBTInput`), `IsSegWitOutput`, and `EvalDescriptorStringOrObject` (implementing the string or object notation parsing used in `scantxoutset`).

ACKs for top commit:
  jnewbery:
    utACK 26fe9b9
  laanwj:
    utACK 26fe9b9 (will hold merging until response to promag's comments)
  promag:
    ACK 26fe9b9, checked refactors and tests look comprehensive. Still missing a release note but can be added later.

Tree-SHA512: 1d833b7351b59d6c5ded6da399ff371a8a2a6ad04c0a8f90e6e46105dc737fa6f2740b1e5340280d59e01f42896c40b720c042f44417e38dfbee6477b894b245
@laanwj laanwj removed this from Blockers in High-priority for review Jul 3, 2019
@fanquake
Copy link
Member

@fanquake fanquake commented Jul 5, 2019

Release notes for this are being added as part of #16326.

Loading

@jnewbery
Copy link
Member

@jnewbery jnewbery commented Jul 5, 2019

This had 3 ACKs. master is always expected to have bugs and if they are trivially fixed in a follow-up, why not?

Because comments should at least be acknowledged by the author, even if that's just to say "This can be addressed in a follow-up PR".

Loading

vansergen added a commit to vansergen/rpc-bitcoin that referenced this issue Mar 26, 2020
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue Jun 23, 2020
…cantxoutset

Summary:
bitcoin/bitcoin@fb90ec3

---

Partial backport of Core [[bitcoin/bitcoin#15427 | PR15427]]

Test Plan:
  ninja check-all

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Subscribers: Fabien

Differential Revision: https://reviews.bitcoinabc.org/D6616
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue Jun 23, 2020
Summary:
bitcoin/bitcoin@3135c1a

---

Depends on D6616

Partial backport of Core [[bitcoin/bitcoin#15427 | PR15427]]

Test Plan:
  ninja check

Reviewers: #bitcoin_abc, Fabien

Reviewed By: #bitcoin_abc, Fabien

Subscribers: Fabien

Differential Revision: https://reviews.bitcoinabc.org/D6624
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue Jun 23, 2020
Summary:
This adds a descriptors argument to the utxoupdatepsbt RPC. This means:
* Input and output scripts and keys will be filled in when known
* P2SH-witness outputs will be filled in from the UTXO set when a descriptor
  is provided to show they're segwit outputs.

bitcoin/bitcoin@26fe9b9

---

Depends on D6624

Concludes backport of Core [[bitcoin/bitcoin#15427 | PR15427]]

Test Plan:
  ninja check-all

Reviewers: #bitcoin_abc, Fabien, deadalnix

Reviewed By: #bitcoin_abc, Fabien, deadalnix

Subscribers: deadalnix, Fabien

Differential Revision: https://reviews.bitcoinabc.org/D6626
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet