Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Additional utility RPCs for PSBT #13932

Merged
merged 5 commits into from Feb 16, 2019
Merged

Additional utility RPCs for PSBT #13932

merged 5 commits into from Feb 16, 2019

Conversation

@achow101
Copy link
Member

@achow101 achow101 commented Aug 9, 2018

This PR adds 3 new utility RPCs for interacting with PSBTs.

utxoupdatepsbt updates a PSBT with UTXO information from the node. It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

joinpsbts joins the inputs from multiple distinct PSBTs into one PSBT. e.g. if PSBT 1 has inputs 1 and 2, and PSBT 2 has inputs 3 and 4, joinpsbts would create a new PSBT with inputs 1, 2, 3, and 4.

analyzepsbt analyzes a PSBT and determines the current state of it and all of its inputs, and the next step that needs to be done.

@DrahtBot
Copy link
Contributor

@DrahtBot DrahtBot commented Aug 9, 2018

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #15404 ([test] Remove -txindex to start nodes by amitiuttarwar)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Loading

@gmaxwell
Copy link
Contributor

@gmaxwell gmaxwell commented Aug 10, 2018

It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

It doesn't look in the wallet?

Loading

@sipa
Copy link
Member

@sipa sipa commented Aug 10, 2018

@gmaxwell walletprocesspsbt already exists for that. This is a node RPC that works without a wallet.

Loading

@achow101
Copy link
Member Author

@achow101 achow101 commented Aug 14, 2018

Rebased

Loading

@achow101
Copy link
Member Author

@achow101 achow101 commented Aug 28, 2018

Rebased

Loading

@achow101 achow101 force-pushed the psbt-util-rpcs branch 2 times, most recently from eeafff2 to f29d03c Aug 28, 2018
{
if (request.fHelp || request.params.size() != 1)
throw std::runtime_error(
"utxoupdatepsbt \"psbt\"\n"
Copy link
Member

@promag promag Aug 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could reduce indentation?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

for (auto it = std::next(psbtxs.begin()); it != psbtxs.end(); ++it) {
for (unsigned int i = 0; i < it->tx->vin.size(); ++i) {
if (!merged_psbt.AddInput(it->tx->vin[i], it->inputs[i])) {
throw JSONRPCError(RPC_INVALID_PARAMETER, "An input exists in multiple PSBTs");
Copy link
Member

@promag promag Aug 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be more informative?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What else could be said?

Loading

Copy link
Member

@promag promag Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like the conflicting input index?

Loading

}
}
for (unsigned int i = 0; i < it->tx->vout.size(); ++i) {
merged_psbt.AddOutput(it->tx->vout[i], it->outputs[i]);
Copy link
Member

@promag promag Aug 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should allow duplicate outputs? Or should sum values into one output?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate outputs should be allowed. The idea is that there are two distinct transactions with separate inputs and outputs. They are just being combined into one transaction. Thus you can have duplicate outputs as outputs are still unique. However the inputs must be enforced to be unique.

Loading

Copy link
Member

@promag promag Nov 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@achow101 could you ack/nack on #12419, esp @meshcollider #12419 (comment)

Loading


// Clear signatures from all inputs
for (auto& input : merged_psbt.inputs) {
input.partial_sigs.clear();
Copy link
Member

@promag promag Aug 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is necessary because the first is a copy and the remaining psbt inputs are cleared because of AddInput. Maybe remove this "optimization" and merge all psbts to an empty psbt?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how that is better.

Loading

Copy link
Contributor

@gwillen gwillen Nov 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be less code, and less complexity, which I would say is always better, absent a reason to write more code. Why write more code?

Loading

Copy link
Contributor

@gwillen gwillen Dec 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Also, this is copied from combinepsbt, where it is easiest to treat the first tx specially, because we need something to compare all the others to, to make sure the underlying transaction is the same. That restriction doesn't exist here, so the need for the extra step is gone.)

Loading

// Unserialize the transactions
std::vector<PartiallySignedTransaction> psbtxs;
UniValue txs = request.params[0].get_array();
for (unsigned int i = 0; i < txs.size(); ++i) {
Copy link
Member

@promag promag Aug 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should force txs.size() > 1?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

CAmount fee = in_amt - out_amt;

// Estimate the size
size_t size;
Copy link
Contributor

@practicalswift practicalswift Sep 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scope of size could be reduced?

Loading

Copy link
Member Author

@achow101 achow101 Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

@achow101
Copy link
Member Author

@achow101 achow101 commented Sep 13, 2018

Rebased

Loading

@@ -7,6 +7,7 @@

from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import assert_equal, assert_raises_rpc_error, find_output
from decimal import Decimal
Copy link
Contributor

@practicalswift practicalswift Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Sort imports :-)

Loading

@laanwj
Copy link
Member

@laanwj laanwj commented Feb 15, 2019

very strange!-one of the travis runs seems to be failing with

/bin/bash: test/test_bitcoin: No such file or directory
Makefile:12966: recipe for target 'test/addrman_tests.cpp.test' failed
make[3]: *** [test/addrman_tests.cpp.test] Error 1
make[3]: *** Waiting for unfinished jobs....
/bin/bash: test/test_bitcoin: No such file or directory
Makefile:12966: recipe for target 'test/arith_uint256_tests.cpp.test' failed
make[3]: *** [test/arith_uint256_tests.cpp.test] Error 1
PASS: qt/test/test_bitcoin-qt

Loading

@MarcoFalke MarcoFalke closed this Feb 15, 2019
@MarcoFalke MarcoFalke reopened this Feb 15, 2019
@laanwj
Copy link
Member

@laanwj laanwj commented Feb 15, 2019

utACK 0c35f0e

Loading

@@ -1773,6 +1773,67 @@ UniValue converttopsbt(const JSONRPCRequest& request)
return EncodeBase64((unsigned char*)ssTx.data(), ssTx.size());
}

UniValue utxoupdatepsbt(const JSONRPCRequest& request)
{
if (request.fHelp || request.params.size() != 1)
Copy link
Member

@sipa sipa Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically this violates the style guide, as it's not putting the then clause on the same line as the if, so it needs braces + indenting on the next line.

Loading


std::vector<std::vector<unsigned char>> solutions_data;
txnouttype which_type = Solver(coin.out.scriptPubKey, solutions_data);
if (which_type == TX_WITNESS_V0_SCRIPTHASH || which_type == TX_WITNESS_V0_KEYHASH || which_type == TX_WITNESS_UNKNOWN) {
Copy link
Member

@sipa sipa Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess there's an obvious solution which can be added later, namely letting the RPC take in descriptor(s) that apply to its inputs. I'll look into that after this PR is merged.

Loading

}

// Check if it is final
if (input.final_script_sig.empty() && input.final_script_witness.IsNull()) {
Copy link
Member

@sipa sipa Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use PSBTInputSigned here?

Loading

Copy link
Member

@sipa sipa Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may want to add a check here for whether the input UTXO is present (if not, it's possible that you first set next="updater", and then overwrite it in this conditional).

Loading

Copy link
Member Author

@achow101 achow101 Feb 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading


// If we are only missing signatures and nothing else, then next is signer
if (outdata.missing_pubkeys.empty() && outdata.missing_redeem_script.IsNull() && outdata.missing_witness_script.IsNull() && !outdata.missing_sigs.empty()) {
only_missing_sigs = true;
Copy link
Member

@sipa sipa Feb 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this (cross-input) variable can't be set to true here. If this branch is executed for the last input in a transaction, the global "next" will report "signer", even if keys/script or even UTXOs are missing for other inputs.

You probably want to have the variable start at true, and then set it to false if anything anywhere more than a signature is missing.

Loading

Copy link
Member Author

@achow101 achow101 Feb 16, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

achow101 added 4 commits Feb 16, 2019
Adds a joinpsbts RPC which combines multiple distinct PSBTs into
one PSBT.
When signing an input, figure out what was requested for but was unable
to be found and store it in a SignatureData.

Return this information in SignPSBTInput.
@achow101 achow101 force-pushed the psbt-util-rpcs branch 2 times, most recently from 310b395 to 43bbc2c Feb 16, 2019
@laanwj
Copy link
Member

@laanwj laanwj commented Feb 16, 2019

re-utACK 540729e

Loading

@laanwj laanwj merged commit 540729e into bitcoin:master Feb 16, 2019
2 checks passed
Loading
laanwj added a commit that referenced this issue Feb 16, 2019
540729e Implement analyzepsbt RPC and tests (Andrew Chow)
77542cf Move PSBT UTXO fetching to a separate method (Andrew Chow)
cb40b3a Figure out what is missing during signing (Andrew Chow)
08f749c Implement joinpsbts RPC and tests (Andrew Chow)
7344a7b Implement utxoupdatepsbt RPC and tests (Andrew Chow)

Pull request description:

  This PR adds 3 new utility RPCs for interacting with PSBTs.

  `utxoupdatepsbt` updates a PSBT with UTXO information from the node. It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

  `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. e.g. if PSBT 1 has inputs 1 and 2, and PSBT 2 has inputs 3 and 4, `joinpsbts` would create a new PSBT with inputs 1, 2, 3, and 4.

  `analyzepsbt` analyzes a PSBT and determines the current state of it and all of its inputs, and the next step that needs to be done.

Tree-SHA512: 3c1fa302201abca76a8901d0c2be7b4ccbce334d989533c215f8b3e50e22f2f018ce6209544b26789f58f5980a253c0655111e1e20d47d5656e0414c64891a5c
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue May 13, 2020
Summary:
bitcoin/bitcoin@7344a7b

---

This is a partial backport of Core [[bitcoin/bitcoin#13932 | PR13932]]

Test Plan:
  ninja check
  test_runner.py rpc_psbt

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Subscribers: deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D6043
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue May 13, 2020
Summary:
Adds a joinpsbts RPC which combines multiple distinct PSBTs into
one PSBT.

bitcoin/bitcoin@08f749c

---

Depends on D6043

This is a partial backport of Core [[bitcoin/bitcoin#13932 | PR13932]] plus a test case

Test Plan:
  ninja check
  test_runner.py rpc_psbt

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D6044
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue May 13, 2020
Summary:
bitcoin/bitcoin@77542cf

---

Depends on D6044

This is a partial backport of Core [[bitcoin/bitcoin#13932 | PR13932]]

Test Plan: unfortunately no tests are included with this commit

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D6045
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue May 13, 2020
Summary:
When signing an input, figure out what was requested for but was unable
to be found and store it in a SignatureData.

Return this information in SignPSBTInput.

bitcoin/bitcoin@cb40b3a

---

Depends on D6045

This is a partial backport of Core [[bitcoin/bitcoin#13932 | PR13932]]

Test Plan:
  ninja check check-functional

Reviewers: #bitcoin_abc, deadalnix, nakihito

Reviewed By: #bitcoin_abc, deadalnix, nakihito

Subscribers: nakihito, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D6046
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this issue May 16, 2020
Summary:
bitcoin/bitcoin@540729e

---

This completes the backport of Core [[bitcoin/bitcoin#13932 | PR13932]]

Test Plan:
  cmake .. -GNinja -DENABLE_WERROR=ON
  ninja check-all

Reviewers: #bitcoin_abc, nakihito, deadalnix

Reviewed By: #bitcoin_abc, nakihito, deadalnix

Subscribers: nakihito

Differential Revision: https://reviews.bitcoinabc.org/D6058
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 5, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Aug 9, 2021
5tefan added a commit to 5tefan/dash that referenced this issue Aug 12, 2021
kittywhiskers added a commit to kittywhiskers/dash that referenced this issue Dec 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

10 participants