API to find funding utxos in psbt #853

violet360 · 2022-02-28T14:07:56Z

Current status

The API returns a vector of UTXOs and has return type Result<Vec<&TxOut>, Error>

Expected

The return statement should be of type sighash::Prevouts as pointed in #849

Eunoia1729 · 2022-02-28T22:54:22Z

Nit: Next time you may avoid adding/removing blank lines.

tcharding

High level review only, thanks!

src/util/psbt/mod.rs

Kixunil

Thanks, the code looks almost good but we really have to fix two more issues I found.

There's another thing: should we use a dedicated error type? I tend towards yes but it can be done later as well. I suggest @violet360 if it's too overwhelming to you to add another error type leave it for later, if you feel like adding it you can do it within this PR.

FTR, this PR will most likely not go to the Taproot release anyway.

src/util/psbt/error.rs

Kixunil · 2022-03-03T11:33:52Z

src/util/psbt/mod.rs

@@ -72,6 +72,24 @@ pub struct PartiallySignedTransaction {
 }

 impl PartiallySignedTransaction {
+    /// Returns an iterator for the funding UTXOs of the psbt
+    /// function throws error if transaction inputs have unequal lengths


It doesn't "throw" and error though. It panics. In Rust the usual style of documenting panics is like this:

/// Returns an iterator for the funding UTXOs of the psbt /// /// For each PSBT input that contains UTXO information `Ok` is returned containing that information. /// The order of returned items is same as the order of inputs. /// /// ## Errors /// /// The function returns error when UTXO information is not present or is invalid. /// /// ## Panics /// /// The function panics if the length of transaction inputs is not equal to the length of PSBT inputs.

(See below for "or invalid" case.)

Kixunil · 2022-03-03T11:40:58Z

src/util/psbt/mod.rs

+    pub fn get_funding_utxos(&self) -> impl Iterator<Item = Result<&TxOut, Error>> {
+        assert_eq!(self.inputs.len(), self.unsigned_tx.input.len());
+        self.unsigned_tx.input.iter().zip(&self.inputs).map(|(tx_input, psbt_input)| {
+            if let Some(ref witness_utxo) = psbt_input.witness_utxo {


Note: you can avoid ref by borrowing: if let Some(witness_utxo) = &psbt_input.witness_utxo
I personally prefer this style and I think it's used in this library but not a big deal. ref has it's own appeal too.

Also perhaps using match on the whole thing would be clearer:

match (&psbt_input.witness_utxo, &psbt_input.non_witness_utxo) { (Some(witness_utxo), _) => Ok(witness_utxo), (None, Some(non_witness_utxo) => { ... }, (None, None) => Err(Error::MissingUtxo), }

Thanks, the code looks almost good but we really have to fix two more issues I found.

There's another thing: should we use a dedicated error type? I tend towards yes but it can be done later as well. I suggest @violet360 if it's too overwhelming to you to add another error type leave it for later, if you feel like adding it you can do it within this PR.

FTR, this PR will most likely not go to the Taproot release anyway.

yes I too was thinking about generating a seperate error type for psbts but thought it would be an overkill, will try committing that in this PR itself.....

Note that it still makes sense to also have the variants in Error and From<PsbtUtxoError> for Error

src/util/psbt/mod.rs

Kixunil

The code is acceptable now. Just some minor details.

Because it's a new non-essential feature this should be merged after 0.28 release.

Kixunil · 2022-03-03T16:53:33Z

src/util/psbt/mod.rs

@@ -171,6 +197,7 @@ mod display_from_str {
 #[cfg_attr(docsrs, doc(cfg(feature = "base64")))]
 pub use self::display_from_str::PsbtParseError;

+


This empty line seems unrelated. It'd be nice to remove it during next push.

Kixunil · 2022-03-03T16:55:01Z

src/util/psbt/error.rs

    /// The separator for a PSBT must be `0xff`.
    InvalidSeparator,
+    /// Triggered when output index goes out of bounds of when trying to get 


This doesn't seem grammatically correct but as a non-native I'll defer this to @tcharding

Kixunil

Looks great, you just need to squash the commits into one.

Kixunil · 2022-03-03T23:06:24Z

src/util/psbt/error.rs

@@ -98,6 +102,8 @@ impl fmt::Display for Error {
            }
            Error::NoMorePairs => f.write_str("no more key-value pairs for this psbt map"),
            Error::HashParseError(e) => write!(f, "Hash Parse Error: {}", e),
+            Error::MissingUtxo => f.write_str("UTXO information is not present in PSBT"),
+            Error::PsbtUtxoOutOfbounds => f.write_str("Output index is out of bounds of non witness script output array"),


Oh, forgot to comment before that "Output index should probably be "output index to be consistent with other messages. Sorry.

Hash Parse Error: is wrong too, a patch put at the front of this PR cleaning that all up would be super useful if you can be bothered. Don't feel obliged though. You will likely touch this line when the error comment is changed anyways.

tcharding

Just a few comments, thanks.

tcharding · 2022-03-04T14:34:58Z

src/util/psbt/error.rs

@@ -37,8 +37,12 @@ pub enum Error {
    /// Magic bytes for a PSBT must be the ASCII for "psbt" serialized in most
    /// significant byte order.
    InvalidMagic,
+    ///Missing both the witness and non-witness utxo


Suggested change

///Missing both the witness and non-witness utxo

/// Missing both the witness and non-witness utxo.

tcharding · 2022-03-04T14:43:39Z

src/util/psbt/error.rs

    /// The separator for a PSBT must be `0xff`.
    InvalidSeparator,
+    /// Triggered when output index goes out of bounds of the output in non witness utxo


Suggested change

/// Triggered when output index goes out of bounds of the output in non witness utxo

/// Triggered when output index does not exist in PSBT input (non-witness) UTXO.

Not sure if the content of this comment is correct but it shows a grammatical improvement, please modify as you see fit.

While strictly speaking correct, thinking of it in terms of "out of bounds" feels more natural to me. Won't block if others prefer your suggestion.

Fair comment, this is a tricky little rustdoc to write succinctly.

out of bound seemed natural to me too, but whatever you guys prefer at the end though.

How's this?

/// Triggered when output index is out of bounds in relation to the output in non-witness UTXO.

Sounds good. But should it be "triggered"? Maybe "returned is better"?

In case of error, words like triggered, panic, etc. Sound more suitable than "returned".
Maybe "Triggered" would be more suitable here ?

My mental model in Rust is that errors are returned (well, they actually are :)). But honestly, I don't care much if others find "triggered" better.

FWIW we have 'triggered' in one other place in the repo

git grep trig src/blockdata/script.rs:1279: // which triggerred overflow bugs on 32-bit machines in script formatting in the past.

I'm not fussed either way.

tcharding · 2022-03-04T14:46:32Z

src/util/psbt/error.rs

@@ -98,6 +102,8 @@ impl fmt::Display for Error {
            }
            Error::NoMorePairs => f.write_str("no more key-value pairs for this psbt map"),
            Error::HashParseError(e) => write!(f, "Hash Parse Error: {}", e),
+            Error::MissingUtxo => f.write_str("UTXO information is not present in PSBT"),
+            Error::PsbtUtxoOutOfbounds => f.write_str("Output index is out of bounds of non witness script output array"),


Hash Parse Error: is wrong too, a patch put at the front of this PR cleaning that all up would be super useful if you can be bothered. Don't feel obliged though. You will likely touch this line when the error comment is changed anyways.

tcharding · 2022-03-04T14:48:42Z

src/util/psbt/mod.rs


 use prelude::*;

 use io;
-


Please no 'random' whitespace changes. The reason is that it makes reviewing the PR harder because reviewers must stop to think what is this, why is it being done, is it correct, is it a step backwards in readability. All easy questions but unnecessary for getting your work merged. The easier you make it for reviewers to ack your work the easier you'll get PR's merged, that's a win for everyone :)

Yes, silly mistake of mine ... will take care of it from next commits.

Cheers mate, no stress. If I had a dollar for ever mistake I made in this repo in the last three months I'd be fiat rich :)

dr-orlovsky

Agree with other comments, just adding my 5 cents

dr-orlovsky · 2022-03-12T07:17:01Z

src/util/psbt/mod.rs

+    ///
+    /// The function panics if the length of transaction inputs is not equal to the length of PSBT inputs.
+    pub fn get_funding_utxos(&self) -> impl Iterator<Item = Result<&TxOut, Error>> {
+        assert_eq!(self.inputs.len(), self.unsigned_tx.input.len());


Suggested change

assert_eq!(self.inputs.len(), self.unsigned_tx.input.len());

assert_debug_eq!(self.inputs.len(), self.unsigned_tx.input.len());

Hi !
you meant debug_assert_eq! instead of assert_debug_eq! right ?

Yes, sorry.

sanket1729

Sorry for the delayed review. This is a great shape after several iterations and thanks for addressing all the feedback @violet360 .

Everything is good from my side, just the final API naming.

sanket1729 · 2022-03-13T16:47:08Z

src/util/psbt/mod.rs

+    /// ## Panics
+    ///
+    /// The function panics if the length of transaction inputs is not equal to the length of PSBT inputs.
+    pub fn get_funding_utxos(&self) -> impl Iterator<Item = Result<&TxOut, Error>> {


On a similar theme for #861, I think we rename this to iter_funding_utxos?

yes, consider it done 👍🏻

Kixunil · 2022-03-14T20:32:21Z

@dr-orlovsky I disagree with it being debug assert. If the two have different length it's a clear inconsistency of input data and there's no good solution. Debug assert will just allow silent weird bugs in release.

Regarding naming, iter_ is acceptable but unnecessary. Neither str::bytes() nor str::chars() have iter_ prefix and it's fine. Agree with dropping get_ prefix. funding_utxos looks idiomatic and reads well to me.

violet360 · 2022-03-14T21:51:01Z

I also want to improve the error management of psbt, I think there is a lot of room for that, planning to get that done in this PR itself, or maybe a new PR.
Although my suggestions can be wrong but point me in the right direction.

Should I create a separate enum psbt and add PsbtHash as the child enum, and add psbt as a child enum of Error, kinda like :

pub enum psbt {
...
  PsbtHash, // psbthash child enum of psbt
/// Missing both the witness and non-witness utxo.
  MissingUtxo,
/// Returned when output index is out of bounds in relation to the output in non-witness UTXO.
  PsbtUtxoOutOfbounds,
..
}
...
pub enum Error {
...
  psbt, 
...
}

seems like use psbt::raw <--> psbt::Error and consensus::encode<--> psbt::Error have a circular dependency, do we need to address that?

sanket1729

ACK 9f523e0. If there are further iterations, feel free to request review from me.

@dr-orlovsky I disagree with it being debug assert. If the two have different length it's a clear inconsistency of input data and there's no good solution. Debug assert will just allow silent weird bugs in release.

Indeed, asserts should be assert! instead of debug_assert! unless it benchmarked that it affects performance. I had a similar issue in taproot review.
#677 (comment)

Perhaps, that should also be updated to assert! and we can inspect the codebase for misc debug_assert! and change those to assert!

sanket1729 · 2022-03-14T22:12:03Z

I also want to improve the error management of psbt, I think there is a lot of room for that, planning to get that done in this PR itself, or maybe a new PR.

That should be a separate PR.

@violet360, we should take this discussion to #837

dr-orlovsky

utACK 9f523e0

dr-orlovsky · 2022-03-18T11:10:05Z

@sanket1729 sorry, IDK why but your comment does not displays to me.

@Kixunil
My understanding is that whenever there is internal error in the logic of the code, it should be debug_assert*. assert*s are for tests, and if there might be a error in the input data, than it should be a Result with error type returned.

Here, if the supplied PSBT (as an input) have different number of inputs in tx and map, it must error at the moment it was unparsed into PartiallySignedTransaction structure. And if it was not, it is an internal program error.

sanket1729 · 2022-03-18T17:01:01Z

@dr-orlovsky See: https://doc.rust-lang.org/std/macro.debug_assert.html

Quote from the docs:

Replacing assert! with debug_assert! is thus only encouraged after thorough profiling, and more importantly, only in safe code!

dr-orlovsky · 2022-03-30T16:33:16Z

@Kixunil seems like all your comments were addressed. Can you confirm so we can merge this one?

Kixunil · 2022-04-07T14:01:24Z

I view the logic bug similar to indexing out of bounds or calling copy_from_slice on non-equal-length slices. These always panic. If the type guaranteed consistency then I'd be in favor of debug_assert!. The code currently can not guarantee consistency because the fields are public. You can construct inconsistent PSBT by just forgetting to push stuff. We could also return Err instead of panicking but this kind of error can only be caused by buggy code, not input data.

You could argue this is not memory-safety issue but I view bugs in bitcoin-related code as seriously as memory safety. People could lose sats.

V2 API will remove this by construction but until then assert_eq! should be the most idiomatic approach.

violet360 · 2022-04-17T09:13:36Z

sorry for the late comment...was busy with some other stuff lately
it seems there is a disagreement at the panic statement, altho I too lean towards !assert_eq but not by experience but by the docs as pointed out by @sanket1729 .
can we conclude?

tcharding · 2022-04-17T23:44:01Z

it seems there is a disagreement at the panic statement

FWIW, from my reading of this thread there is consensus on assert_eq!.

Kixunil

ACK 5afb0ea

tcharding

ACK 5afb0ea

sanket1729

ACK 5afb0ea. Thanks for being patient with this.

DanGould · 2022-07-20T11:13:49Z

This interface does not return complete information about the funding utxo. it only returns psbt v0 input map info: TxOut which has script_pubkey and value. Unfortunately, txid, vout, sequence number, and locktime are still only the global map's unsigned_tx. So if for example you want to check some psbt set of input utxos against those in your wallet, as is a standard receiver payjoin check, you need data beyond what this function returns.

It would be nice in the PSBTv2 implementation to return an Input map element with that complete set of data rather than just a TxOut { script_pubkey, locktime}

5afb0ea API to get an iterator for funding utxos in psbt (violet360) Pull request description: ### Current status The API returns a vector of UTXOs and has return type `Result<Vec<&TxOut>, Error>` ### Expected The return statement should be of type `sighash::Prevouts` as pointed in #849 ACKs for top commit: Kixunil: ACK 5afb0ea tcharding: ACK 5afb0ea sanket1729: ACK 5afb0ea. Thanks for being patient with this. Tree-SHA512: 724fc3dffdbb1331584f89bbe84527e1af0d193a344fe43b36f2f2a628652d259001a3abf6b3909df53524cd3fbdbe3af760b7004d40d3bee1848fbb83efff5b

violet360 mentioned this pull request Feb 28, 2022

Add a method to PSBT to compute sighash::Prevouts #849

Closed

tcharding reviewed Mar 1, 2022

View reviewed changes

src/util/psbt/mod.rs Outdated Show resolved Hide resolved

src/util/psbt/mod.rs Outdated Show resolved Hide resolved

violet360 requested review from tcharding and Kixunil March 3, 2022 09:40

violet360 commented Mar 3, 2022

View reviewed changes

src/util/psbt/mod.rs Show resolved Hide resolved

Kixunil requested changes Mar 3, 2022

View reviewed changes

violet360 requested a review from Kixunil March 3, 2022 16:41

Kixunil reviewed Mar 3, 2022

View reviewed changes

violet360 requested a review from Kixunil March 3, 2022 18:17

Kixunil reviewed Mar 3, 2022

View reviewed changes

tcharding reviewed Mar 4, 2022

View reviewed changes

dr-orlovsky reviewed Mar 12, 2022

View reviewed changes

violet360 force-pushed the psbt branch from f7d0c63 to 04c4b1f Compare March 12, 2022 18:55

violet360 requested review from Kixunil, tcharding and dr-orlovsky March 12, 2022 19:02

sanket1729 reviewed Mar 13, 2022

View reviewed changes

violet360 force-pushed the psbt branch from 04c4b1f to 9f523e0 Compare March 14, 2022 06:15

violet360 requested a review from sanket1729 March 14, 2022 06:17

sanket1729 previously approved these changes Mar 14, 2022

View reviewed changes

dr-orlovsky previously approved these changes Mar 18, 2022

View reviewed changes

dr-orlovsky added this to Review in progress in PSBT Apr 1, 2022

tcharding mentioned this pull request Apr 18, 2022

Remove calls to debug_assert #953

Closed

API to get an iterator for funding utxos in psbt

5afb0ea

violet360 dismissed stale reviews from dr-orlovsky and sanket1729 via 5afb0ea April 25, 2022 12:54

violet360 force-pushed the psbt branch from 9f523e0 to 5afb0ea Compare April 25, 2022 12:54

violet360 requested a review from sanket1729 April 25, 2022 12:58

Kixunil approved these changes Apr 25, 2022

View reviewed changes

tcharding approved these changes Apr 25, 2022

View reviewed changes

PSBT automation moved this from Review in progress to Ready for merge Apr 25, 2022

sanket1729 approved these changes Apr 27, 2022

View reviewed changes

sanket1729 merged commit ee411a4 into rust-bitcoin:master Apr 27, 2022

PSBT automation moved this from Ready for merge to Done Apr 27, 2022

		@@ -171,6 +197,7 @@ mod display_from_str {
		#[cfg_attr(docsrs, doc(cfg(feature = "base64")))]
		pub use self::display_from_str::PsbtParseError;

	///Missing both the witness and non-witness utxo
	/// Missing both the witness and non-witness utxo.

	/// Triggered when output index goes out of bounds of the output in non witness utxo
	/// Triggered when output index does not exist in PSBT input (non-witness) UTXO.

	assert_eq!(self.inputs.len(), self.unsigned_tx.input.len());
	assert_debug_eq!(self.inputs.len(), self.unsigned_tx.input.len());

API to find funding utxos in psbt #853

API to find funding utxos in psbt #853

Conversation

violet360 commented Feb 28, 2022

Current status

Expected

Eunoia1729 commented Feb 28, 2022

tcharding left a comment

Choose a reason for hiding this comment

Kixunil left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kixunil left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kixunil left a comment

Choose a reason for hiding this comment

Kixunil Mar 3, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tcharding left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

violet360 Mar 9, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

violet360 Mar 7, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dr-orlovsky left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

violet360 Mar 12, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sanket1729 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kixunil commented Mar 14, 2022 • edited

violet360 commented Mar 14, 2022 • edited

sanket1729 left a comment

Choose a reason for hiding this comment

sanket1729 commented Mar 14, 2022 • edited

dr-orlovsky left a comment

Choose a reason for hiding this comment

dr-orlovsky commented Mar 18, 2022

sanket1729 commented Mar 18, 2022

dr-orlovsky commented Mar 30, 2022

Kixunil commented Apr 7, 2022

violet360 commented Apr 17, 2022

tcharding commented Apr 17, 2022

Kixunil left a comment

Choose a reason for hiding this comment

tcharding left a comment

Choose a reason for hiding this comment

sanket1729 left a comment

Choose a reason for hiding this comment

DanGould commented Jul 20, 2022

Kixunil Mar 3, 2022 •

edited

violet360 Mar 9, 2022 •

edited

violet360 Mar 7, 2022 •

edited

violet360 Mar 12, 2022 •

edited

Kixunil commented Mar 14, 2022 •

edited

violet360 commented Mar 14, 2022 •

edited

sanket1729 commented Mar 14, 2022 •

edited