Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow LDK node to send payjoin transactions #295

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

jbesraa
Copy link
Contributor

@jbesraa jbesraa commented May 21, 2024

Partially addresses #177

This adds the ability for the on chain wallet to send payjoin transactions as described in BIP 77 https://github.com/bitcoin/bips/blob/d7ffad81e605e958dcf7c2ae1f4c797a8631f146/bip-0077.mediawiki

The payjoin receiver part will be added in a separate PR with e2e tests with this pr code.

Copy link
Collaborator

@tnull tnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! Let me know when this is ready for a first round of review.

Until then only one early comment:

Cargo.toml Outdated
@@ -57,7 +57,7 @@ lightning-liquidity = { version = "0.1.0-alpha.4", features = ["std"] }

bdk = { version = "0.29.0", default-features = false, features = ["std", "async-interface", "use-esplora-async", "sqlite-bundled", "keys-bip39"]}

reqwest = { version = "0.11", default-features = false, features = ["json", "rustls-tls"] }
reqwest = { version = "0.11", default-features = false, features = ["json", "rustls-tls", "blocking"] }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I'd prefer to stick with the async variant, as the blocking variant introduces even more downstream dependencies, some of which just recently had some security issues, see #283 (comment).

@jbesraa
Copy link
Contributor Author

jbesraa commented May 22, 2024

@tnull this is ready for review

Copy link
Collaborator

@tnull tnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, did a first pass!

Cargo.toml Outdated
@@ -68,6 +68,7 @@ tokio = { version = "1", default-features = false, features = [ "rt-multi-thread
esplora-client = { version = "0.6", default-features = false }
libc = "0.2"
uniffi = { version = "0.26.0", features = ["build"], optional = true }
payjoin = { version = "0.15.0", features = ["send", "receive", "v2"] }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need receive if this is only the sending side?

Copy link
Contributor Author

@jbesraa jbesraa May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea we dont, it should be fixed in the next payjoin crate release payjoin/rust-payjoin#258

src/builder.rs Outdated
@@ -248,6 +257,12 @@ impl NodeBuilder {
self
}

/// Configures the [`Node`] instance to enable sending payjoin transactions.
pub fn set_payjoin_sender_config(&mut self, payjoin_relay: payjoin::Url) -> &mut Self {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To expose this in bindings we can't use Url as-is, we should likely just take a String here.

nit: Also possibly just?:

Suggested change
pub fn set_payjoin_sender_config(&mut self, payjoin_relay: payjoin::Url) -> &mut Self {
pub fn set_payjoin_relay(&mut self, relay: payjoin::Url) -> &mut Self {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this naming would be confusing when we add the payjoin receiver.
the payjoin receiver config will also require payjoin_relay argument.

the receiver config will be:

{
payjoin_directory: Url,
payjoin_relay: Url,
ohttp_keys: Option<OhttpKeys>
}

src/builder.rs Outdated
@@ -454,6 +471,11 @@ impl ArcedNodeBuilder {
self.inner.write().unwrap().set_gossip_source_p2p();
}

/// Configures the [`Node`] instance to enable sending payjoin transactions.
pub fn set_payjoin_sender_config(&self, payjoin_relay: payjoin::Url) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above: could use a rename and we need to use a String.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to String, please see the comment above regarding the naming

src/builder.rs Outdated
@@ -524,6 +546,7 @@ fn build_with_store_internal(
gossip_source_config: Option<&GossipSourceConfig>,
liquidity_source_config: Option<&LiquiditySourceConfig>, seed_bytes: [u8; 64],
logger: Arc<FilesystemLogger>, kv_store: Arc<DynStore>,
payjoin_sender_config: Option<&PayjoinSenderConfig>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this argument up to the other _configs, i.e., before seed_bytes.

src/builder.rs Outdated
@@ -973,6 +996,16 @@ fn build_with_store_internal(
};

let (stop_sender, _) = tokio::sync::watch::channel(());
let payjoin_sender = if let Some(payjoin_sender_config) = payjoin_sender_config {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use payjoin_sender_config.as_ref().and_then(|psc| { .. }) pattern as for liquidity_source above.

use std::ops::Deref;
use std::sync::Arc;
use std::time::Instant;
use tokio::time::sleep;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please don't import this globally, but rather use tokio::time::sleep in the code itself. Otherwise it's a bit confusing which sleep is meant and the blocking one from std really can't be used in async contexts.

) -> Option<Vec<u8>> {
let duration = std::time::Duration::from_secs(3600);
let sleep = || sleep(std::time::Duration::from_secs(10));
loop {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use the dedicated tokio macros for these control flows: tokio::select/tokio::time::timeout/tokio::time::interval, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure I fully follow how tokio::select can be utilised here, should I add a ticker similar to how its implement in the lib.rs and replace the loop with tokio::select ?

fixed the imports

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, if you figure out a way to do so, a combination of select and interval similar to lib.rs would be preferable to loop. If not, it's not that important.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still think we should clean this up using interval or similar.

}

// get original inputs from original psbt clone (ocean_psbt)
let mut original_inputs = input_pairs(&mut ocean_psbt).peekable();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ocean?

let mut ocean_psbt = ocean_psbt.clone();
// for BDK, we need to reintroduce utxo from original psbt.
// Otherwise we wont be able to sign the transaction.
fn input_pairs(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, here is a link explaining why this is needed bitcoindevkit/bdk-cli#156 (comment)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the link but I mostly meant the local function? Given the complex function definition and the additional Box, couldn't we just psbt.unsigned_tx.input.iter().zip(&mut psbt.inputs) directly where it's used?

}
}

let mut sign_options = SignOptions::default();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, I think these specifics should live in the Wallet-side methods.

@jbesraa
Copy link
Contributor Author

jbesraa commented May 22, 2024

Thanks @tnull
Ill address the review fully tomorrow.

@jbesraa jbesraa force-pushed the pj-sender branch 10 times, most recently from 21d1070 to c050240 Compare May 23, 2024 16:32
src/payjoin_sender.rs Outdated Show resolved Hide resolved
let sender = Arc::clone(&self.sender);
let (mut original_psbt, request, context) =
sender.create_payjoin_request(payjoin_uri, amount, fee_rate)?;
let response = sender.fetch(&request).await;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the idea here is that we try to fetch a response first, if we dont get one after the timeout we set on the request, we will start the background process to wait for the receiver response. I think we could emit an event in line 63

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's kind of confusing to provide two separate control flows to the user. Can't we just tell the user to check back regularly instead of polling for an ~arbitrary amount of time after which we'll fail quietly?

If we really think we need the background polling, why not avoid returing anything here and just return the necessary information via an event? This would also solve the async/sync issue above.

}
}

impl Filter for PayjoinHandler {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't implment but just use Filter.

self.internal_best_block_updated(height);
}

fn transaction_unconfirmed(&self, _txid: &Txid) {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to handle unconfirmations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea. how do you think it should be approached?

Thought about restarting states(confirmation hight, and others) and try to rebroadcast.

fn internal_transactions_confirmed(
&self, header: &Header, txdata: &TransactionData, height: u32,
) {
let (_, tx) = txdata[0];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we're only ever interested in the first confirmed transaction in a block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I might have misunderstood how this behaves. let me revisit this.

@jbesraa jbesraa force-pushed the pj-sender branch 3 times, most recently from 8cdcfc0 to 9ce890d Compare July 19, 2024 13:17
Copy link
Collaborator

@tnull tnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more comments, have yet to conduct a full top-bottom review of the current state.

src/types.rs Show resolved Hide resolved
type Builtin = String;

fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
Ok(ScriptBuf::from_hex(&val).map_err(|_| Error::InvalidPublicKey)?)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be a PublicKey? What if we'd ever want to use ScriptBuf outside of the narrow context you're using it for currently?

) -> Result<Psbt, Error> {
let fee_rate = self
.fee_estimator
.get_est_sat_per_1000_weight(ConfirmationTarget::OutputSpendingFee) as f32;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we really need to define a new LDK-Node specific ConfirmationTarget type that wraps LDK's but also adds other/custom variants. As this is mostly independent from this PR, it should probably happen in a separate prefactor PR. Let me know if you want to pick this up, otherwise I'm happy to do this in the coming days.

if proposed_txin.previous_output == original_txin.previous_output {
proposed_psbtin.witness_utxo = original_psbtin.witness_utxo.clone();
proposed_psbtin.non_witness_utxo = original_psbtin.non_witness_utxo.clone();
original_inputs.next();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think only advancing this if proposed_txin.previous_output == original_txin.previous_output means that we require the same ordering for original and proposed PSBTs. Is this a known requirement?

break;
},
Ok(None) => {
dbg!("Payjoin request sent, waiting for response...");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove dbg lines before pushing.

@jbesraa jbesraa force-pushed the pj-sender branch 2 times, most recently from db47845 to aad3e09 Compare July 22, 2024 11:28
@jbesraa
Copy link
Contributor Author

jbesraa commented Jul 22, 2024

@tnull bear with me a moment please as I am adding tests and polishing the pr to make it more clear

@tnull
Copy link
Collaborator

tnull commented Jul 22, 2024

@tnull bear with me a moment please as I am adding tests and polishing the pr to make it more clear

Of course, please ping me whenever you feel it's ready for the next round of review!

@jbesraa jbesraa force-pushed the pj-sender branch 8 times, most recently from a152d29 to 9c6b018 Compare July 25, 2024 08:58
@jbesraa
Copy link
Contributor Author

jbesraa commented Jul 25, 2024

@tnull Thanks for the ongoing review.

Few things done:

  1. Broke down the commit into smaller commits, hopefully it will be easier to review. Maybe before merging its worth squashing the last four commits into a single one as they are all around the Payjoin integration/implementation.
  2. Instead of introducing PayjoinTransaction struct in order to be able to track the transaction status, I added a couple of new properties to PaymentStore that would make it possible to track onchain transactions with the Confirm trait. I tried to not break the current API by setting both new properties, txid and best_block to None in the insert function.
  3. Reduced the number of Payjoin specific events to 3 after your last review.
  4. Added tests to cover the different responses the Payjoin receiver can communicate with the Payjoin sender.
  5. Updated the documentation to be more comprehensive.

There are still few things to cover/discuss, but I would appreciate a review over the usage of the PaymentStore, the Confirm trait and the overall integration of the PayjoinPayment in the code. I hope to get most of the commits ACK'd and to eventually focus on reviewing cd30d62 as its introducing the main BIP77 implementation.

Copy link
Collaborator

@tnull tnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did another ~half pass, have yet to take a closer look at the PayjoinHandler and Confirm logic (note we should probably also implement Listen while we're at it, as syncing full blocks is coming up).

Copy link
Collaborator

@tnull tnull Aug 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NACK on this change adding fields that only make sense in payjoin context to PaymentDetails directly. Please rather add a PaymentKind::Payjoin variant for payjoin transactions as we had discussed before.

@@ -40,6 +40,15 @@ pub(crate) const RESOLVED_CHANNEL_MONITOR_ARCHIVAL_INTERVAL: u32 = 6;
// The time in-between peer reconnection attempts.
pub(crate) const PEER_RECONNECTION_INTERVAL: Duration = Duration::from_secs(10);

// The time before a payjoin http request is considered timed out.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/http/HTTP/ here and below.

// The duration between retries of a payjoin http request.
pub(crate) const PAYJOIN_RETRY_INTERVAL: Duration = Duration::from_secs(3);

// The total duration of retrying to send a payjoin http request.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment isn't very helpful. What does 'total duration of retrying' mean? Do we abort the flow afterwards, or do we just give up?

PayjoinPaymentSuccessful {
/// This can refer to the original PSBT or to the finalised Payjoin transaction.
///
/// If [`is_original_psbt_modified`] field is `true`, this refers to the finalised Payjoin
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this doc link is broken.

/// This event is emitted only after one onchain confirmation. To determine the current number
/// of confirmations, refer to [`PaymentStore::best_block`]:.
///
/// [`PaymentStore::best_block`]: crate::payment::store::PaymentStore::best_block
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this doc link is broken, PaymentStore doesn't (and shouldn't) hold the best block.

mod spontaneous;
pub(crate) mod store;
mod unified_qr;

pub use self::payjoin::PayjoinPayment;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub use self::payjoin::PayjoinPayment;
pub use payjoin::PayjoinPayment;

@@ -0,0 +1,265 @@
use bitcoin::address::NetworkChecked;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move these imports down after the crate/lightning ones.


use lightning::chain::{BestBlock, Confirm, Filter};
use lightning::ln::channelmanager::PaymentId;
use lightning::log_error;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please import logging-specific things via the logger module, i.e., use crate::logger::{log_error, Logger, FilesystemLogger}


impl Confirm for PayjoinHandler {
fn transactions_confirmed(&self, header: &Header, txdata: &TransactionData, height: u32) {
self.internal_transactions_confirmed(header, txdata, height);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not inline these internal_ methods here? Do we gain something from them being separate?

Maybe they would be reusabe for implementing Listen, which we should probably also do?

)?;
let original_psbt = payjoin_handler.start_request(payjoin_uri.clone())?;
let payjoin_handler = Arc::clone(payjoin_handler);
let runtime = rt_lock.as_ref().unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we do this in other places, too, but might be best to avoid this unwrap. Could do this above:

		let runtime = if let Some(runtime) = rt_lock.as_ref() {
			runtime
		} else {
			return Err(Error::NotRunning);
		};

This adds two new properties to `PaymentStore, `txid` and `best_block`.
Both properties can be handy when handling on-chain payments that
require tracking. For example, in order to implement the `Confirm`
trait, its needed to access the transaction id and the best known block
in `Confirm::get_relevant_txid`.
Payjoin [`BIP77`] implementation. Compatible with previous Payjoin version [`BIP78`].

Should be retrieved by calling [`Node::payjoin_payment`].

Payjoin transactions can be used to improve privacy by breaking the common-input-ownership
heuristic when Payjoin receivers contribute input(s) to the transaction. They can also be used
to save on fees, as the Payjoin receiver can direct the incoming funds to open a lightning
channel, forwards the funds to another address, or simply consolidate UTXOs.

In a Payjoin transaction, both the sender and receiver contribute inputs to the transaction in
a coordinated manner. The Payjoin mechanism is also called pay-to-endpoint(P2EP).  The Payjoin
receiver endpoint address is communicated through a [`BIP21`] URI, along with the payment
address and an optional amount parameter.  In the Payjoin process, parties edit, sign and pass
iterations of the transaction between each other, before a final version is broadcasted by the
Payjoin sender.  [`BIP77`] codifies a protocol with 2 iterations (or one round of interaction
beyond address sharing).

[`BIP77`] Defines the Payjoin process to happen asynchronously, with the Payjoin receiver
enrolling with a Payjoin Directory to receive Payjoin requests. The Payjoin sender can then
make requests through a proxy server, Payjoin Relay, to the Payjoin receiver even if the
receiver is offline. This mechanism requires the Payjoin sender to regularly check for response
from the Payjoin receiver as implemented in [`Node::payjoin_payment::send`].

A Payjoin Relay is a proxy server that forwards Payjoin requests from the Payjoin sender to the
Payjoin receiver subdirectory. A Payjoin Relay can be run by anyone. Public Payjoin Relay
servers are:
- https://pj.bobspacebkk.com

A Payjoin directory is a service that allows Payjoin receivers to receive Payjoin requests
offline. A Payjoin directory can be run by anyone. Public Payjoin Directory servers are:
- https://payjo.in
The `Confirm` trait is implemented in order to track the Payjoin
transaction(s). We track two different transaction:

1. Original PSBT, which is the initial transaction sent to the Payjoin
   receiver. The receiver can decide to broadcast this transaction
   instead of finishing the Payjoin flow. Those we track it.
2. Final Payjoin transaction. The transaction constructed after
   completing the Payjoin flow, validated and broadcasted by the Payjoin
   sender.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants