Add API for constructing blinded payment paths #2412

valentinewallace · 2023-07-13T19:14:09Z

Lays some groundwork for route blinding support, helping address #1970. Lets us build blinded payment paths from the TLV payloads contained within them. A lot of the diff is code moves so we can separate out onion message-specific code from blinded payment-specific code.

We don't pad any values at the moment, which would increase privacy.

More testing is added in #2413.

alecchendev

Nice!

lightning/src/blinded_path/payment.rs

lightning/src/util/ser_macros.rs

codecov-commenter · 2023-07-20T20:37:43Z

Codecov Report

Patch coverage: 84.57% and project coverage change: -0.13% ⚠️

Comparison is base (b149279) 91.07% compared to head (9d481d1) 90.94%.
Report is 16 commits behind head on main.

❗ Current head 9d481d1 differs from pull request most recent head 63b60e2. Consider uploading reports for the commit 63b60e2 to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2412      +/-   ##
==========================================
- Coverage   91.07%   90.94%   -0.13%     
==========================================
  Files         106      108       +2     
  Lines       63310    62914     -396     
  Branches    63310    62914     -396     
==========================================
- Hits        57657    57220     -437     
- Misses       5653     5694      +41

Files Changed	Coverage Δ
lightning/src/events/mod.rs	`44.54% <0.00%> (-3.42%)`	⬇️
lightning/src/blinded_path/mod.rs	`85.10% <50.00%> (-9.15%)`	⬇️
lightning/src/blinded_path/payment.rs	`83.15% <83.15%> (ø)`
lightning/src/blinded_path/message.rs	`88.57% <88.57%> (ø)`
lightning/src/blinded_path/utils.rs	`94.73% <95.65%> (+0.98%)`	⬆️
lightning/src/ln/chan_utils.rs	`94.62% <100.00%> (ø)`
lightning/src/ln/channelmanager.rs	`89.05% <100.00%> (-0.60%)`	⬇️
lightning/src/offers/invoice_error.rs	`82.35% <100.00%> (ø)`
lightning/src/onion_message/messenger.rs	`86.75% <100.00%> (+0.06%)`	⬆️
lightning/src/onion_message/packet.rs	`95.19% <100.00%> (-0.10%)`	⬇️
... and 3 more

... and 17 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

valentinewallace · 2023-08-01T18:34:25Z

lightning/src/blinded_path/payment.rs

+		/// Used to authenticate the sender of a payment to the receiver and tie MPP HTLCs together.
+		payment_secret: PaymentSecret,


Note for reviewers: it's a bit LDK-specific and not generic that we take a PaymentSecret here. But as discussed previously with @TheBlueMatt, the spec should not be prescribing anything for the final payload at all because it's constructed for the receiver, by the receiver, so I don't see an issue with doing this instead of calling this field path_id and taking a Vec<u8> (which is an unnecessary allocation as well). But I get that this might be a bit confusing coming from the spec so thoughts welcome here.

Where is this encoded in the BlindedPath?

Not sure I understood the question but the payment_secret is written in the Writeable implementation for BlindedPaymentTlvs

Ah, I see. It's encoded in the BlindedPath when utils::encrypt_payload is called.

valentinewallace · 2023-08-01T18:42:10Z

Rebased due to conflicts and squashed minor fixups.

lightning/src/blinded_path/utils.rs

lightning/src/onion_message/messenger.rs

lightning/src/blinded_path/payment.rs

jkczyz · 2023-08-04T20:41:44Z

lightning/src/blinded_path/payment.rs

+		/// Used to authenticate the sender of a payment to the receiver and tie MPP HTLCs together.
+		payment_secret: PaymentSecret,


Where is this encoded in the BlindedPath?

lightning/src/blinded_path/mod.rs

jkczyz · 2023-08-08T19:33:27Z

lightning/src/blinded_path/payment.rs

+		/// Used to authenticate the sender of a payment to the receiver and tie MPP HTLCs together.
+		payment_secret: PaymentSecret,


Ah, I see. It's encoded in the BlindedPath when utils::encrypt_payload is called.

lightning/src/blinded_path/payment.rs

TheBlueMatt

Basically lg

TheBlueMatt · 2023-08-14T00:32:57Z

lightning/src/blinded_path/message.rs

+
+// Advance the blinded onion message path by one hop, so make the second hop into the new
+// introduction node.
+pub(crate) fn advance_path_by_one<NS: Deref, T: secp256k1::Signing + secp256k1::Verification>(


Don't we need the same utility for blinded payment paths (and do they have the same control tlvs)?

I was planning to do this as a follow-up to #2413, is your preference to get it out sooner? The control TLVs aren't the same.

No, that's alright, I just wanted to make sure that the move made sense - I guess the follow-up will mean basically copying this and writing some new code to do the same thing for blinded payment paths?

No, that's alright, I just wanted to make sure that the move made sense - I guess the follow-up will mean basically copying this and writing some new code to do the same thing for blinded payment paths?

Yep, that's the thinking

lightning/src/util/ser_macros.rs

lightning/src/blinded_path/utils.rs

lightning/src/blinded_path/payment.rs

TheBlueMatt · 2023-08-14T00:49:17Z

lightning/src/blinded_path/payment.rs

+		cltv_expiry_delta: path.iter().map(|(_, tlvs)| tlvs.cltv_expiry_delta())
+			.try_fold(0u16, |acc, delta| acc.checked_add(delta)).ok_or(())?,
+		htlc_minimum_msat: path.iter().map(|(_, tlvs)| tlvs.htlc_minimum_msat()).max().unwrap_or(0),
+		// TODO: this field isn't present in route blinding encrypted data


What's this TODO? Get the spec updated, or...?

I think the spec needs an update here, I commented on the route blinding PR lightning/bolts#765 (comment) and t-bast is supposed to get back to me soon.

Ended up adding this field as a separate parameter to BlindedPath::new_for_payment. See linked thread but it shouldn't be part of the blinded payment TLVs themselves.

lightning/src/blinded_path/payment.rs

valentinewallace · 2023-08-15T16:11:06Z

Rebased because the new spec test vector failed with the new code, fixed now.

lightning/src/blinded_path/payment.rs

lightning/src/blinded_path/mod.rs

lightning/src/blinded_path/payment.rs

TheBlueMatt · 2023-08-16T21:22:29Z

lightning/src/blinded_path/payment.rs

+			.map(|f| f / 1_000_000)
+			.ok_or(())?;
+	}
+	let htlc_minimum_msat = path.iter().map(|(_, tlvs)| tlvs.htlc_minimum_msat()).max().unwrap_or(0);


This needs to consider the fees charged earlier - if the second node in the path wants at least 1 sat, and the previous node charges 1 sat in fee, the min should be 2 sats.

Digging into this a bit, it looks like eclair/CLN don't do this. Are we sure the htlc_minimum_msat field isn't the minimum final receive amount to the blinded path, i.e. the min amount prior to the aggregated fees being added? At first glance looks like get_route will treat the min to send to the blinded path as BlindedPayInfo::htlc_minimum_msat + aggregated_fees.

Hmm, we should ask them, then, cause that doesn't make much sense to me - if we have a blinded path along A -> B -> C, and A charges a 10% fee and B has a min_htlc of 100 sats, then the min HTLC for the full path needs to be 110 sats. The total min depends on where the max-min is in the path.

lightning/src/blinded_path/payment.rs

This way it can be more easily reused for blinded payment paths.

We'll similarly separate blinded path payments code into its own module.

Useful for blinded payment path construction.

We want a similar macro for reading TLV streams without a length prefix, so rename this one to disambiguate.

valentinewallace · 2023-08-22T17:26:43Z

Rebased due to conflicts and squashed.

TheBlueMatt

Basically all LGTM. Only real issue is the 8 byte restirction on the padding cause we didn't tell rustc what type we meant (which is maybe the most obvious rustc bug I've ever come across that they insist is definitely a Feature 🤦 )

lightning/src/util/ser_macros.rs

lightning/src/blinded_path/utils.rs

lightning/src/blinded_path/mod.rs

TheBlueMatt · 2023-08-22T23:30:01Z

lightning/src/blinded_path/payment.rs

+impl Readable for BlindedPaymentTlvs {
+	fn read<R: io::Read>(r: &mut R) -> Result<Self, DecodeError> {
+		_init_and_read_tlv_stream!(r, {
+			(1, _padding, option),


By not defining a type here, I believe rustc magically assumed you wanted an i64 🎉 , because this crate uses i64s a ton, all over the place, its definitely what we wanted.....and also we'll fail to read if the padding isn't either empty or exactly 8 bytes :(

Would the trick used in ControlTlvs work?

let _padding: Option<Padding> = _padding;

Yeah, that should work. I think I'll add some docs to impl_writeable* saying that you can't have unused read values. It would be nice if there were a way to enforce that programmatically, haven't looked into it though.

Useful for when you want to use _init_and_read_len_prefixed_tlv_fields but there is no length byte at the start of the TLV stream.

Also adds a util for general blinded hop creation to be reused for blinded payment paths.

The previous name can be confused for the shared secret that the rho is derived from.

valentinewallace mentioned this pull request Jul 13, 2023

BOLT12 Offers Tracking Issue #1970

Open

57 tasks

valentinewallace added this to the 0.0.117 milestone Jul 13, 2023

valentinewallace mentioned this pull request Jul 13, 2023

Route blinding MVP #2413

Merged

1 task

alecchendev reviewed Jul 20, 2023

View reviewed changes

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from cd8952d to a6d08fe Compare July 20, 2023 19:27

valentinewallace commented Aug 1, 2023

View reviewed changes

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch 2 times, most recently from fd88452 to 1fa4ecd Compare August 1, 2023 18:41

jkczyz reviewed Aug 4, 2023

View reviewed changes

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from 387e06e to 4b7b7e8 Compare August 8, 2023 18:36

jkczyz reviewed Aug 8, 2023

View reviewed changes

alecchendev reviewed Aug 9, 2023

View reviewed changes

lightning/src/blinded_path/payment.rs Show resolved Hide resolved

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch 2 times, most recently from 7f8cec6 to 4bb7ea6 Compare August 9, 2023 21:43

TheBlueMatt reviewed Aug 14, 2023

View reviewed changes

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch 2 times, most recently from bed22b0 to a52d43e Compare August 15, 2023 16:10

TheBlueMatt reviewed Aug 15, 2023

View reviewed changes

lightning/src/blinded_path/payment.rs Outdated Show resolved Hide resolved

lightning/src/blinded_path/payment.rs Outdated Show resolved Hide resolved

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from a52d43e to 9d481d1 Compare August 16, 2023 18:51

TheBlueMatt reviewed Aug 16, 2023

View reviewed changes

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch 2 times, most recently from 3b75164 to 74f728f Compare August 21, 2023 17:59

TheBlueMatt linked an issue Aug 21, 2023 that may be closed by this pull request

Pass RoutingFees to channel_penalty_msat #2509

Closed

TheBlueMatt removed a link to an issue Aug 21, 2023

Pass RoutingFees to channel_penalty_msat #2509

Closed

jkczyz reviewed Aug 21, 2023

View reviewed changes

lightning/src/blinded_path/payment.rs Outdated Show resolved Hide resolved

lightning/src/blinded_path/payment.rs Outdated Show resolved Hide resolved

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from 74f728f to c9d2968 Compare August 22, 2023 15:16

Move blinded path util into blinded_path::utils

a5b7cf2

This way it can be more easily reused for blinded payment paths.

valentinewallace added 6 commits August 22, 2023 13:18

Move some blinded path message code into message submodule.

381cc64

We'll similarly separate blinded path payments code into its own module.

Move blinded message path util into message submodule

fe5a076

Move Padding into blinded_path module for use in blinded payments

1b35661

Update blinded path util to take iterator instead of slice

7c1726b

Useful for blinded payment path construction.

Minor BlindedHop docs update

9777485

Rename ser macro

4a30d9e

We want a similar macro for reading TLV streams without a length prefix, so rename this one to disambiguate.

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from c9d2968 to 63b60e2 Compare August 22, 2023 17:26

jkczyz previously approved these changes Aug 22, 2023

View reviewed changes

valentinewallace mentioned this pull request Aug 22, 2023

Aggregate BlindedPayInfo for blinded routes #2514

Merged

TheBlueMatt reviewed Aug 22, 2023

View reviewed changes

valentinewallace added 6 commits August 23, 2023 11:24

Add new _init_and_read_tlv_stream ser macro

cf64e3f

Useful for when you want to use _init_and_read_len_prefixed_tlv_fields but there is no length byte at the start of the TLV stream.

Simplify onion message blinded hop construction

d224f98

Also adds a util for general blinded hop creation to be reused for blinded payment paths.

Support constructing BlindedPaths for payments.

76f8cc1

Blinded paths: rename encrypted_tlvs_ss to *_rho for precision

0ddd3cb

The previous name can be confused for the shared secret that the rho is derived from.

Fix documentation on onion message packet ControlTlvs

ebb0676

Document _init_and_read_* ser macro requirements

ea84f2a

valentinewallace dismissed jkczyz’s stale review via ea84f2a August 23, 2023 15:29

valentinewallace force-pushed the 2023-07-construct-blinded-paths branch from 63b60e2 to ea84f2a Compare August 23, 2023 15:29

TheBlueMatt approved these changes Aug 23, 2023

View reviewed changes

jkczyz approved these changes Aug 23, 2023

View reviewed changes

TheBlueMatt merged commit 0211daa into lightningdevkit:main Aug 23, 2023
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add API for constructing blinded payment paths #2412

Add API for constructing blinded payment paths #2412

valentinewallace commented Jul 13, 2023 •

edited

alecchendev left a comment

codecov-commenter commented Jul 20, 2023 •

edited

valentinewallace Aug 1, 2023

jkczyz Aug 4, 2023

valentinewallace Aug 5, 2023

jkczyz Aug 8, 2023

valentinewallace commented Aug 1, 2023

jkczyz Aug 4, 2023

jkczyz Aug 8, 2023

TheBlueMatt left a comment

TheBlueMatt Aug 14, 2023

valentinewallace Aug 15, 2023

TheBlueMatt Aug 15, 2023

valentinewallace Aug 16, 2023

TheBlueMatt Aug 14, 2023

valentinewallace Aug 14, 2023

valentinewallace Aug 16, 2023

valentinewallace commented Aug 15, 2023 •

edited

TheBlueMatt Aug 16, 2023

valentinewallace Aug 21, 2023

TheBlueMatt Aug 21, 2023

valentinewallace commented Aug 22, 2023

TheBlueMatt left a comment

TheBlueMatt Aug 22, 2023

jkczyz Aug 23, 2023

valentinewallace Aug 23, 2023

		/// Used to authenticate the sender of a payment to the receiver and tie MPP HTLCs together.
		payment_secret: PaymentSecret,

Add API for constructing blinded payment paths #2412

Add API for constructing blinded payment paths #2412

Conversation

valentinewallace commented Jul 13, 2023 • edited

alecchendev left a comment

Choose a reason for hiding this comment

codecov-commenter commented Jul 20, 2023 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace commented Aug 1, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace commented Aug 15, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace commented Aug 22, 2023

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace commented Jul 13, 2023 •

edited

codecov-commenter commented Jul 20, 2023 •

edited

valentinewallace commented Aug 15, 2023 •

edited