Add a new `RecipientOnionFields` and replace `PaymentSecret` with it #2139

TheBlueMatt · 2023-03-30T00:06:37Z

This is the bulk of #2127 by LoC changes, but none of the actual work - it imply adds the RecipientOnionFields struct and pipes it through all the send calls. #2127 will then be relatively simple in just adding the metadata feature/field and putting it in the onion.

valentinewallace

Looks good after CI is fixed and tnull takes a look

lightning/src/ln/channelmanager.rs

valentinewallace · 2023-03-30T18:49:01Z

lightning/src/ln/channelmanager.rs

+	/// If you do not have one, the [`Route`] you pay over must not contain multiple paths as
+	/// multi-path payments require a recipient-provided secret.


Just checking, do we still plan to do #1222? I may have recently suggested it to a new contributor

I do not, but someone totally should!

TheBlueMatt · 2023-03-30T22:19:50Z

Oops, broke bench like always lol. The other issues are upstream cause futures broke us.

codecov-commenter · 2023-03-31T00:16:59Z

Codecov Report

Patch coverage: 98.46% and project coverage change: +0.25 🎉

Comparison is base (3b8bf93) 91.35% compared to head (c86099c) 91.60%.

❗ Current head c86099c differs from pull request most recent head f368f80. Consider uploading reports for the commit f368f80 to get more accurate results

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2139      +/-   ##
==========================================
+ Coverage   91.35%   91.60%   +0.25%     
==========================================
  Files         102      102              
  Lines       49909    52715    +2806     
  Branches    49909    52715    +2806     
==========================================
+ Hits        45592    48289    +2697     
- Misses       4317     4426     +109

Impacted Files	Coverage Δ
lightning/src/ln/channel.rs	`90.04% <ø> (-0.04%)`	⬇️
lightning/src/ln/functional_test_utils.rs	`94.35% <33.33%> (+1.76%)`	⬆️
lightning-invoice/src/payment.rs	`83.01% <66.66%> (+0.16%)`	⬆️
lightning/src/ln/channelmanager.rs	`89.83% <96.77%> (+0.70%)`	⬆️
lightning/src/ln/outbound_payment.rs	`90.68% <98.33%> (+0.15%)`	⬆️
lightning-invoice/src/utils.rs	`97.01% <100.00%> (-0.05%)`	⬇️
lightning/src/chain/chainmonitor.rs	`94.65% <100.00%> (-0.08%)`	⬇️
lightning/src/chain/channelmonitor.rs	`94.46% <100.00%> (ø)`
lightning/src/ln/chanmon_update_fail_tests.rs	`98.61% <100.00%> (+0.01%)`	⬆️
lightning/src/ln/functional_tests.rs	`98.92% <100.00%> (+0.69%)`	⬆️
... and 7 more

... and 11 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

TheBlueMatt · 2023-04-03T22:24:32Z

Rebased.

tnull

Generally looks good, only a few nits after a first pass.

CI is still sad though.

lightning-invoice/src/payment.rs

tnull · 2023-04-04T11:39:19Z

lightning/src/ln/channelmanager.rs

@@ -226,6 +226,38 @@ impl Readable for PaymentId {
 	}
 }

+/// Information which is provided, encrypted, to the payment recipient when sending HTLCs.
+///
+/// This should generally be constructed with data communicated to us from the recipient (via some


This sentence is very vague and I'm actually not sure what it tries to tell me. How is which data communicated to us? What forms of invoices are there and which should I expect..?

BOLT11 or BOLT12?

I don't know. Maybe the sentence can even be dropped, but in its current form I don't find it to add any value: "something might happen sometimes in a variety of ways, possibly".

We could make it stronger, I mean the point is that you're definitely getting the info from an invoice unless you're doing something super custom (or some "extension" protocol).

tnull · 2023-04-04T11:41:24Z

lightning/src/ln/channelmanager.rs

+/// form of invoice).
+#[derive(Clone)]
+pub struct RecipientOnionFields {
+	/// The [`PaymentSecret`] which is an arbitrary 32 bytes provided by the recipient for us to


nit: "(..) which consists of 32 random bytes that were provided by the recipient and should be included in the onion."

There's nothing that says it has to be random? In fact its not in LDK.

Fair enough, still the sentence could use some restructuring and "an 32 bytes provided" is simply missing a word, no?

Ah, right, needed to drop which.

TheBlueMatt · 2023-04-04T17:27:29Z

Oops, realized there were some small fixups from the parent PR that didnt end up here, pushed those. CI should fail on the each-commit-builds test but should pass after squash.

tnull · 2023-04-05T07:36:04Z

LGTM, will give it a final round after squash.

valentinewallace

LGTM after squash

lightning/src/ln/channelmanager.rs

Many of the fields in `HTLCSource::OutboundRoute` are used to rebuild the pending-outbound-payment map on reload if the `ChannelManager` was not serialized though `ChannelMonitor`(s) were after an HTLC was sent. As of 0.0.114, however, such payments are not retryable without allowing them to fail and doing a full, fresh, send. Thus, some of the fields can be safely removed - we only really care about having enough information to provide the user a failure event, not being able to retry. Here we drop one such field - the `payment_secret`, making our `ChannelMonitorUpdate`s another handful of bytes smaller.

This moves the public payment sending API from passing an explicit `PaymentSecret` to a new `RecipientOnionFields` struct (which currently only contains the `PaymentSecret`). This gives us substantial additional flexibility as we look at add both `PaymentMetadata`, a new (well, year-or-two-old) BOLT11 invoice extension to provide additional data sent to the recipient. In the future, we should also add the ability to add custom TLV entries in the `RecipientOnionFields` struct.

While most lightning nodes don't (currently) support providing a payment secret or payment metadata for spontaneous payments, there's no specific technical reason why we shouldn't support sending those fields to a recipient. Further, when we eventually move to allowing custom TLV entries in the recipient's onion TLV stream, we'll want to support it for spontaneous payments as well. Here we simply add the new `RecipientOnionFields` struct as an argument to the spontaneous payment send methods. We don't yet plumb it through the payment sending logic, which will come when we plumb the new struct through the sending logic to replace the existing payment secret arguments.

This passes the new `RecipientOnionFields` through the internal sending APIs, ensuring we have access to the full struct when we go to construct the sending onion so that we can include any new fields added there.

TheBlueMatt · 2023-04-05T16:32:35Z

Squashed and added one commit to correct events docs.

tnull self-requested a review March 30, 2023 08:19

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch from 855e240 to 3323137 Compare March 30, 2023 18:13

valentinewallace reviewed Mar 30, 2023

View reviewed changes

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch 3 times, most recently from 9ee94f2 to c86099c Compare March 30, 2023 22:31

TheBlueMatt added this to the 0.0.115 milestone Apr 3, 2023

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch from c86099c to c77a59d Compare April 3, 2023 22:24

tnull reviewed Apr 4, 2023

View reviewed changes

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch 2 times, most recently from fb26548 to f129e9a Compare April 4, 2023 17:26

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch 2 times, most recently from fa3c946 to f368f80 Compare April 4, 2023 17:42

valentinewallace reviewed Apr 5, 2023

View reviewed changes

lightning/src/ln/channelmanager.rs Show resolved Hide resolved

TheBlueMatt added 4 commits April 5, 2023 16:28

Pipe the new RecipientOnionFields through send pipeline

36235c3

This passes the new `RecipientOnionFields` through the internal sending APIs, ensuring we have access to the full struct when we go to construct the sending onion so that we can include any new fields added there.

TheBlueMatt force-pushed the 2023-03-metadata-prefactors branch from f368f80 to 36235c3 Compare April 5, 2023 16:28

Correct documentation of payment_id in events

77b1a22

tnull approved these changes Apr 7, 2023

View reviewed changes

valentinewallace approved these changes Apr 7, 2023

View reviewed changes

TheBlueMatt merged commit 1016e1f into lightningdevkit:main Apr 7, 2023
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a new `RecipientOnionFields` and replace `PaymentSecret` with it #2139

Add a new `RecipientOnionFields` and replace `PaymentSecret` with it #2139

TheBlueMatt commented Mar 30, 2023

valentinewallace left a comment

valentinewallace Mar 30, 2023

TheBlueMatt Mar 30, 2023

TheBlueMatt commented Mar 30, 2023

codecov-commenter commented Mar 31, 2023 •

edited

TheBlueMatt commented Apr 3, 2023

tnull left a comment

tnull Apr 4, 2023

TheBlueMatt Apr 4, 2023

tnull Apr 4, 2023

TheBlueMatt Apr 4, 2023

tnull Apr 4, 2023

TheBlueMatt Apr 4, 2023

tnull Apr 4, 2023

TheBlueMatt Apr 4, 2023

TheBlueMatt commented Apr 4, 2023

tnull commented Apr 5, 2023

valentinewallace left a comment

TheBlueMatt commented Apr 5, 2023

		/// If you do not have one, the [`Route`] you pay over must not contain multiple paths as
		/// multi-path payments require a recipient-provided secret.

Add a new RecipientOnionFields and replace PaymentSecret with it #2139

Add a new RecipientOnionFields and replace PaymentSecret with it #2139

Conversation

TheBlueMatt commented Mar 30, 2023

valentinewallace left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt commented Mar 30, 2023

codecov-commenter commented Mar 31, 2023 • edited

Codecov Report

TheBlueMatt commented Apr 3, 2023

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt commented Apr 4, 2023

tnull commented Apr 5, 2023

valentinewallace left a comment

Choose a reason for hiding this comment

TheBlueMatt commented Apr 5, 2023

Add a new `RecipientOnionFields` and replace `PaymentSecret` with it #2139

Add a new `RecipientOnionFields` and replace `PaymentSecret` with it #2139

codecov-commenter commented Mar 31, 2023 •

edited