Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BOLT04: Atomic Multi-path Payments [Draft] #658

Open
wants to merge 3 commits into
base: master
from

Conversation

@cfromknecht
Copy link
Collaborator

commented Aug 16, 2019

Hi all, decided to submit this draft concurrently with @rustyrussell's Base AMP since there is likely a bit of overlap, and comments can likely be shared between the two. The structure of the proposal and requirements is based on #643 to make that process simpler. The chosen feature bits are just temporary.

Background

This proposal outlines the requirements for Atomic Multi-path Payments originally outlined in @Roasbeef and I's email to lightning-dev. The proposal has also been referred to as OG AMP, or Moon AMP due to the moon-math involved :)

The core of the proposal is mostly unchanged, though I will highlight some distinctions:

  • The Extra Onion Blob (EOB) concept has been ditched in favor of the variable-sized hop-payloads via #619
  • Termination is signaled via total_msat (as in Base AMP) instead of a transmitting the number of shares. This is more flexible when sending partial payments adaptively, since the required number of partial payments may not be known upfront, while the total amount being paid should be known. Committing to the total amount being paid also eliminates weaknesses in zero-value invoices, or any scenario involving overpayment.
  • The 11-byte payment ID is replaced with a full 32-byte set_id
  • Various modifications in nomenclature

Summary

One of the biggest distinctions (some would argue drawback) from Base AMP is that the sender does not receive a preimage as a result of a successful payment. However, the lack of a "proof" of payment unlocks a range of novel features:

  • Partial Payment Decorrelation: each subpayment uses a different payment hash, offering stronger privacy from intermediaries
  • Concurrent-Safe Reusable Invoices: As a by-product of the sender generating the preimages and payment hashes, an invoice is no longer single use. The payer conveys to the payee which invoice is being settled by including the invoice's payment hash. All payments made to the invoice can be tracked as a single subscription or account, and each payment receives it's own unique set_id. Multiple, concurrent payments can be made to the same invoice and properly reconstructed using the enclosed set_id.
    • A subscription invoice might be generated as an invoice with the desired subscription amount, which will be verified each time the user pays.
    • An account invoice, e.g. a recurring invoice for funding an exchange, would use a zero-value invoice, allowing deposits (or withdrawals) of arbitrary amounts.
  • Spontaneous AMPs: More often discussed in the single-shot case, this describes users making payments to other nodes using just their public key (handwaves hop hints). However, AMP is really a generalization of the single-shot case, so we propose to merge these two concepts under a single, unified proposal. All of the tangential benefits of AMP are carried over to spontaneous single-shot payments (SSSP?), without introducing any additional code complexity.

Open Questions

  • Some of the fields included in option_amp are to be included in other proposals as well. Should we reuse the total_msat field from option_mpp, which can make use of the truncated encoding? Similarly, the set_id is analogous to the proposal to generically include a nonce in all payments, should that be it's own record? The signaling of which feature you are attempting to use may be more complex, but it's RISC-ier in some sense.

Feedback appreciated!

@cfromknecht cfromknecht force-pushed the cfromknecht:amp branch from 0032bdc to 80769cf Aug 16, 2019

@cfromknecht cfromknecht force-pushed the cfromknecht:amp branch from 80769cf to a29239a Aug 16, 2019

@Roasbeef

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

Some of the fields included in option_amp are to be included in other proposals as well. Should we reuse the total_msat field from option_mpp, which can make use of the truncated encoding?

I think that makes sense, as you can consider this to be a super set of the basic mpp in a sense. If we go in this direction, then should we switch to more of an "open coded" format for the AMP payload? This direction allows different use protocols to share common fields, but then create a blurry distinction between the allocated onion space amongst the various protocols. The other down side is that it consumes more type space, but with 64-bits available, that really isn't a concern IMO.

@Roasbeef

This comment has been minimized.

Copy link
Member

commented Aug 17, 2019

Similarly, the stream_id is analogous to the proposal to generically include a nonce in all payments, should that be it's own record? The signaling of which feature you are attempting to use may be more complex, but it's RISC-ier in some sense.

If we go this route, then we may need to allocate an additional tag to indicate the context, so receivers know how to interpret this field. In the MPP case, it's just to be matched against the invoice, while in the AMP case, it may need to be passed to the receiver via some sort of hook for additional validation (the account ID had actually authorized a deposit, etc).

| Bits | Name | Description | Link |
|-------|-------------------|--------------------------------------------------------------------|---------------------------------------|
| 8/9 | `var_onion_optin` | This node requires/supports variable-length routing onion payloads | [Routing Onion Specification][bolt04] |
| 12/13 | `option_amp` | Payee supports AMP | [BOLT #4](04-onion-routing#atomic-multi-path-payments)

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 17, 2019

Member

I wonder if we need a way to signal or communicate interdependent feature bits, as the option_amp bits are dependent on the node also supporting the var_onion_optin bit as well. On the other-hand, one could say that option_amp implies the other, though some clients may be stricter and only send an AMP payment if the var onion bit is set as well.

This also needs to define new feature bits and possibly tags in the BOLT 11 invoices as well.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

we may have to start defining which features depend on others in BOLT 9. then checking for option_amp would also recursively check w/e DAG of dependent features also required.

04-onion-routing.md Outdated Show resolved Hide resolved

The writer:
- MUST NOT include `option_amp` for any non-final node.
- if the sender has an invoice and `option_amp` feature was not set in the invoice:

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 17, 2019

Member

Should specify how this is to be communicated after #656 is merged.

- if it does include `option_amp`:
- MUST generate a random `stream_id` to be used on all HTLCs in the set.
- MAY send more than one HTLC using the same `stream_id`.
- MUST set the `share` values of all HTLCs such that their xor is a random

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 17, 2019

Member

MUST ensure all share values are unique?

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

I added SHOULD since it's not a hard requirement, and receiver will accept it all the same. I don't have a strong opinion tho, if others think we should use MUST

04-onion-routing.md Outdated Show resolved Hide resolved
04-onion-routing.md Show resolved Hide resolved
s_n = s_1 ^ ... ^ s_n-1 ^ r
```

If a partial payment fails, this process can be applied recursively to generate

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 17, 2019

Member

Def an underrated feature of this scheme!

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 19, 2019

Contributor

Is there a problem with the receiver reading the share value, but then canceling the htlc with for example an invalid onion key error? To the sender is looks like a failure that may even have been caused by the second last node, but in reality the receiver has already obtained the root seed.

04-onion-routing.md Show resolved Hide resolved
@joostjager

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

Could a proof of payment for atomic multi-path payments be achieved by locking the htlcs to two hashes (logical AND in the bitcoin script)? In order to settle such an htlc, the receiver needs to reveal both the preimage of the payment hash in the invoice and the preimage that was generated by the sender and embedded in the final onion payload.

s_n = s_1 ^ ... ^ s_n-1 ^ r
```

If a partial payment fails, this process can be applied recursively to generate

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 19, 2019

Contributor

Is there a problem with the receiver reading the share value, but then canceling the htlc with for example an invalid onion key error? To the sender is looks like a failure that may even have been caused by the second last node, but in reality the receiver has already obtained the root seed.

The `amt_to_forward` value will be the amount for this partial payment only. The
`option_amp` flag flag is a promise by the sender that the rest of the payment
will follow in succeeding HTLCs with the same `stream_id`; we call these HTLCs,
which that the same `stream_id`, an "HTLC set".

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 19, 2019

Contributor

I found set_id more descriptive than stream_id.

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

I think that this confusion is mostly because of the payment_id being different from our usual invoice-based payment_id. I think that stream_id should be renamed payment_id (it identifies this particular payment, which is sent in multiple parts), and payment_id should be renamed something else (subscription_id or something that makes sense for identifying a recurring payment?).

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

I found set_id more descriptive than stream_id.

I did too, reverted back to set_id for now.

I think that this confusion is mostly because of the payment_id being different from our usual invoice-based payment_id. I think that stream_id should be renamed payment_id (it identifies this particular payment, which is sent in multiple parts), and payment_id should be renamed something else (subscription_id or something that makes sense for identifying a recurring payment?).

The intention behind calling it payment_id was to keep some overlap with payment_hash, since they reuse the same field in the invoice. I'm not totally convinced on subscription_id, since that is only one of many use cases. Perhaps payment_addr is better than payment_id? In many ways it does behave more like an addr than an id

- MUST set the `payment_id` of each HTLC to the `payment_hash` in the
invoice.
- otherwise:
- MUST set the `payment_id` of each HTLC to zero.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 19, 2019

Contributor

Or define it as a separate tlv type? There may be some overlap with the random identifier generated by the receiver that was discussed before for regular payments and mpp. Both are ids generated by the receiver and both are not used to lock the htlc onto.

- otherwise:
- MAY fulfill the `i-th` HTLC in the set using `p_i`.
- otherwise:
- MUST fail an HTLC in set if its `cltv_expiry` elapses.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 19, 2019

Contributor

Shouldn't it be failed earlier than that? In case of an (amp) hodl invoice, an application assumes that when the invoice is marked as accepted (we know the root seed, but haven't pulled yet), the invoice cltv expiry condition holds.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

Wasn't sure exactly what you meant here, can you elaborate on "failed earlier"?

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 28, 2019

Contributor

Example:

  • current height: 100
  • (hodl) invoice cltv delta: 40, amt: 100
  • first htlc for 50 sats comes in with expiry 140 -> accepted as partial payment
  • new block comes in
  • second htlc for 50 sats comes in with expiry 141 -> accepted as partial payment

The hodl invoice should now move to the accepted state, because enough has been paid. We'd send the accepted event to rpc subscribers. But at that point, there are only 39 blocks left before the first htlc expires. The subscriber will probably assume that when the event comes in, the final cltv delta of 40 is still met.

@t-bast t-bast added this to Scheduled in Specification Meeting Agenda Aug 19, 2019

04-onion-routing.md Outdated Show resolved Hide resolved
The `amt_to_forward` value will be the amount for this partial payment only. The
`option_amp` flag flag is a promise by the sender that the rest of the payment
will follow in succeeding HTLCs with the same `stream_id`; we call these HTLCs,
which that the same `stream_id`, an "HTLC set".

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

I think that this confusion is mostly because of the payment_id being different from our usual invoice-based payment_id. I think that stream_id should be renamed payment_id (it identifies this particular payment, which is sent in multiple parts), and payment_id should be renamed something else (subscription_id or something that makes sense for identifying a recurring payment?).

04-onion-routing.md Outdated Show resolved Hide resolved
@@ -261,6 +268,197 @@ The reader:

The requirements for the contents of these fields are specified [above](#legacy-hop_data-payload-format).

## Atomic Multi-path Payments

If the final node receives an onion packet with `option_amp` field,

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

I think having both option_amp and option_mpp is very confusing (but hopefully the two proposals merge at least partially). I think that the main feature this proposal adds to rusty's MPP proposal is the spontaneous part (because option_mpp is currently also atomic - controlled by the recipient). Maybe the naming should reflect that (option_spontaneous_mpp)?

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

It might even make sense that this PR mutates in a specification of spontaneous payments (not invoice-based), encompassing both the multi-part aspect and the non multi-part?

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

I think that the main feature this proposal adds to rusty's MPP proposal is the spontaneous part

I disagree here, the primary goal here is not reuse payment hashes for better privacy:

  • intermediaries can't correlate subpayments
  • forwarding a payment doesn't leak anything about the invoice being paid. since all identifiable information is enclosed only for the receiver's eyes (if the invoice is public)
  • eliminates known probing vectors (of the payment hash) since it is never exposed directly over the network

The spontaneous + recurring pieces are useful side-effects of the sender generating the required randomness.

- SHOULD choose a unique child_index_i for each HTLC.
- MUST derive the `payment_hash` for an HTLC using `amp_child(r, child_index_i)`.
- if the invoice specifies a non-zero `amount`:
- MUST set `total_msat` to `amount`.

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

Shouldn't this allow over-payment (as specified for standard invoice payments)?

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Aug 28, 2019

Author Collaborator

yes probably, i think it should mirror whatever is finalized in #643

04-onion-routing.md Show resolved Hide resolved
04-onion-routing.md Outdated Show resolved Hide resolved
04-onion-routing.md Outdated Show resolved Hide resolved
04-onion-routing.md Outdated Show resolved Hide resolved
payment.

None of the requirements enforce that more than one HTLC is sent, permitting the
base case of 1 HTLC to function as a standalone spontaneous payment.

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

👏 I think this is important and this is why I view this work more as a "spontaneous payments" overall feature

@t-bast

This comment has been minimized.

Copy link
Collaborator

commented Aug 19, 2019

Great stuff @cfromknecht I think this adds very valuable features.

One quick thought about share failures.
Imagine the sender uses 3 shares and one fails. In order to preserve r, the sender has to either re-use the same share (with a different index of course) or split that share into multiple payments. If it's the recipient that failed one of the shares, the recipient will have learnt r. Could this be an issue (if for example the recipient has other nodes that may end up being intermediate nodes that route shares)?

Not sure it can be exploited, but just putting it out there to make sure it's obvious to everyone.

@joostjager joostjager referenced this pull request Aug 19, 2019
@cfromknecht

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 28, 2019

Okay I tried to respond to all of the immediately actionable comments (lmk if i've missed anything).

The major things that still seem in flux:

  • should we reuse option_mpp to encode total_msat? i'm leaning towards yes
  • should we have an additional option for set_id/nonce? part of me thinks it should really be a part of option_mpp, but if we also plan to use a nonce for all payments then maybe it should be its own feature

Could a proof of payment for atomic multi-path payments be achieved by locking the htlcs to two hashes (logical AND in the bitcoin script)? In order to settle such an htlc, the receiver needs to reveal both the preimage of the payment hash in the invoice and the preimage that was generated by the sender and embedded in the final onion payload.

Yes I think it is possible, but also requires the entire path to be upgraded before it can be used, unlike the current proposal which only requires sender/receiver to upgrade. If one really wants proof of payment tho, it's probably better to just use #643, since by adding the payment hash in all scripts along the path you lose decorrelation (and reveal that you're using AMP).

Is there a problem with the receiver reading the share value, but then canceling the htlc with for example an invalid onion key error? To the sender is looks like a failure that may even have been caused by the second last node, but in reality the receiver has already obtained the root seed.

If the receiver gets a subpayment and then cancels it back it can learn the root seed, but also won't have a payment to settle. At worst, the sender may try another shard (or split further into subshards), but the total amounts should never exceed what was already presented. In this sense it behaves like base amp, except that only the receiver can initiate the settlement (if they aren't colluding w/ intermediaries), instead of any intermediary who already knows the preimage to a subpayment.

Imagine the sender uses 3 shares and one fails. In order to preserve r, the sender has to either re-use the same share (with a different index of course) or split that share into multiple payments. If it's the recipient that failed one of the shares, the recipient will have learnt r. Could this be an issue (if for example the recipient has other nodes that may end up being intermediate nodes that route shares)?

Related to the above, if the receiver is actually in control of an intermediate node then it can perform a wormhole attack. But idt it is any different from what can already be done (for regular payments or base amp) when using payment hashes, and will eventually be fixed by moving to DLOG challenges :)

@cfromknecht cfromknecht force-pushed the cfromknecht:amp branch from 6a6017a to d0d4c7f Aug 28, 2019

@cfromknecht cfromknecht force-pushed the cfromknecht:amp branch from d0d4c7f to 34909d3 Aug 28, 2019

@cfromknecht

This comment has been minimized.

Copy link
Collaborator Author

commented Aug 28, 2019

should we have an additional option for set_id/nonce? part of me thinks it should really be a part of option_mpp, but if we also plan to use a nonce for all payments then maybe it should be its own feature

OTOH I would propose that we plan to transition all payment types over to mpp or amp, and deprecate the existing single-shot payments. Both proposals support sending 1 shard, and both schemes address known issues in the current single-shot payment flow.

In that case a reasonable split of the tlv records might be:

    1. type 8 (`option_mpp`)
    2. data:
        * [`32*byte`:`set_id`]
        * [`u64`:`total_msat`]
    1. type: 10 (`option_amp`)
    2. data:
        * [`32*byte`:`payment_id`]
        * [`32*byte`:`share`]
        * [`u16`:`child_index`]
@cfromknecht

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 4, 2019

Based on discussion in #643, mpp will include a receiver-generated nonce rather than a sender-generated one. Therefore the prior split would be better defined as:

    1. type 8 (`option_mpp`)
    2. data:
        * [`32*byte`:`payment_addr`]
        * [`u64`:`total_msat`]
    1. type: 10 (`option_amp`)
    2. data:
        * [`32*byte`:`set_id`]
        * [`32*byte`:`share`]
        * [`u16`:`child_index`]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
You can’t perform that action at this time.