Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Base AMP #643

Open
wants to merge 20 commits into
base: master
from

Conversation

@rustyrussell
Copy link
Collaborator

commented Jul 17, 2019

This is a commit-separated and TLV-enhanced version of #511 by @ZmnSCPxj

We need the total amount, due to amountless invoices.

It assumes, and is based on, #619, so only read the last 3 commits!

@ZmnSCPxj
Copy link
Collaborator

left a comment

Otherwise seems OK to me.

Show resolved Hide resolved 04-onion-routing.md
@@ -277,6 +277,10 @@ The field is big-endian. The least-significant bit is numbered 0,
which is _even_, and the next most significant bit is numbered 1,
which is _odd_.

| Bits | Name |Description | Link |
|------|-------------|-----------------------|------------------------------------------------|
| 0/1 | `basic_mpp` | Payee understands mpp.| [BOLT #4](04-onion-routing.md#basic-multi-part-payments) |

This comment has been minimized.

Copy link
@ZmnSCPxj

ZmnSCPxj Jul 17, 2019

Collaborator

What would be the point of using the even bit?

For odd bits, it seems to me to mean, "payer may use basic_mpp".

For even bits, what is it mean? "Payer must use basic_mpp"?

This comment has been minimized.

Copy link
@rustyrussell

rustyrussell Jul 18, 2019

Author Collaborator

We always assign in pairs. In future, it's theoretically possible you could set this if no single channel had sufficient capacity, I guess, but it's a bit weird!

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

We may also want to add an additional section here w.r.t path finding advisory using routing hints. If the amount of the invoice is greater than available inbound bandwidth of any of their channels, then they can "guide" the sender with regular or "special" routing hints to increase the likelihood of success.

@rustyrussell rustyrussell force-pushed the rustyrussell:guilt/base-amp branch from 435c1ed to 7dc7a40 Jul 17, 2019

@t-bast
Copy link
Collaborator

left a comment

Good stuff, this is great to get started with rather simple multi-path payments and learn from real-world usage before adding more advanced features.

Show resolved Hide resolved 04-onion-routing.md
Show resolved Hide resolved 04-onion-routing.md Outdated
Show resolved Hide resolved 04-onion-routing.md Outdated
Show resolved Hide resolved 04-onion-routing.md Outdated

cdecker and others added some commits Mar 28, 2019

bolt04: Formatting cleanup and fold clarifications into conventions
The clarifications were tacked on after the fact, but they should really be
part of the conventions. I also updated the links to use the reference style,
which results in better text flow and makes it easier to read the source.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Describe the variable size `hop_payload`
Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Amend the filler generation and onion decoding to varpayload
This actually introduces the variable size shift and filler generation.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Remove in-spec test vector in favor of JSON test vector
Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Add the TLV types for the new payload format
Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Note that realm 0x01 is reserved and not usable for TLVs
As proposed during the spec meeting, realm 0x01 provides us with an escape
hatch should we ever come up with the new best thing since sliced bread.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
bolt04: Introduce the destination_signal to the tlv_payload
As discussed during the spec meeting this allows us not to use the 32 byte
HMAC to identify the last hop, and use a 2-byte signal instead.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
BOLT 11: Add feature bits.
Most obviously, we want this for BASE AMP, but it's useful in future.

Even though even bits won't cause existing implementations to know
they can't pay the invoice, it will allow it in future once everyone
has upgraded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
BOLT 11: Add basic_mpp feature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
BOLT 11: Add test vectors for feature bitfield.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
BOLT 4: basic_mpp support (a.k.a. Base AMP)
We send the total they expect to receive, and they wait >= 60 seconds
to get all the pieces.

Based-on: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

@rustyrussell rustyrussell force-pushed the rustyrussell:guilt/base-amp branch from e63c872 to 70ec87d Jul 18, 2019

@rustyrussell

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 18, 2019

Typos fixed, spellcheck made happy, rebased to solve conflict.

No non-formatting changes.

#### Requirements

The writer:
- MUST not include `basic_mpp` for any non-final node.

This comment has been minimized.

Copy link
@flack

flack Jul 18, 2019

not should be uppercase

@rustyrussell

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 19, 2019

More typo fixes...

@rustyrussell rustyrussell force-pushed the rustyrussell:guilt/base-amp branch from bc78a2f to 8ba54e6 Jul 19, 2019

fixup! More typos
Tabs-> 4 spaces (thanks @ZmnSCPxj)
"MUST not" -> "MUST NOT" (thanks @flack)

@rustyrussell rustyrussell force-pushed the rustyrussell:guilt/base-amp branch from 8ba54e6 to e79f8a8 Jul 19, 2019

@rustyrussell rustyrussell referenced this pull request Jul 19, 2019

Merged

Simple tooling fixes #650

@Roasbeef

This comment has been minimized.

Copy link
Member

commented Jul 27, 2019

Can be rebased now that the onion stuff has been merged in.

@@ -140,6 +141,9 @@ Currently defined tagged fields are:
* `fee_base_msat` (32 bits, big-endian)
* `fee_proportional_millionths` (32 bits, big-endian)
* `cltv_expiry_delta` (16 bits, big-endian)
* `9` (5): `data_length` variable. One or more bytes containing features

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

This is already covered for regular AMP, but for regular MPP, I think we should also include a random identifier here and require that the sender include this for end-to-end security (in the onion). Otherwise, if it's a 0-value invoice, then intermediate nodes can probe, and then use that information to possibly take most/all of the HTLC as "fees".

This comment has been minimized.

Copy link
@t-bast

t-bast Jul 29, 2019

Collaborator

Great point, there is indeed an attack vector here for the next-to-last node!
Your suggested fix assumes that the payment request will stay hidden from the intermediate nodes though (otherwise it doesn't fix anything if the intermediate nodes also know this random identifier).
It could be important to highlight this point to applications that want to leverage MPP: using MPP for donations should be a two-steps process, where a user who wants to donate needs to go to your website to get an invoice generated for him (and only him). That invoice must not be shared publicly.

@@ -277,6 +277,10 @@ The field is big-endian. The least-significant bit is numbered 0,
which is _even_, and the next most significant bit is numbered 1,
which is _odd_.

| Bits | Name |Description | Link |
|------|-------------|-----------------------|------------------------------------------------|
| 0/1 | `basic_mpp` | Payee understands mpp.| [BOLT #4](04-onion-routing.md#basic-multi-part-payments) |

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

We may also want to add an additional section here w.r.t path finding advisory using routing hints. If the amount of the invoice is greater than available inbound bandwidth of any of their channels, then they can "guide" the sender with regular or "special" routing hints to increase the likelihood of success.

@@ -257,6 +257,9 @@ This is a more flexible format, which avoids the redundant `short_channel_id` fi
1. type: 6 (`short_channel_id`)
2. data:
* [`short_channel_id`:`short_channel_id`]
1. type: 8 (`basic_mpp`)
2. data:
* [`tu64`:`total_msat`]

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

I suggest a minimal two-tuple of (amt, randID) as mentioned in my earlier comment.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 5, 2019

Shouldn't randID be defined as a separate type, so that it can be used to pay non-amp to zero-valued invoices?

- MAY send more than one HTLC to pay the invoice.
- MUST use the same `payment_preimage` on all HTLCs in the set.
- MUST set `total_msat` to the amount it wishes to pay.
- SHOULD send all payments at approximately the same time.

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

This conflicts w/ two bullets below (try to re-derive if they fail).

We should also have a section below here detailing failure modes due to partial settles, as it's possible that some portion of the funds just never get to the receiver, and become windfall for intermediate nodes.

This comment has been minimized.

Copy link
@t-bast

t-bast Jul 29, 2019

Collaborator

We should also have a section below here detailing failure modes due to partial settles, as it's possible that some portion of the funds just never get to the receiver, and become windfall for intermediate nodes.

Can you explain what you mean? I didn't get that.

- if the total `amount_msat` of this HTLC set equals or exceeds `total_msat`:
- SHOULD fulfill all HTLCs in the HTLC set
- otherwise:
- SHOULD wait for at least 60 seconds for remaining members of the HTLC set.

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

A static value here isn't very encompassing. It may be the case that a higher level application has the sender purposefully delay the remaining shards.

This comment has been minimized.

Copy link
@t-bast

t-bast Jul 29, 2019

Collaborator

True but I think the receiver should enforce a maximum duration for a receive session, otherwise a malicious sender can lock funds from a lot of people almost indefinitely (if what you were thinking of was some kind of sliding window on the receiver end where you reset the timeout each time you receive a valid share).

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 5, 2019

Could it be useful to specify the hold timeout in the onion payload, so that the sender picks this value themselves?

This comment has been minimized.

Copy link
@rustyrussell

rustyrussell Aug 5, 2019

Author Collaborator

Yeah, we need to pick something, and it can't really be up to the sender or recipient since it effects the entire network. Implementations will pick something, so might as well specify it.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 12, 2019

In addition to the seconds timeout, does something need to be specified in terms of block height? It could be that the first htlc comes in, meets the invoice cltv delta requirement, but its value isn't high enough to settle the invoice. The htlc is held. Then a block comes in. At that point, the cltv delta requirement isn't met anymore. Should the htlc be canceled?


Because invoices do not necessarily specify an amount, and because
payers can add noise to the final amount, the total amount must be
sent explicitly. The requirements allow exceeding this slightly, as

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Jul 27, 2019

Member

If we don't add a mandatory random identifier along the way, then we should discourage sending more than the invoice amount.

- otherwise:
- MUST add it to the HTLC set corresponding to that `payment_hash`.
- SHOULD fail the entire HTLC set if `total_msat` is not the same for
all HTLCs in the set.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 5, 2019

Does this mean that an adversary can prevent an amp payment from completion by periodically sending a very small shard with a random total_msat, thereby cancelling any shards that have been acquired?

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 5, 2019

I guess those htlcs are cancelled early in the process because their total_msat doesn't match what is in the invoice database. How would it work in that case for zero-valued invoices? First htlc that arrives sets the amount?

This comment has been minimized.

Copy link
@rustyrussell

rustyrussell Aug 6, 2019

Author Collaborator

You're right, will change to ignore & fail if total_msat is different to avoid griefing.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 12, 2019

We must make sure that an attacker can't cancel other shards, but also that they can't block the genuine shards from being accepted. If the attacker is the first to set the tracked total_msat to a random value, they could possibly block the payment for the timeout duration (and repeat this thereafter).

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 12, 2019

In that light, the proposed randID may actually be required not only to prevent fee stealing with 0-valued invoices, but also to keep an attacker from interfering with the multi path settle process.


| Bits | Name |Description | Link |
|------|-------------|-----------------------|------------------------------------------------|
| 0/1 | `basic_mpp` | Payee understands M.P.P.| [BOLT #4](04-onion-routing.md#basic-multi-part-payments) |

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 5, 2019

Will there also be a global feature bit that advertises mpp in the graph, so that spontaneous AMP payments can be made too?

I think that in that case the invoice flag is still required, because the payer may not have the (up to date) node features of the payee? Does this mean that basic_mpp implies var_onion_optin?

@rustyrussell rustyrussell referenced this pull request Aug 6, 2019

Open

Bolt11 feature bits #656

- MUST set `total_msat` to at least that `amount`, and less
than or equal to twice `amount`.
- MUST ensure that the total `amount_msat` of the HTLC set which arrive at the payee
is greater or equal to `total_msat`.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 12, 2019

What is the rationale for allowing the total htlc set amount_msat to be greater than total_msat? And should this be capped too, to be consistent with single path payments? Otherwise the following would be possible (if I understood this correctly): invoice amt: 1000 sat, total_msat: 1000 sat, first htlc: 500 sat, second htlc: 1 btc?

- MUST ensure that the total `amount_msat` of the HTLC set which arrive at the payee
is greater or equal to `total_msat`.
- MUST ensure that no (incomplete) non-failed subset of the HTLC set adds
to a total `amount_msat` greater or equal to `total_msat`.

This comment has been minimized.

Copy link
@joostjager

joostjager Aug 12, 2019

What does this mean exactly? If the sender has enough htlcs in flight to pay the invoice but the htlcs are not settled, they shouldn't add another htlc?

This comment has been minimized.

Copy link
@t-bast

t-bast Aug 19, 2019

Collaborator

Exactly that's it. If you've sent an HTLC set that pays the total amount, you should wait for (some of) them to fail or settle. If you send another HTLC you will end up paying more than what you intended to pay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.