Accommodating fixed receive amount and streaming payments

[adrianhopebailie]

This is a counter-proposal to #309 , #311 and #312 which takes what I consider to be the best parts of these and combine with a new constraint on ILP Addressing to accommodate all of the use cases we have explored.

Delivery vs Forwarding

This has been well-documented in the other issues. My position on this is that it is absolutely necessary to be able to deliver an exact amount to a receiver and that this constitutes the majority of payments today. So rather than trying to get around it I propose we find ways for it to work alongside other features of ILP that are needed to support streaming payments use cases.

Connectors MUST ignore destination amounts of zero

I support adopting #311 because I get that there is a use case it supports but I propose using zero and not MAX_UINT64 as the destination amount. As described in my response to that proposal I don't believe that the case against zero is strong and the use of MAX_UINT64 is ugly.

If a protocol requires implementation logic to look for special constants that suggests to me that there is something wrong with the protocol or something that has been hacked on.If we decide we need to switch to UInt128 for amounts in future or even Floats then this becomes very unwieldy.

In fact, I would argue that connectors that do not explicitly reject an amount of zero today would already behave correctly (i.e. as we require them to to support this use case) if they encountered a zero destination amount.

Maybe we don't need to do anything (or at most the implementation logic we need to change is not documented anyway)? That suggests to me this is the better of the two options.

Getting end-to-end quotes with ILQP for exact destination amount

Using failed payments to get a quote makes little sense if the intent is to make a fixed destination amount payment. It's possible in a volatile market that the sender will need to send multiple payments to get progressively closer to ultimately delivering the correct amount and may never actually succeed.

This also ignores the fact that connectors may not always have a linear pricing curve (i.e. they may have a fixed cost per payment irrespective of the size).

Therefor I think it is necessary to have a way to get an end-to-end quote that is precise (or at least has a high chance of being precise) but also gets around the possibility that a connector in the chain mistakenly believes it can deliver a payment when it should be forwarding that payment and therefor responds to the quote when it should forward it.

I believe that this is the use case that @emschwartz and @justmoon are trying to solve by using payments that are intended to be rejected.

Rather than abandon ILQP, let's make delivery vs forwarding more predictable for the sender and receiver...

Proposal: Connectors MUST forward all ILP packets unless the destination address differs from the ledger prefix by 2 segments or less.

In other words we specify that if a receiver appends transaction specific data to their address they will constrain this to only the last segment. If they wish to delimit this data they should use a delimiter other than ..

Also, receivers MUST always append a segment to their account address even if that segment is not transaction specific.

Examples:
Account address = g.ledger.bob
Address with tx specific suffix = g.ledger.bob.12345
Address with placeholder suffix = g.ledger.bob.~
Note the use of ~ could be adopted as the convention but need not be a standard.

If a connector g.ledger.alice receives a quote or payment that is addressed to g.ledger.* then they can attempt to deliver it but if the address is g.ledger.*.* then they MUST forward it.

We should apply this logic to forwarding ILQP QuoteByDestinationAmount and QuoteBySourceAmount packets, and ILP InterledgerPayment packets.

With regard to the complexity of supporting liquidity curves, I think it is reasonable to say that support for these is optional. All that is required is for any two nodes to be able to query:

1. The transfer amount required to deliver a destination amount to a destination address
2. The destination amount that will be delivered to a destination address if a specific amount is transferred

Depending on your use case (need for an exact delivery amount or not) you could use either a probing payment (as described below) or an ILQP quote request.

In the future you may also be able to source this data externally from a service that has indexed the network's available liquidity and rates. This proposal does not prevent this evolution.

Getting a quote by sending a "probing" payment

In #309 @emschwartz proposed that a sender could send a quote they expect to fail and in so doing establish the rate of exchange between the sender and receiver over that route. He proposes this as ILQPv2.

My position on this is that it doesn't actually require any changes to the protocol, and should not be viewed as a replacement for ILQP. If there is a use case for this, to accommodate streaming payments, then we should explore this mechanism further but it should not be seen as ILQPv2.

Further, if a "probing" payment is sent that senders want to be treated by connectors as closely as possible to a regular payment it should have no "special" features that make it identifiable to connectors. If connectors implement undocumented logic to treat these payments differently then we create a situation where further changes to the protocol are very difficult because we don't know how they will impact the intermediaries in the network. (i.e. This is a violation of the end-to-end principal)

My proposal is to use a random condition that can't be fulfilled and a destination amount of 0. Connectors should never do anything special with a destination amount of zero as this may be a regular streaming payment.

For this "probing" payment quoting mechanism to work the payment must be forwarded all the way to the receiver (not prematurely delivered) and the receiver must be able to express how much they actually would have received in the final transfer.

This requires that the Insufficient Destination Amount error (F04) should carry the actual amount received and that the sender use a destination amount of zero.

If we consider the address restriction I propose above then it's possible that this quoting mechanism may not be necessary at all but rather than make this an either/or decision we should allow both to be explored and let experience tell us which is best (or if we need both for different use cases).

Dealing with over-delivery when an exact amount is required

These new constraints make it more likely that a connector will over-deliver to a receiver because they attempt to forward a payment instead of delivering the final amount. (A receiver mistakenly puts too many segments on their address for example).

A new Destination Amount Too Large error (R03) should be defined and be thrown by receivers if the amount they receive is too large. Connectors that get this error can attempt to resend the last transfer with an adjusted amount.

Rejecting payments

As pointed out by @sharafian , all of this assumes that the ability to reject a payment is supported by ledgers. I.e. If the last connector sends a transfer to the receiver that is too small/large and it is rejected they can simply send a new transfer.

This may make the "probing" mechanism less attractive.

[michielbdejong]

propose using zero and not MAX_UINT64 as the destination amount.

We should not change the semantics of ILP packet type 1. Instead, we should just define ILP packet type 9 to mean the same as type 1, but with the amount moved into the data.

Proposal: Connectors MUST forward all ILP packets unless the destination address differs from the ledger prefix by 2 segments or less.

I think that solves a non-existing problem? As a connector, you know which ledgers you have an account on. Each ledger has a unique prefix, and no ledger prefix can ever start with the prefix of another ledger. So if for any of the ledgers on which you have an account, the destination address starts with that ledger's prefix, you send to the destination account, if not, you send to some connector account on some ledger, which you think may be able to forward it.

Example:

• Connie1 has three accounts: g.foo.connie1, g.bar.yep.connie1, g.b.a.z.connie1.
• Connie2 has two similar accounts: g.foo.connie2, g.some.connie2
• If Connie1 gets an incoming transfer with a payment for g.bar.yep.sub.ledger.bob, where should she send it? To g.bar.yep.sub, right? Because g.bar.yep. is a ledger, so that means g.bar.yep.sub is an account.
• If Connie1 gets an incoming transfer with a payment for g.some.sub.ledger.bob, where should she send it? She only has accounts on g.foo., g.bar., and g.b.a.z., so she cannot deliver in a single on-ledger transfer. Therefore, she can just try to forward to Connie2, over the g.foo. ledger. Connie2 will then deliver to g.some.sub, right?
• For the payment to g.bar.yep.sub.ledger.bob she will set the transfer amount to equal the packet amount, because since Connie1 knows g.bar.yep. is a ledger (she has an account there herself), she can know that it's g.bar.yep. that defines the unit-of-value for the packet amount; neither the shorter prefixes g. and g.bar., nor the longer prefixes g.bar.yep.sub. and g.bar.yep.sub.ledger. influence the meaning of the amount.
• For the payment to g.some.sub.ledger.bob there are three ways in which Connie1 can know the liquidity curve: either Connie2 sent her a route broadcast which didn't expire yet, or she recently asked for a quote and cached it, or she asks for a quote last-minute.

My position on [ILQP v2] is that it doesn't actually require any changes to the protocol, and should not be viewed as a replacement for ILQP.

I agree! If we introduce ILP packet type 9, payment-with-undisclosed-destination-amount, it will be a guessing game for intermediate connectors, they probably need to set their pricing based on extensive and continuous market research, or just try adjusting their pricing to find the sweet spot, but I think Evan and Stefan also evolved the idea now and it's now more a 'first payment in a stream' than a 'probe payment preceding the real payments'.

[adrianhopebailie]

We should not change the semantics of ILP packet type 1. Instead, we should just define ILP packet type 9 to mean the same as type 1, but with the amount moved into the data.

I disagree and unfortunately there is no scientific argument either way, it's a matter of opinion. I am advocating for a simple protocol (1 packet type) with the ability to use it different ways so that we allow real world use to drive design. If nobody uses the amount in the packet it will get deprecated.

I think that solves a non-existing problem?

If that were true then we wouldn't have a delivery vs forwarding discussion at all.

All of this theoretical discussion about extending payment chains is great but in the real world the likelihood is for people and businesses to have accounts with a service provider (bank or similar) and want the ability to accept a payment into that account of a specific amount. We should expect that service provider to take any buffer available when they deliver the payment. That requires them to perform some delivery vs forwarding logic.

Each ledger has a unique prefix, and no ledger prefix can ever start with the prefix of another ledger.

Not true. There could be a ledger called g.bigbank and another called g.bigbank.smallbank. If that is not allowed then you are simply replacing my address restriction with another new restriction.

In all of this we need to be cognizant that we have a working system that is usable in today's payment environment. I really like @justmoon 's description of payments using inexact amounts in #312 (comment) but that requires some fundamental changes a the application layer of a payment which I think is a way off.

Will we evolve to that eventually, quite likely, but ILP needs to be usable today and in that future which means we need to accommodate both and let the network evolve.

[michielbdejong]

If that is not allowed then you are simply replacing my address restriction with another new restriction.

If the understanding until now was that the unit of account on a subledger can differ from that of the parent ledger, then yes, I think we should restrict that. I guess I thought that restriction was already in place, but maybe it wasn't.

[emschwartz]

Can this be closed now?