ILQPv2: Replace ILQP with End-to-End Quoting #309
Conversation
emschwartz
requested review from
adrianhopebailie,
justmoon,
sappenin,
michielbdejong,
sharafian,
earizon and
diminator
|
Also, if quotes are sent over transfers we might not need the ability to send plain messages (or "requests") on the ledger layer... |
|
On one hand I think this would be really nice, because it means the ledger layer and interledger layer get a lot simpler. It also makes it possible for quote results to travel out of band, or for rates to be determined from recent payments. Having a flexible quoting layer would be really nice because of the point you raised that different applications need to rely on quoting differently. On the other hand, I think it makes our stack a little more illogical. Sending a message and sending a payment are logically two separate operations, even if you can perform both of them with the same sendTransfer primitive. For instance, if a transfer is rejected, that's because of some problem. A connector could use that to flag malicious behavior, if someone is sending a lot of failed payments through them. But if rejected payments just carry messages, the connector can no longer do that. Because sending a transfer puts liquidity on hold, most plugins will want to implement ILQPv2 specially by not actually reserving funds when the ILQPv2 condition is included. But from the developer's perspective this is very strange; the way to send a message is by passing a cryptic argument into the transfer function. It also means that the incoming reject event also means that a response has come in. I get that ledger messaging is a strange concept, but this seems even harder to explain to people. There's also the question of caching. Can a connector keep rates and then reject these payments early with the cached rate? Are connectors ever advised to do so, if they recognize the protocol? Overall I like the simplification but I'm not sure if it's just too "clever" and might turn out to be impractical in a practical context. It saves us some fixed amount of headache when we're trying to get an MVP integration of some system, but in all practicality we're going to want real messaging before we call it production ready. Furthermore, it makes administrating a connector that much harder, because it makes it difficult for them to flag malicious traffic. The upgradability is a good point though, and it might be worth it for that. |
Connectors could still look at your rate of failed payments versus other people and put some limit on the percentage they're willing to tolerate. Overall, I hear you but I think it's well worth the weirdness. Even if everyone or mostly everyone implements special functionality for quoting, the timeline is important. If quoting is a function that all connectors implement, you would need everyone to upgrade before a new quoting protocol becomes useful. That's a change on the scale of IPv4 -> IPv6. In contrast, if there's an incentive to treat these special messages differently but it still works perfectly fine if you don't, that's a much better upgrade path. I'd compare that to something like SPDY/HTTP2. Google implemented it in Chrome and on google.com, then a bunch of other people said "hey I want my website to be faster too", and then it was worked into a proper standard. If quoting is an end-to-end concern it turns into that sort of situation. People don't have to implement it, but if it benefits you you'll probably get around to it at some point. The more widespread a standard is, the bigger the difference this will make. For an example of this, think of how long it took us to update ILQP to the binary version, how many components that touched, and how many people we had working on that. In contrast, I implemented this basic version of the end-to-end quoting in a couple of hours yesterday and today. It needed one small change to the connector and the rest was completely localized to the My reading of the Internet's end-to-end principle is basically "anything that can be end-to-end should be -- for tons of reasons". I wasn't sure we should do this until I a) was arguing against making quoting end-to-end in the discussion with @justmoon and ended up convincing myself it was the way to go and b) actually implementing it and realizing how quick and easy it was to get a basic version working. |
|
I absolutely agree with the idea of end to end quoting and that quoting is an application layer concern. This is illustrated clearly by the fact that an ILQP quote can be routed differently to a payment and a quote is non-binding anyway. Personally I think it is very unlikely that quoting will be done at the ILP layer as the network grows. Finding the route for a payment (and therefor the cost to send it) is something that will be best served by data services with a good view of the network. A sender will get a route, required expiry and cost from such a service and then dispatch the payment with appropriate parameters. In the absence of such a service I think a very simple end-to-end protocol is useful but not essential. My suggestion would be to have a simple end-to-end standard that everyone SHOULD implement using the existing Wrt to this specific implementation of an e2e protocol, I don't like the hacky nature of it with predefined constants, special behaviour and overloading of functions for other purposes. As a side note, I wouldn't get bogged down by being backwards compatible right now. We have very few implementations in the wild that are doing proper quoting anyway. Why not simply drop support for liquidity curves from the existing connectors and see how we go. Next task will be to define a way of expressing a route to a sender that they can pass with their initial transfer to provide the first connector with preferred routing data. |
|
@adrianhopebailie just to clarify: I think you're using the phrase "end-to-end" in two very different ways. When you say quoting is an end-to-end concern, do you mean that a) it is strictly the responsibility of the sender and receiver (the ends) or b) that it is the concern of everyone in the path?
If all the intermediaries should implement it, it is not "end-to-end" in the way that term is used in the Internet stack. For example, saying "encryption should be end-to-end" means that it's between only the sender and receiver and the intermediaries have nothing to do with it. Same thing with reliability, etc. ILQPv1 is not end-to-end, v2 is.
When I mentioned being backwards compatible I meant that v2.x or even v3+ implementations could keep supporting v2, not about being backwards compatible with v1. Supporting multiple versions is easier if it's end-to-end because then you're only talking about the sender and receiver needing to support the same (maybe old) version, rather than the entirety of the network.
The difference is not just about liquidity curves, it's about whether connectors need to implement anything related to quoting. If we can make it work without that, it means half as many protocols for connectors to support. |
|
|
||
| This details the steps for a sender to get a quote for a fixed source amount ("if I send this much, how much will the receiver get?"): | ||
|
|
||
| 1. The sender constructs an ILP packet with the receiver's ILP Address, an amount equal to the maximum value for unsigned 64-bit integers (18446744073709551615), and no data. |
There was a problem hiding this comment.
would it be theoretically possible that the money actually gets reserved or the receiver redeems?
There was a problem hiding this comment.
Nope. Connectors use the transfer amount, not the amount in the ILP packet to determine how much they're going to put on hold. And by using a special hash that no one knows the preimage for, it's impossible for the payment to be redeemed.
There was a problem hiding this comment.
I would use 0 for the amount. See #312.
| The motivations for replacing ILQPv1 are as follows: | ||
|
|
||
| 1. **Simplification of the Interledger Layer and Connector** - Previously, all Interledger connectors needed to implement both the Interledger Protocol and the Interledger Quoting Protocol in order to be fully compatible with the protocol. By making quoting an end-to-end concern, this simplifies connector implementations and what we think of as the Interledger Layer of the stack. | ||
| 2. **Futility of Relying on Connector Quotes** - ILQP quotes were never intended to be binding, but rather to give senders a sense of the expected cost of a payment. Aside from the problems posed by malicious connectors, there were other reasons that quotes could not be relied upon. For example, exchange rates and connectors' liquidity positions are likely to change over time, rendering even honest quotes untrustworthy. |
There was a problem hiding this comment.
I agree, if the quote is not binding, it's merely an information poll.
I do wonder if it's possible to catch fluctuations between package prices?
For example, a chunked payment: <chunk1>, <chunk2>, ..., <chunkN>
Ideally the cost is somewhat bounded... but if no limits are set a malicious connector could suddenly bump the price during a stream.
These or other protections could go on the app level? Which implies that the receiver would be honest to te sender (and not accept the bumped price over the mutually agreed limit). Otherwise, one could put limits inside of the package?
There was a problem hiding this comment.
This is definitely a tough part about doing streaming payments. We'd have to build this into the streaming payments transport protocol, but there are limits to what it can do. It would probably be a good idea to have the receiver reject the incoming payment if the amount that arrives is less than expected, because that would disincentivize connectors from deliberately changing their rates in the middle of the stream.
However, you'd also need to have logic related to the specific application about whether you would switch provider or just stop if it gets too expensive. For example, if you're streaming a movie, maybe you could switch to a different host if one gets too expensive (whether it's the receiver's fault or some connector doesn't matter to you). If you're doing a streaming (chunked) payment, there might not be that much recourse. If it gets sufficiently expensive you might just want to stop and hope the receiver sends you your money back 馃槙
|
@michielbdejong This wouldn't affect BTP at all. It might mean that we don't need to use the messaging functionality right now but I wouldn't suggest taking that out. Might be useful for other things aside from quoting. |
|
@emschwartz sure, we never explicitly froze the ILP packet. But this change would make all current software look old-fashioned, and that is something which we also did as recently as 10 days ago. I would propose we keep version 1 of ILQP as "recommended" instead of "deprecated" for at least another 6 weeks, and preferably even for another 6 months. |
|
@michielbdejong This doesn't change the ILP (payment) packet, it just means we don't need some of the other packet types in there. If they're there and not being used, that doesn't hurt anyone. Also, which software do you mean exactly by "all current software"? Users of the I know you're eager to stop changing things. We've talked about this before but I see it slightly differently. I'm eager to get to the point where there's nothing left to change. The way I see the stack is as follows: On the ledger layer, we have a clear way to support multiple protocols so it's more important to clearly define the requirements for that layer than agree on one protocol for it. There are some benefits to having lots of people use the same protocol, so we've got BTP for that. The Interledger layer is the part I expect will be as difficult a change as IPv6. As soon as people start using Interledger in the wild it'll take us 20 years, not 6 weeks or months to change this. I thought this layer was as simple as we could get it (maybe with the exception of ILQP liquidity curve quotes), until @justmoon pointed out that we could halve the number of protocols involved at this layer. With this change there is only 1 protocol that we need everyone to support, and that's ILP. That hasn't changed in a long while and I feel confident that each of the 3 fields in there serves an important purpose. The fact that there isn't anything left to take out makes me confident that this is pretty damn close to finished. (The one thing we might later regret not adding when we had the chance is the ability to pass back data with the fulfillment, though as @sharafian pointed out: we haven't needed it thus far.) Finally, there are the Application and Transport layers. I'm actually excited for us to not set these in stone, but rather to let developers experiment with new protocols on top of ILP. Anything that we can make a concern of these layers, great, because different applications can choose whatever protocols and upgrade schedules they want. (This is why I got super excited about this idea when @justmoon mentioned end-to-end quoting) |
|
I'm also thinking about the reimplementations of ILP in other programming languages and I'm sensitive to @adrianhopebailie's point from this discussion about not wanting to spend time implementing ILQP from scratch if it won't be needed in the long run. |
|
|
||
| ### Fixed Destination Amount | ||
|
|
||
| The steps are the same as for Fixed Source Amount quotes, except that the transfer amount is set to an arbitrary probe value. When the sender gets the error response from the receiver, they compute the exchange rate based on the amount they sent and the amount that arrived. The sender then divides the fixed destination amount from the quote request by that exchange rate. |
There was a problem hiding this comment.
This the e2e implementation of quote-by-destination makes the assumption that all curves are simple rates (i.e. they only have 2 points, [0,0] and [x,y]).
There was a problem hiding this comment.
Yup. That's definitely one of the debatable points about this proposal. I would argue that's a good simplification to make for a number of reasons (note I'm partially trying to channel @justmoon's arguments here):
- If paths have a relatively small Maximum Transfer Size (MTS), you wouldn't need the rates to change much - Whether we like it or not, there will be a maximum size for an individual payment that the network will support. If I had to guess, I'd imagine that it would be on the order of $10-100, not $10k. Between the Interledger Universal mode risk and liquidity issues, I think connectors will be incentivized to keep the MTS relatively small. If that's the case, there's less of a reason to vary the rate based on the size. Do you really care about charging a different exchange rate for a payment of $0.01 as compared with $10? You might care more if the MTS was $100 million, but I think that's unrealistic.
- Curves don't help for streaming payments - If the MTS is smaller than the payments some people want to make, we're going to need to chunk those payments up. If you do streaming payments, then you really want to know liquidity information over time, not just a static snapshot. Attempting to express that would be so complicated it's not worth it. But if you don't do that, the snapshot curve doesn't help because it could change anyway. You just need to start sending, constantly monitor the rate, and adjust who you're sending through if the rate changes a lot.
- Assuming "curves" are just lines simplifies routing a lot - No matter what, routing money is probably going to be more complex than routing IP packets, which is already very complex. If you have to route packets in different directions based on the size, that means you need to keep multiple routing tables for different payment sizes. That would make it much more complex.
- We didn't use curves in practice - When we were running the community network of
ilp-kits, everyone was just using a % spread on top of a linear exchange rate pulled from fixer.io. It doesn't seem like you really need them to be much more complicated than that -- and @justmoon has made the good point that connectors may not charge customers on a per-payment basis anyway. - Simplification - if there's any way we can make do without liquidity curves, then we should do without them. They'll go on the long list of things we've spent loads of time on and then axed because they weren't strictly needed (nested transfers / source routing, multi-credit multi-debit transfers, fees in transfers, split payment paths, atomic mode, crypto condition-based payments, and now maybe ILQP).
...these points should probably go in the RFC. Thanks @sentientwaffle for asking the question!
|
@emschwartz asked:
I meant a). A quote is a hint at the amount and expiry to use when sending a payment. Where that Note also that I proposed support for the Over time I expect that the sender will determine how much to send and what timeout to use based on heuristics and eternal data services, until that evolves properly ILQP will be useful.
I'm suggesting they don't HAVE TO but until there is another way for senders to be aware of their available liquidity and rates they SHOULD. Implementing support for the |
|
Gotcha.
That is what this RFC proposes. Suggestions welcome on how to make that more explicit!
I don't think ILQPv1 works well unless you can make the assumption that all connectors support it. If I'm a connector and you ask me for a quote and I don't have a route (from a broadcast or some cached information), how do I answer your query? I would want to ask the next connector for a quote. If one connector along the path doesn't support ILQPv1 then everyone is going to have to answer "sorry, don't know", which would make for a pretty bad experience. That's why I think it's kind of an all or nothing question. Either quoting is on the ILP layer and you can assume every connector will speak it, or it's not and you don't assume that any connector will speak it.
It's not that much extra but it is extra. You need to listen for incoming quote requests and do slightly different things depending on whether it's a source quote or destination quote. Then if you don't want to remote quote everything you need a cache of other connectors' rates. That means you need to do cache invalidation, etc. If you're only forwarding payments, you need a routing table that says your next hop for every destination prefix and you need your local rates. No cache necessary. Also, connectors need to make sure they aren't DoS'ed with free requests like quotes. You might need to split out that logic into a different service. So there are definitely some extra things if the connector does quoting as well. If there are only payments, everything goes through one path and you only need one set of mitigations to deal with someone failing payments a lot. |
I don't think it does. It goes half way but then instead of removing the quoting protocol it has a janky way to use payments that are guaranteed to fail as a way to get a quote. That's just another quoting protocol. I don't think we will need a quoting protocol at the ILP layer in the long term, but it's useful to have something as the network grows. All that something needs to to offer is a way for a node to be able to query a peer "How much must I transfer to you and with what expiry to deliver X payment?" or "If I send you X how much will be delivered to Y, and what expiry should I use on the transfer?" In reality this is a host-to-host communication (unless it also carries a query about application layer fees that MUST get to the receiver, but that's a different issue and is probably best done at the application layer). Most importantly, it doesn't matter how the peer gets the data to respond to the query, it could be cache or it could ask the next peer. It should also be valid for the peer to respond and say: "I don't know". In this case the sender has two choices:
I think of quoting as simply a query service. In this case the service is being provided by nodes to their peers. I suggest we define a simple protocol for making those queries from your peers but recognize that there will be other protocols in future for making those queries from other sources. This could easily be a BTP sub-protocol.
That's not important. You can respond with an error. We need to consider the evolution of the network here. Initially we will likely have very few hops in a payment so a simple protocol relayed between peers is probably sufficient (let's define a standard one, freeze it for a while and see if it is sufficient). Over time separate services that provide routing/quoting information will emerge that define their own APIs. Nodes will query those APIs for data, either as the sender or as the an intermediary. |
This proposal removes quoting from the Interledger Layer. It makes it an Application/Transport/End-to-End concern.
This is the crux of the difference between ILQPv1 and v2. Do I ask my peer or do I ask the receiver?
Since I have no idea what your units are worth I would need to do a binary search with loads of payments that will fail in order to get it right. Is that better than using a single payment that I know will fail?
Who is that in practice? What if I only have a relationship with one connector? What if I'm trying to watch a video right now and I don't have time to rethink who my provider is? A big advantage I see of ILQPv2 is that there are no external dependencies and you only need the network to support the same functionality required to actually deliver a payment. We don't have to wait for some rate services to pop up in the future, and we don't need anything special from connectors. Also, at the same time you're asking the receiver for the quote details you're checking whether they're online and listening. |
I think this is the same type of janky as building email on top of FTP (how it was originally created), and that's a type of innovation I would actually love to encourage! |
No it doesn't, quoting is now done through failed payments, which are sent through ILP. If it was an end-to-end concern in the way you are suggesting then there would be no need to send it via any of the intermediaries. You could just get the quote during setup using a direct connection to the receiver. How is this any different to ILQP without curves?
But you aren't, you are asking your peer. The query is, "If I send you X how much gets to Y?" The amount you send your peer is one of the parameters of the query. That never gets to the receiver so this isn't really end-to-end at all.
The chances of this are almost 0. If I am making a payment to you I either; a) already know how much I have to send (the terms of the payment are defined in my units) or b) I already know what your units are worth.
Then that's a pretty poor provider if they can't route payments to an address you want to pay you'll probably stop using them.
Go for it! As I said, I think there will be multiple ways for senders to get a quote, you're just defining a new one. That said, I don't think you should call it ILQPv2 because it's not an evolution of ILQP. What this should be is a protocol between senders and receivers to predetermine that a payment will fail and put some specific data in the error. If connectors start explicitly supporting that it would be a violation of the end-to-end principal so I'd drop the constants and have a way for the sender and receiver to agree on the condition during setup. Why not rather define an application layer protocol that indicates the payment is actually a quote OR a setup protocol where the receiver specifies a second condition and ILP Address to send the quote payment to? |
According to this definition, HTTP isn't end-to-end because it sends IP packets through intermediary routers.
I am asking my peer to forward a payment for me, just as I do for any other Interledger payment. They can be completely oblivious to its purpose. I am really asking the receiver to do something out of the ordinary, which is to reject the payment with a specific error message.
Let's take the case of HTTP-ILP. You go to a website, they give you an ILP address, a PSK secret, and an amount they want you to pay. How exactly do you know what those units are worth? How do you decide on the source amount?
This is like ISPs handling traffic differently for video streaming versus other things. It is a violation of the end-to-end principle but it's also inevitable.
That's how I would propose using this protocol. SPSP would say "we'll either expect transfers to come in looking like the normal SPSP payments or ILQPv2 quotes". The only difference with what you're proposing is that it requires end-to-end communication of a condition instead of using a well-known one, which seems pretty minor (unless there are concerns about ILSPs blocking transfers with this condition). |
No, because ILQPv2 uses data sent at the ILP layer (the transfer amount at each hop). What you're suggesting is something like trying to do path MTU discovery but with some weird dependancies on the application layer protocol.
I am proposing that they MUST be oblivious.
Only because it's possible. We shouldn't design this protocol in a way that allows this. The risk is that this becomes a common way of getting quotes and then if we want to change it we have to be sure we aren't going to break any connectors that have done this. Summary: I can see the value of using a failed payment to probe rates along a route but the current proposal has too many foot-guns for intermediaries. Such a protocol should be properly end-to-end and have nothing that would induce different behavior from intermediaries vs a regular payment. |
|
What are you talking about? Did you see the change I needed to make to the connector to support this? |
|
I don't think you fully understood my comment. It has nothing to do with the amount of effort required to implement your proposal. My concern is with the things you haven't implemented but have said in the spec connectors could implement, like depending on magic amounts or conditions as an indicator that they should behave differently. As a separate concern, I don't think we can assume all connectors will or should do this:
That is basically saying we expect connectors to ignore the amount in the ILP packet which I think is a very bad assumption to make |
Could you be more specific about what the concern is? What exactly is the harm or problem you are envisioning?
Whether or not we like it, people will always be able to send payments through the network with unfulfillable conditions (for example: Connectors have two options: either try to block all known unfulfillable conditions or add some special logic for handling them. Trying to block them is futile because the sender can just as easily generate a random condition that the receiver doesn't know the fulfillment for. It's nicer for everyone if such conditions are well-known and connectors can treat them separately if they so choose (like not actually putting funds on hold if they recognize the condition as unfulfillable).
This comment made me think of a couple of other problems that can arise from connectors trying to deliver the exact amount in the packet. I wrote up #311 as a standalone proposal for special-casing the max uint64 value. Curious to hear your thoughts. |
|
This proposal uses a special condition to influence receiver behavior. I think it would be cleaner to make the quoting protocol part of the transport layer. So PSK would use ILQP for quoting. Consequently, a PSK receiver would know how to handle ILQP quote requests. Generally, the way an ILP payment should be structured is that:
So for a quoting payment, I would use: {
"address": "example.myledger.bob",
"data": "ILQP 2.0\n",
"condition": "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
}Ledger plugins are welcome to treat the "NULL" condition by not actually putting money on hold. So functionally this is equivalent to the proposal here. |
|
I'm confused, I'm probably missing something here. How would the I guess the condition is then either the one that PSK would derive from the packet, or null? And address would be equal to packet.account? |
Let's say you had a bunch of XRP and wanted You would send an ILP packet with the following fields:
Your transfer would use a condition of In response, you would get a rejection that contains the amount that the recipient got. So if I sent 40 XRP and the receiver says they got 10 USD, I would know the rate was 4:1. So if my actual payment had a source_amount of 200 XRP, I would know the correct destination amount was 50 USD. You would then make an ILP packet with the following fields:
The condition is the condition defined by PSK and the local transfer would be for XRP 200. |
|
|
||
| ILQPv2 uses payments sent with the Interledger Protocol. In order for this protocol to work, Interledger connectors MUST NOT reject payments where the amount in the ILP Packet exceeds what the connector thinks should be delivered to the receiver based on the accompanying transfer amount. Connectors SHOULD forward the payment and apply their normal exchange rate. Connectors MUST also relay error messages back to the sender. | ||
|
|
||
| ILQPv2 is intended to be used alongside Transport Layer protocols. It uses a transfer condition with a special prefix (`ffffffffffffff` in hexadecimal or `__________` in base64url encoding). Other end-to-end protocols SHOULD ignore to avoid interfering with ILQPv2 and other protocols that use well-known conditions. |
There was a problem hiding this comment.
Use null condition instead (AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA).
|
|
||
| ILQPv2 test payments use the condition: | ||
|
|
||
| `fffffffffffffff8a5aa9fefdbed3fffffffffffffffffffffffffffffffffff`, encoded as hexadecimal. |
There was a problem hiding this comment.
Condition is AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA.
Add a section to say that data MUST start with ILQP 2.0\n or some other identifier like that.
|
|
||
| This details the steps for a sender to get a quote for a fixed source amount ("if I send this much, how much will the receiver get?"): | ||
|
|
||
| 1. The sender constructs an ILP packet with the receiver's ILP Address, an amount equal to the maximum value for unsigned 64-bit integers (18446744073709551615), and no data. |
There was a problem hiding this comment.
I would use 0 for the amount. See #312.
|
|
||
| ## Open Questions | ||
|
|
||
| 1. Should this be a standalone protocol or should it be built into Transport Layer protocols like PSK? One advantage of building it into something like PSK is that in that case you could use the shared secret to authenticate and encrypt the quote message. |
There was a problem hiding this comment.
Into transport layer protocols. Regardless of the question whether encryption is necessary / desirable, the address defines the receiver and the receiver implements the transport layer, so if you want to quote the same address (which I think is highly desirable), it's up to the transport layer to say: "If data == ILQP, treat as a quote request".
Encryption + integrity could be used to make quote payments harder to distinguish from other types of payments. (Note that obviously is a totally different design direction than having an unfulfillable condition.) I don't think this is strictly necessary, so we should leave it out for now.
| ## Open Questions | ||
|
|
||
| 1. Should this be a standalone protocol or should it be built into Transport Layer protocols like PSK? One advantage of building it into something like PSK is that in that case you could use the shared secret to authenticate and encrypt the quote message. | ||
| 2. Will senders want to hide the fact that they are requesting a quote from connectors (for example by using data in the ILP Packet as opposed to a well-known condition to indicate that the payment meant as a quote request)? Connectors may have good reasons (such as DoS prevention) for wanting to avoid having lots of failed payments going through them. On the other hand, using a known condition could allow connectors that recognize this protocol to avoid placing funds on hold. |
There was a problem hiding this comment.
I don't think hiding the fact that you're quoting is necessary. It would be quite involved to truly make it impossible to tell that you're quoting. For now, it's probably good enough to detect if connectors treat quotes differently and depeer if a connector is acting badly.
|
|
||
| 1. Should this be a standalone protocol or should it be built into Transport Layer protocols like PSK? One advantage of building it into something like PSK is that in that case you could use the shared secret to authenticate and encrypt the quote message. | ||
| 2. Will senders want to hide the fact that they are requesting a quote from connectors (for example by using data in the ILP Packet as opposed to a well-known condition to indicate that the payment meant as a quote request)? Connectors may have good reasons (such as DoS prevention) for wanting to avoid having lots of failed payments going through them. On the other hand, using a known condition could allow connectors that recognize this protocol to avoid placing funds on hold. | ||
| 3. Should we try to figure out quoting for streaming payments now or is this fine for the moment? |
There was a problem hiding this comment.
Quoting for streaming payments would be similar, except you would probably not mind fulfilling the quoting payment itself. We would almost certainly want to support returning data with a fulfillment in order to provide feedback about the payment stream.
| 1. Should this be a standalone protocol or should it be built into Transport Layer protocols like PSK? One advantage of building it into something like PSK is that in that case you could use the shared secret to authenticate and encrypt the quote message. | ||
| 2. Will senders want to hide the fact that they are requesting a quote from connectors (for example by using data in the ILP Packet as opposed to a well-known condition to indicate that the payment meant as a quote request)? Connectors may have good reasons (such as DoS prevention) for wanting to avoid having lots of failed payments going through them. On the other hand, using a known condition could allow connectors that recognize this protocol to avoid placing funds on hold. | ||
| 3. Should we try to figure out quoting for streaming payments now or is this fine for the moment? | ||
| 4. How should the sender determine the expiry duration for the transfer they send? How likely are we to run into very slow ledgers? Should they have logic for increasing the timeout if necessary? They might not want to put the transfer on hold for too long if their ledger actually requires them to put their funds on hold (as opposed to recognizing that this is meant just as a quote request and won't be fulfilled). |
There was a problem hiding this comment.
The sender can observe how long the quote request took. In the case of Ethereum and XRP Escrow the sender would observe that the quote request took 15 seconds or 3.7 seconds respectively. It could assume that the fulfillment path takes about as long as the preparation path and submit the payment with a timeout of 2.5x the time that the quote took.
I'm not too concerned about supporting escrow plugins because I think we will quickly evolve towards fast ledgers / payment channels. At that point we may move to having a constant timeout (e.g. 60 seconds) most of the time. (Interplanetary ILP will be a fun project, but perhaps a bit premature to design around that use case right now.)
There was a problem hiding this comment.
I meant how long they should set the timeout for the quote request. If you set the expiry too soon the payment will timeout before it gets to the receiver. Do you send another one automatically with a higher expiry? What's the max you should set (/ what's the default we should set in the ilp module)?
|
|
||
| ### Error Message Format | ||
|
|
||
| **TODO: Define a binary format** |
There was a problem hiding this comment.
First attempt:
IlqpResponse ::= SEQUENCE {
destinationAmount UInt64 -- Amount that arrived at the destination
}| - Error code: either `F08` (which is currently unused) or `F99: Application Error` | ||
| - Error name: `ILQPv2` | ||
| - Amount that arrived in the final transfer to the receiver | ||
| - Milliseconds between the final transfer expiry and when the receiver processed the notification |
There was a problem hiding this comment.
Why do we care about that number?
There was a problem hiding this comment.
I put it in initially because I was trying to keep the destinationExpiryDuration feature. But isn't necessary if you assume that the time the receiver needs to fulfill and reject are the same, so the fact that they had enough time to reject means they'll have enough time to fulfill.
This is an attempt to write up an idea proposed by @justmoon to replace ILQP with an end-to-end quoting protocol. IMHO, @justmoon brilliantly identified that quoting was an application-specific concern that would need to work differently depending on whether you are doing streaming micropayments or individual larger payments. He is due all the credit for having this idea. I take responsibility for any bad or questionable details contained in this proposal. The protocol described here is implemented in the ILP module here: https://github.com/interledgerjs/ilp/compare/es-e2e-quoting It requires one minor change to the JS connector implementation: https://github.com/interledgerjs/ilp-connector/compare/es-e2e-quoting
also include explanation of linear exchange rates incorporates feedback from @justmoon and @sentientwaffle
| @@ -24,6 +24,8 @@ The motivations for replacing ILQPv1 are as follows: | |||
| 3. **Recognition of Quoting as an Application-Specific Concern** - Given the unreliability of quotes, different applications will need to handle fluctuating rates and liquidity differently. For example, the mechanism used to judge the cost of a stream of payments versus a single payment must actively take into account the changing rates. This suggests that different applications may need different types of quoting functionality or may not use quoting at all. | |||
| 4. **Enabling Evolution of the Quoting Protocol** - Protocols on the Interledger Layer must be implemented by all participants and will thus be more difficult to update. Taking quoting out of the realm of connectors will enable different quoting protocols to evolve over time. | |||
|
|
|||
| Note that ILQPv2 assumes all connector exchange rates are linear, rather than using Liquidity Curves like ILQPv1. See [Appendix A: Linear Exchange Rates](#appendix-a-linear-exchange-rates) for details. | |||
There was a problem hiding this comment.
Probably worth taking a moment to stop and clarify that we're not just assuming variable fees are linear, but also the absence of fixed fees. (I think that's an ok assumption, we can have fixed fees be deducted without being represented in the amounts OR even charge a monthly fee instead of fixed fees per payment.)
| 2. Will senders want to hide the fact that they are requesting a quote from connectors (for example by using data in the ILP Packet as opposed to a well-known condition to indicate that the payment meant as a quote request)? Connectors may have good reasons (such as DoS prevention) for wanting to avoid having lots of failed payments going through them. On the other hand, using a known condition could allow connectors that recognize this protocol to avoid placing funds on hold. | ||
| 3. Should we try to figure out quoting for streaming payments now or is this fine for the moment? | ||
| 4. How should the sender determine the expiry duration for the transfer they send? How likely are we to run into very slow ledgers? Should they have logic for increasing the timeout if necessary? They might not want to put the transfer on hold for too long if their ledger actually requires them to put their funds on hold (as opposed to recognizing that this is meant just as a quote request and won't be fulfilled). | ||
| 1. **If paths have a relatively small Maximum Transfer Size (MTS), you wouldn't need the rates to change much** - Whether we like it or not, there will be a maximum size for an individual payment that the network will support. Between the Interledger Universal mode risk and liquidity issues, connectors will likely be incentivized to keep the MTS relatively small. If that's the case, there is less of a reason to vary the rate based on the size. Charging a different rate for a payment of $0.01 versus $10 makes less sense than charging a different rate for a payment of $100 million. |
There was a problem hiding this comment.
you wouldn't need the rates to change much
Colloquial tone. Who is "you"?
| 2. **Curves don't help for streaming payments** - If the MTS is smaller than the payments some people want to make, streaming payments will be required to chunk those payments up. In the case of streaming payments, the sender needs to understand the liquidity information over time, rather than just a static snapshot. Attempting to express liquidity over time would be too complicated to be worth it. But without that dynamic view, the snapshot curve does not help because it could change anyway. Therefore, in the case of streaming payments, senders will need to start sending, constantly monitor the rate, and adjust who they are sending through if the rate changes a lot. | ||
| 3. **Assuming liquidity curves are just lines significantly simplifies routing** - Routing money is more complex than routing IP packets, which is already very complex. If connectors needed to route packets in different directions based on the payment size, connectors would need to maintain multiple routing tables for different sizes. This would make routing even more complex. | ||
| 4. **We didn't use curves in practice** - When we were running the community network of ilp-kits, everyone was just using a % spread on top of a linear exchange rate pulled from fixer.io. It seems unlikely that connectors will need more complicated fee structures than that, and connectors may end up charging customers on monthly or other bases rather than per-payment. | ||
| 5. **Simplification** - if there's any way we can make do without liquidity curves, then we should do without them. They will go on the long list of things we have spent loads of time on and then axed because they weren't strictly needed (nested transfers / source routing, multi-credit multi-debit transfers, fees in transfers, split payment paths, atomic mode, crypto condition-based payments, and now maybe ILQP). |
There was a problem hiding this comment.
Tone. "make do", "loads of time"
|
During your summary at the community call just now, I realized that quoting has the specific goal of giving the sender the option to opt-out of extortionate exchange rates. The sender checks the quote before saying yes to it. Currently, the quote is produced to your ILSP, they tell you either "look at our supercheap rates" or "it's really that expensive". Making quoting end-to-end creates a distance between your ILSP's choice of route, each connector's choice of commission fee, and the receiver's decision to fulfill. It would, for instance, become easy for a receiver to say you have to pay more, and pretend they only got the amount from the packet. Same for all connectors on the path. So as a sender I would just stick to using ILQP v1, because it gives me a way know I'm not paying extortionate fees to anybody except my own ILSP (the first connector). So with ILQP v1, as a sender I have the power to choose a cheap ILSP and not pay too much. With ILQP v2 the sender would lose that power, right? |
Incorrect. With ILQPv1 you always need to think about the quote as end-to-end. Let's say I'm the malicious receiver you describe. I have an account on some obscure ledger that no one knows the rates to, and I run the connector that gets to me. No one else in the network knows how much my units are worth so they will need to fall back to end-to-end quoting. When they ask my connector for a quote, I give them a very distorted rate. So no matter how much the sender's ILSP tries to help out, it's always possible for the receiver to mess with the exchange rate to them. (This is why you can only trust your units, never the destination units. You should always treat them as opaque) Also, because you always need to be able to fall back to end-to-end quoting with ILQPv1, you need to think of it as primarily an end-to-end mechanism that can be optimized with local caches or broadcasted rates in some but not all cases. With ILQPv2 you use the same path the payment is going to take to figure out how much that path is going to deliver. That doesn't expose you to any more exorbitant rates than using ILQPv1 to explicitly ask all of the intermediaries in the path for that same information. |
|
I'd also add that there's an argument to be made that the alternate Encrypted Quotes proposal would actually do more to keep the quotes in line with the actual payment rates. Arguably if the connectors can't tell whether a payment is meant for a quote or is for real they would have to give their normal rate. The reason I didn't go for that is that I'm not sure you could actually hide the fact that you're doing an end-to-end quote in practice so it might just be better to let connectors directly handle that (with the known unfulfillable condition). |
incorporates feedback from @justmoon and @stevenroose
That just means the 'distorted' rate is the real rate for getting to that ledger.
I think I don't agree at all with that, but maybe I don't properly understand what it means to treat a unit as opaque. I do agree that if I don't understand the destination currency, then quote by source amount is useless. But if I'm required to pay a bill in destination ledger units, then I want to minimize source amount, given a destination amount.
Not what you want, you want the network to route around high fees. |
There was a problem hiding this comment.
Although I agree that end-to-end quoting may not be worse than remote quoting, I think it's definitely worse than the cost-based routing which is currently implemented in the community network. If people would stop using ILQP1 in favor of ILQP2, that would remove the incentive for connectors to charge low fees, and I think that's not what we want.
Right.
I mean you should always judge prices in your source currency. Agreed that you may need to pay a bill in a destination currency, but you shouldn't try to understand whether it's a good price or not except by converting to your units.
How does it help you if the network routes around high fees for answering quote requests but not for payments? You want the quote to tell you how much the route the actual payment is going to cost. How the connector determines where to forward your payment is a separate concern.
How does it change the incentive? The incentive is there if senders and connectors have multiple options. If you're a connector and I'm routing payments through you and someone else, I can go through whoever has the better rates no matter how I'm determining the rate, no? |
|
After discussing this more with @justmoon, we concluded that it makes more sense to build another full transport layer protocol that handles things like chunked payments and end-to-end quoting together, rather than having end-to-end quoting as a standalone protocol in the form of ILQPv2. I'm closing this PR now and I'll replace it later with a proposal for another transport layer protocol that may include some of these ideas. |
This is an attempt to write up an idea proposed by @justmoon to replace ILQP with an end-to-end quoting protocol. This simplifies the Interledger architecture by removing one of the two protocols that connectors need to implement (:exploding_head:).
IMHO, @justmoon brilliantly identified that quoting was an application-specific concern that would need to work differently depending on whether you are doing streaming micropayments or individual larger payments. He is due all the credit for having this idea. I take responsibility for any bad or questionable details contained in this proposal.
The protocol described here is implemented in the ILP module here: https://github.com/interledgerjs/ilp/compare/es-e2e-quoting
It requires one minor change to the JS connector implementation: https://github.com/interledgerjs/ilp-connector/compare/es-e2e-quoting
Thoughts? 馃檲
Update: The only connector change required to support this is the same one proposed in #311.