Proposal: Include data with fulfillment #314
Comments
|
What's a connector's incentive to pass on the data with the fulfillment? And if a connector doesn't pass on the data with the fulfillment, is there any way for the sender to find that out? It seems like connectors would have an interest in not passing on data with the fulfillment when an end-to-end quote is being performed, because they would have to retry the end to end quote. They obviously don't want to do that all the time, but as much as they can without making the network traffic go down significantly |
|
Repeat dealings. If my streaming transport protocol relies on getting a message back and I don't, that's an error. I wouldn't send more payments through you and if this happens repeatedly I'd switch to using another connector that does work. That's another useful thing with streaming payments, is that it's never about a single payment. Would a connector really try to cheat me for a single $0.10 payment, or fail to pass on a couple kb of data, if that means they would miss out on the next $1000 that I'm about to send? Also, this behavior would be part of the expectation of what all connectors implement. If you don't implement this functionality, you haven't implemented ILP properly. |
If we consider this sufficient incentive for connectors to always behave correctly then we should be consistent in taking that position. That assumption makes the debate in #316 kind of meaningless. By extension of the assumption that connectors will always do what's most likely to earn them repeat business, we can assume they will always charge the least they viably can, assuming they have limited knowledge of what other connectors charge for the same route. Starting to feel like the use case for crypto-conditions is rearing its head again |
|
This would require a new (more generic) payment packet type, "payment with response", where each transfer is conditional not on the fulfillment of a sha256 condition, but on a signed message from the receiver. If we conclude sha256-based interledger is doomed without this, then we could switch to that, but otherwise, I think this would not be wise.
For the current payment packet, that's always just the packet amount, right? So I guess this new "payment with response" packet type would then also not have an amount in the packet. IMHO the network dynamics of that would be very different (more collaborative) from those of the current (more competitive) interledger design, where payments always have a well-defined destination amount.
Again, that's not useful for the current interledger design, where payment packets have exact and well-defined destination amounts. |
|
Sorry, I now realize I misinterpreted this proposal. I assumed we would want to do it "the secure way": change our stance of "interledger is a secure network for payments" to "interledger is a secure network for payments as well as (data) purchases". For that, the ledger would have to check a crypto condition, so that a transfer is only executed if the next connector delivers not only the fulfillment, but also the data. I now understand that the intention is to implement this "the insecure way", so interledger guarantees that the payment arrives within the timeout (or your money back), but it doesn't guarantee that the data that is passed back with the fulfillment is delivered. For including data with the fulfillment the secure way, you would need a crypto condition that checks not only the sha256 preimage, but also the receiver's signature of the data (or your money back). For doing it the insecure way, you can still do integrity checks on a higher layer, but if no data is passed back at all, the sender doesn't get what they hoped for, and also doesn't get their money back. |
I agree with @sharafian and I doubt that the argument of "Repeated dealings" is a strong enough incentive for connectors to pass on the data. Even worse, we can think of scenarios where a connector as an incentive to modify the data that is passed back to maximise his profit. If we want to send data back along with the fulfillment, this data needs to be part of the fulfillment, i.e. a connector needs to pass on the fulfillment+data to unlock the funds on hold for him. |
@justmoon, Why not communicate this information back via the application layer? To me, it seems more complex to communicate this information back via the ILP layer than the application layer. @justmoon makes the argument that "there isn't an obvious way for the SPSP server to communicate information back to the SPSP client" and adding some sort of bi-directional communication channel adds a lot of overhead. How about the sender, after receiving a fulfillment, initiates a request to the SPSP server to retrieve this information? |
This can be trivially solved by having the transport layer protocol use authenticated encryption. I'm having a bit of trouble seeing this issue from your perspective. I'm writing to you right now over the Internet, where the intermediaries never have the kind of incentive you're describing to pass on the data (there's no money they're claiming when they pass on individual packets). But they do it. Why? Because it's part of the protocol, and because they may have out-of-protocol agreements with their peers that include expectations of certain levels of service. If they don't pass on the data, they're not real Internet routers. Right now we have ILP errors that are supposed to be relayed back by connectors. You could make the argument that connectors don't have any incentive to do this. However, then we can't build protocols that rely on getting error messages back. Why would we limit the stack in this way? Why is it so hard to imagine that the protocol would just include expectations that you be able to return some response data or an error message back? If you are a connector that doesn't do that, you haven't fully implemented ILP and everyone should avoid dealing with you. |
|
Then why have hashlocks at all? Why not just include expectations that connectors either forward or reject payments in the forward direction, too? It seems a bit arbitrary that we implement guarantees in the forward direction, but then deem then unnecessary in the backward direction. |
True, assuming sender/receiver have a communication channel, like with PSK/SPSP, they can exchange keys and use authenticated encryption. However, if sender and receiver have such a communication channel, why not use this channel directly to exchange the data? It's much more efficient to exchange this data directly than over a series of ILP hops. I am not fundamentally opposing sending data back along with the fulfillment, but we should carefully consider which kind of data to send on the ILP-level and which data on the application level. Does the data concern ILP-components (e.g. connectors)? Great, put it on the ILP-level. Is the data only relevant to sender/receiver and no ILP-component ever looks on the data? Then put it in the application-level. Following that logic, things like how much was received in the last payment, how much is left to pay, etc. belong rather on the application-level than the ILP-level. Some more points to consider when sending data back on the ILP-level:
|
@michielbdejong Fair point. I think you could make a case for all of these options:
I'm coming from the perspective now of trying to enable more use cases to be built on top of ILP while minimizing disruption to our current stack and trying to avoid re-pulling in big dependencies like Crypto Conditions. I also think that if we make a transport protocol now that uses this feature and it is even somewhat widely used in the early network, the network would evolve such that all connectors end up supporting that feature. Response data without adding data on fulfillment path If the protocol doesn't support sending data back on the fulfillment path, transport protocols can achieve the same effect with unfulfillable payments. It's not ideal but it could work. If the sender wants to request data, they could send a payment that the receiver would reject with the data in the error message. If the receiver wants to push some data they could send a payment with the data in the ILP packet. I think this type of (unintended) use of the protocol is inevitable and ultimately good. So the question for us is whether we want to support protocol designers with features like fulfillment data or make them rely on janky workarounds.
Same as the ILP packet size limit.
Depends on the transport protocol. The prototype I'm designing only contains a single number, plus the overhead for encrypting it.
That assumes the communication method they have is connection-oriented and open for the full duration of the payments they're sending.
Maybe. I think we'll need to make connectors and the payments going through them super fast anyway. But the bigger point to me is that I think we should enable people to build other use cases and protocols on top of ILP, rather than limiting the entire stack to the small subset of things we can think of right now. It seems like an unnecessary limitation to say that because sending response data back through the same path is slower right now than using the internet we shouldn't allow you do do that at all. |
|
To respond to @adrianhopebailie and @michielbdejong's comments in #317:
That would unfortunately limit the data to information known before the payment started. In the case of PSK it would mean that the sender would need to know the data also, which would defeat the point of using it as a response from the receiver.
It would be hard for the sender to determine this if they don't have many connections but very easy for a connector to probe the network and figure out who was dropping the data. If the transport protocol is capable of automatically switching between connectors, it would probably be designed to stop sending traffic through whichever path is dropping data. That way, even if your outgoing connector isn't the one dropping data, they'd have a strong incentive to figure out who was and to get them to stop or stop doing business with them. |
Better to always send a signed data string with the fulfillment, even if that's the empty string; that way, even if there was no data to return, each connector can pass on proof that the receiver returned signed empty data. Even if the ledgers don't enforce the crypto condition for it, each connector, as well as the sender, can still see when they were tricked. |
Don't sign an empty string. Add a nonce. Otherwise, there is the risk of replay attacks. |
|
Yeah, nonce should be added, even for non-empty strings. |
I think this should be up to the transport protocol, but it's fine to make a recommendation like that. The main question at issue here is whether we should expose this as a feature for transport protocol designers. |
|
It does matter though, because you would also need a separate payment type that's "payment that expects not only a fulfillment but also data", so that for two peered connectors it's always clear if the service that was paid for (obtaining a fulfillment in one case, or obtaining fulfillment+data in the other) was actually delivered |
If we simply go back to crypto-conditions then I think that's all that would be needed. Transport protocols that want to do this can then use a condition that is fulfilled by a signature over the return data. If we go back to our reasons for dropping CC I believe our concern was too may nodes that may not support them because implementing them is hard and then we have a fragmented network. What if we say support for PREIMAGE-SHA256, RSA-SHA256 and ED25519-SHA256 are the only required types? (Maybe you want PREFIX-SHA256 too...) I appreciate that this means we can't then send payments over networks like Lightning but let's evaluate that before we throw this out completely. |
I love CryptoConditions. Full stop. But, having spent a fair amount of time on the java-crypto-conditions library, and interacting with other devs working on implementations in other languages, I don't think we should underestimate the potential wisdom of our choice to not support all CCs in default ILP. We should look for ways to use the other CC types in optional ways. |
@emschwartz Requiring a connection between sender and receiver is an end-to-end concern that removes the need for every party to forward the data. Given our move to make everything as end-to-end as possible, I'm not sure why we would dismiss this as a solution. Because of our need for features like refunds I think it's reasonable to assume that an end-to-end connection for the duration of a chunked/streaming payment will be useful. Also, moving the data transmission outside of the ILP flow is very good for robustness. If the ILP network is having issues or the route between sender and receiver has a downed link, an end-to-end communication channel lets sender and receiver coordinate whether to abort the payment or not. I'm still of the opinion that we should be sending as much data through the internet as we can get away with, and only use the ILP layer for what has to be there (i.e. payments and integrity data). |
I'm building refunds and I haven't yet come across the need for out of band communication.
If this happens with any regularity no one will ever want to use ILP. Why would we build for this assumption?
You really think it would be better if PSK2 required that you pass in a two-way authenticated and encrypted stream? How would you make that work with We have errors that are relayed back across the chain. I really don't think it's that crazy to argue that there should also be a way to relay back an affirmative response. As I said though, we can make do without it, I would just build it using unfulfillable payments (instead of requiring the out of band communication channel). |
|
After a discussion with @emschwartz we came down to this argument against fulfillment data: Because fulfillment data is not required for the payment to fulfill (unless crypto conditions were introduced, which would be a lot of implementation work), you can never fully rely on fulfillment data being transmitted back without trusting all intermediary connectors. Unlike a rejection message, it is not safe to retry a payment for the fulfillment data. After a rejection your balance is the same as it started, but you have paid money after a fulfillment. That means you need a fallback mechanism to get fulfillment data if the fulfillment data is not passed back. And if you need a fallback mechanism anyways, it would make more sense to use that in the first place rather than requiring everyone to implement two flows, and requiring all connectors to forward another field. Without fulfillment data, the options are:
|
|
It seems unnecessarily limiting for the protocol to be able to pass data on the forward path and backwards failure path, but not on the backwards success path. However, I was more or less convinced by @sharafian and @michielbdejong that if you can't rely on the data being there it's not that helpful (because you need the fallback anyway). Between having no fulfillment data and bringing back full on Crypto Conditions, I'd probably lean towards no fulfillment data but I'm not thrilled about that... 😞 |
|
@emschwartz If we don't introduce fulfillment data on the return path, any opinion about which of the alternative choices listed @sharafian you prefer? |
|
I think requiring an end-to-end, connection-oriented, real-time, authenticated and encrypted channel for a transport layer protocol to work is a no go. Right now I would prefer unfulfillable payments (maybe with the null condition) over a dedicated messaging function. The main reason is that using unfulfillable payments makes supporting messaging an optional feature for connectors, rather than another thing on the list that everyone MUST support. |
These aren't hard requirements. For example, an SPSP server (which already fills the other criteria) could simply have an endpoint to check the status of a chunked payment. After each chunk is fulfilled, the endpoint is queried to determine the size of the next payment. HTTP2 or HTTP with something like keep-alive could be used to make this more efficient without requiring a websocket connection. |
Why? Authentication and encryption come for free with HTTPS. Near real-time communication seems to be rather achievable via a direct connection than the ILP layer. If an ongoing connection for the duration of a payment is considered unfeasible, the sender polls the status of a payment. |
You also can't rely on errors being passed back, yet we still find passing error information useful. It's the same thing here. Sure, you might not always get the fulfillment data back, but that's still strictly better than never being able to get fulfillment data back.
That's not true. In a streaming/chunked payment the fulfillment data is primarily used to update the sender about the status of the stream. If I don't get the fulfillment data for some payment I'll keep going for a little bit. If I get fulfillment data a few packets later, great, I can keep going. If I don't, I'll kill the stream. Note that no connector stole any money from me and now they're out the money that they would have earned if I had kept streaming. So there is a strong incentive for connectors to correctly transmit the fulfillment data. Due to that incentive, fulfillment data will almost always be correctly transmitted and in the few cases ("cosmic rays") where it gets corrupted or lost, the impact is basically zero because the next packet is going to catch the sender up again.
Side note: That's the beauty of PSK, we get a free ride on authentication, key exchange and creation of a shared secret.
You don't just need real-time communication, you also need an interface between the application layer (where the real-time communication is happening) and the transport layer (which is what actually provides/requires the information about ongoing payment streams.) So instead of chunked/streaming payments being totally invisible to the application developer, they are now required to add a real-time channel to their application layer protocol (if that's even possible) and connect it to some API on the PSK module. For example in the W3C Payment Request API, you normally do something like this: const request = new PaymentRequest(
supportedPaymentMethods,
paymentDetails,
options
);
// Call when you wish to show the UI to the user.
request.show()
.then(...).catch(...);So you can pass some static details from the payee to the payer. For example, we might pass
And that's just one example, different application layer protocols may require totally different workarounds.
Polling tends to not have a very good user experience due to the delay. Implementing polling well is difficult - you have to choose an interval that doesn't put too much load on the server, yet is fast enough to keep the client responsive. And ideally, you'd adjust that interval if load is high. It's very silly to poll if there are already real-time notifications going from receiver to sender (fulfillments) and we can trivially include some data with them... And adding fulfillment data is a non-breaking change. (Old clients will simply ignore it.) This is one of those debates where I can't understand how this is even a debate. |
After an error, everyone's balances are in the state they started in, so it's always safe to retry. After a fulfillment, balances have changed and retrying will cause you to spend more money.
This is OK in a case like a streaming payment where you were going to send more anyways, but not if you're making a single-chunk payment or the last chunk in a chunked payment. If you weren't planning on sending a follow-up payment but are forced to because you didn't get your data back, you stand to lose money. We can always say that if connectors drop fulfillment data they'll be depeered, but if people do it intermittently it might not be easy. Depeering is a manual process and it's a pretty drastic measure; we can't expect people to drop long-standing business relationships overnight. And if depeering were a totally effective way of limiting bad behavior, we could have eschewed hashlocks altogether and just said any connectors that steal money will be depeered. There can be situations where there's an incentive to drop fulfillment data. If you can see the destination of a payment is a competitor, you can drop fulfillment data, for instance. This isn't the same as dropping a payment, because a dropped payment can be safely retried. A dropped fulfillment can never be safely retried; it's only recoverable if it happens on a payment in a stream where it isn't that last payment. There might be some workaround for the last-payment problem, and we could just say that the incentive to sabotage is outweighed by the punishment of depeering, but I don't think it's a no brainer |
This has come up many times, but we previously lacked a strong, clear use case for it.
When doing streaming payments, the receiver needs to communicate information back to the sender - how much was received in the last payment, how much is left to pay, etc.
We could say that this needs to be communicated via the application layer. But this seems to add a lot of overhead. Imagine an SPSP payment of $10000 dollars with a max ILP payment size of $100. There isn't an obvious way for the SPSP server to communicate information back to the SPSP client. We could add a websocket connection, but that would complicate our "simple" HTTP based protocol quite a bit. The fulfillment is already being communicated back to the sender. And the prepare and reject state transitions already carry data, so the obvious solution is to add data to the fulfillment as well.
The text was updated successfully, but these errors were encountered: