-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: introduce ledger-ledger lotocol #251
Conversation
Binary protocol to replace RPC API. Resolves #249. I did not pick the name. Created with @emschwartz and @sharafian.
It would be useful to have a way to return an Interledger Error in reponses and also some kind of response code. I assume that this model assumes the response code comes from the transport (e.g. HTTP, 200, 403 etc) but those codes are not really always appropriate unless were designing this API to use REST (which I don't recommend, I like the RPC pattern we've got). The transfer life-cycle also seems a bit odd. There is an async response to a transfer request if it is rejected but not if it is accepted. What is returned in the transfer response for a successful transfer request? |
asn1/LedgerLedgerLotocol.asn
Outdated
} | ||
|
||
LedgerLedgerLotocolPacket ::= SEQUENCE { | ||
-- One byte type ID |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe differentiate between request and response so that response can carry an optional error and a response code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are differentiated by each CallSet
item either being a request or a response. Note that ILP Errors count as IlpPacket
types
; | ||
|
||
SideProtocolData ::= SEQUENCE OF SEQUENCE { | ||
protocolName IA5String, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any examples of what this might be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We were talking with @jimmiebfulton and @sappenin about cases where you might want to encode rules related to a certain payment, like ledgers the payment should not be routed through. In the JS ledger plugin interface this kind of side protocol would be included in the transfer's custom
field. They brought up a good point that that information might be relevant to quoting as well. We figured we might as well put this type of extensibility in all of the messages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would also be useful for sending payment channel claims
p.s. I suggest the name Interledger RPC |
What's the difference between MessageRequest and CustomRequest? Both seem to have a similar role, of making the protocol extensible. For a ledger API, sendMessage makes sense (it's asking the ledger to send something to another account holder), but for RPC I don't think it's as logical to have. They both can be seen as a form of nested "RequestRequest", where you still have to specify what goes inside. As an illustration of this, you can see that the current So I would say extensibility can be done through the |
If we decide that this is the way forward, then:
|
; | ||
|
||
SideProtocolData ::= SEQUENCE OF SEQUENCE { | ||
protocolName IA5String, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We were talking with @jimmiebfulton and @sappenin about cases where you might want to encode rules related to a certain payment, like ledgers the payment should not be routed through. In the JS ledger plugin interface this kind of side protocol would be included in the transfer's custom
field. They brought up a good point that that information might be relevant to quoting as well. We figured we might as well put this type of extensibility in all of the messages.
asn1/LedgerLedgerLotocol.asn
Outdated
} | ||
|
||
LedgerLedgerLotocolPacket ::= SEQUENCE { | ||
-- One byte type ID |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are differentiated by each CallSet
item either being a request or a response. Note that ILP Errors count as IlpPacket
types
data CALL.&Type ({CallSet}{@type}) | ||
} | ||
|
||
END |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since there is no call to get currency code and scale, where should a ledger plugin for this protocol get that info from? We talked about getting that from some kind of webfinger lookup but does that mean the plugin for this protocol would depend on the webfinger lookup as well, or would those options have to be configured?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's reasonable to remove getInfo and getAccount. There is a lot more you also need to negotiate, like version of this protocol, version of underlying socket protocol, endpoint URL in each direction, and auth tokens. Also, it might be that currency info is only defined relatively (e.g. company stock).
@michielbdejong
This spec needs to be hashed out more, implemented as a plugin, and worked into ilp-kit before we talk about any deprecation schedules. We could also make it backwards compatible by enabling kits to peer over different plugins.
Not before an initial version is even hashed out, but my idea with proposing using a binary format in the first place was that I think this could become a fairly long-lived ledger-layer protocol. |
one method I'm missing is 'settle', where one party tells the other party that they think the trustline balance changed in their favor, and includes either proof of on-ledger (unconditional) payment, or a message referring to manual/out-of-band settlement, or an updated payment channel claim. it's currently in the fulfill response side-protocol-data, but I think it would be good to make it more explicit? Another would be 'setup', where ledger A tells ledger B its own RPC endpoint. Either that, or:
Just one more thought on backwards compatibility - we could also reduce this new protocol to an optional extension of the current protocol, where one peer can try a |
I think that would be a good use of the
That sounds a bit unsafe to me
The proposal is that this new protocol would go over raw (TCP/TLS) sockets or WebSockets instead of HTTP, so there's no waiting anyway.
Sounds like something that should be put in the routing protocol, not in the ledger layer protocol. If we make
👎 We should handle backwards compatibility on the ledger layer by making ILP Kit support different plugins better. Also, since this would go over raw sockets or WebSockets it wouldn't really make sense to combine the two. |
ok, so I think a prerequisite for adding this new protocol as a second way in which servers can peer with each other, is for servers to announce which peering method(s) they support. If quote requests, settlements, and route broadcasts are not defined by LLL, but all of these are needed for a working connector-to-connector peering, then a server would have to announce "I support LLL version A + ILQP version B + routing-protocol version C + settlements-protocol version D". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This protocol competes with ilp-kit's current peering protocol. Therefore, before this becomes an RFC, we need to implement a version check in ilp-kit's peering routine, where it can use this new ilp-plugin-LLL, or fall back to ilp-plugin-virtual if the peer doesn't support LLL. This version check could for instance be implemented using a WebFinger lookup between the two hosts.
@michielbdejong I almost agree with that, but not quite.
That is only true if connectors that use this assume all other connectors use it. Peering should include an option about what ledger protocol the connectors are going to use. ILP Kit should be changed to be more flexible and support multiple plugins for peers. |
ok, in any case, when you peer with someone, you want to know exactly what they support on all of those layers. we can start with adding an "LLL support" indication to ilp-kit's WebFinger announcement, and then make it use the correct plugin for each peer based on that. |
asn1/LedgerLedgerLotocol.asn
Outdated
} | ||
|
||
TransferRequest ::= SEQUENCE { | ||
transferId UInt128, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you could also use the condition as a unique id of the transfer?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i vaguely remember this was to allow friendly repeated attempts at the same payment, but now don't see why you couldn't do the same thing using just (repeated) condition.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Enforcing condition uniqueness would also prevent you from doing things like Optimistic over Universal (which might not be a good idea anyway)
} | ||
|
||
FulfillRequest ::= SEQUENCE { | ||
transferId UInt128, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or condition
} | ||
|
||
TransferStateRequest ::= SEQUENCE { | ||
transferId UInt128, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for this to work nicely, it does make sense, i guess, to have a separate id field
asn1/LedgerLedgerLotocol.asn
Outdated
} | ||
|
||
BalanceResponse ::= SEQUENCE { | ||
currentBalance VarInt, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should define more precisely, i think the most generic definition would be:
+
total amounts which you received and peer agrees you have fulfilled in time-
total amounts which you sent and peer claims to have fulfilled in time-
total amounts which you sent and peer claims to still be pending+
total amounts which peer agrees you sent on-ledger (unconditional/fulfilled/paychan-close)-
total amounts which peer claims you received on-ledger (unconditional/fulfilled/paychan-close)+
total of, for each open outgoing paychan, highest claim peer agrees you gave-
total of, for each open incoming paychan, highest claim peer says they gave you
So then:
- your max send limit is currentBalance-minBalance
- your max receive limit is maxBalance-currentBalance
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
another view is saying settlement can only occur through circular conditional payments to self, "into this trustline", so then this protocol can only be used for one-to-one trustlines, and you only need the first 3 items to describe the balance. Apart from LLL, we would still on-ledger escrow plugins as well as paychan plugins, but they would only be used for the settlement payments, and LLL would be used for the "main traffic".
-- One byte type ID | ||
type CALL.&typeId ({CallSet}), | ||
-- Allow multiple accounts to be multiplexed on one connection | ||
accountId UInt32, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does account mean "from" or "to", here? Or is it more "prefix"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's more prefix
accountId UInt32, | ||
-- Used to associate requests and corresponding responses | ||
-- If requestId = 0, the server MUST not send a response | ||
requestId UInt32, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doesn't this also duplicate the transferId?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They do slightly different things. I could make multiple calls, in which each request/response would have the same requestId
, regarding the same transfer. First I would prepare it, which has a request and response, then it would get fulfilled, which has another request/response, and then I could check the state, etc. requestId
is meant to correlate a specific call request with its response.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could make multiple calls, in which each request/response would have the same requestId, regarding the same transfer
This doesn't make sense. Do you mean different requestId and same transferId?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, sorry. Same transferId
One thing I'm missing in this protocol is a clear state machine definition of how the balance updates as a result of transfers getting finalized in the symmetric trustline case:
|
To be fair, this is just the data format. We also need an accompanying RFC to explain how its used.
Don't think anything here is stopping you from doing that.
I'm not sure this protocol would make any guarantees about the message order. That only causes an issue though if you're playing around very close to the balance limit and are taking action based on the balance numbers your peer is reporting, right?
This seems more complicated than necessary to me. As as user of this protocol I think I'd prefer to leave that out and just do some little reconciliation when necessary.
Does that need to be in the protocol (on the wire format) or would that just be an implementation feature? |
then we should maybe move BalanceRequest/BalanceResponse out of scope as well |
I think having some kind of |
What is the function of TransferResponse, FulfillResponse and RejectResponse? It would be more efficient to just have an incrementing RequestId for detecting when a message was lost, or maybe the underlying socket guarantees that it can detect and retry when messages get lost (for instance, a WebSocket can have latency and can be disconnected, but never drops messages). |
Acknowledging that the request was processed. I think making every request blocking (which doing this over TCP might do anyway) seems like it would make it much less efficient than if you just fire them off asynchronously. |
BalanceRequest/BalanceResponse is only useful in the asymmetric case (i.e. when only one peer remembers the trustline balance) if its meaning is strictly defined. In the symmetric case (when both peers remember their account of the trustline balance), you would want to know not only how much balance your peer is claiming, but also which events were taken into account while summing up the |
You know your TransferRequest was processed when you get back a FulfillRequest or a RejectRequest. |
Actually that's not totally true. Right now the LPI Maybe we'd want to rely on TCP for resending messages? Although I think there's also a strong argument for doing this whole protocol over UDP, especially with things like quote requests, because you don't want them to block other, more time-sensitive messages. |
I agree BalanceRequest/BalanceResponse can stay in, but only if we mathematically define the meaning of the Note that this gives the preparing party an opportunity to cheat: they can use the fulfillment, and then still tell the fulfiller that they were too late. When this happens, the fulfiller should temporarily lower their trust in their peer; if it happens repeatedly in one direction but never in the other direction, the trustline will eventually dry up, and this is how the fulfiller's exposure to this form of cheating is capped. If we cannot agree on what the mathematical meaning of |
Ok I'm convinced now that we should take out the balance request and response. There are at least three different uses of this protocol, which would all want something different from that call:
It seems crazy to me to take out the balance call from this protocol, but I think @michielbdejong is probably right that it's better to leave it out than have it defined differently by different implementations of this protocol. Since what is meant by "the balance" is necessarily different in different use cases, we should take it out of this spec. |
amount UInt64, | ||
condition UInt256, | ||
expiresAt Timestamp, | ||
packet InterledgerPacket, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instead of including an InterledgerPacket here, with an InterledgerProtocolPayment inside it, and a useless type
field that will always be 1
, we should maybe directly include the InterledgerProtocolPayment itself here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we should do that because there may be other versions of the InterledgerProtocolPayment packet later, and we could also decide to do things like sending quote requests on transfers.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, good point. I removed this from #256 (comment) (which I am now quite starting to like as I look at it again, actually).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I expressed my requested changes as a PR to this PR: #256
slightly different take on: #251 and #256 based on conversations with @michielbdejong, @sharafian, @justmoon By separating the transfer contents from whether they are part of a bilateral (e.g. trustline) or multilateral (e.g. centralized ledger) relationship, the transfer protocol becomes much more general and thus makes sense to include alongside the "core" ILP data formats. InterledgerRpc wraps the InterledgerPacket types in an envelope that adds a to, from (either or both of which can be set to empty strings if they are implied, as is the case in a bilateral connection), a requestId to correlate requests and responses, and sideProtocolData for extensibility. Any type of InterledgerRpc request can become a paid request by including a Transfer or ConditionalTransfer in it (which themselves can contain other InterledgerPackets -- InterledgerProtocolPayments or any other type of data that you might want/need to pay for).
based on comments from @michielbdejong in #251
PrepareRequest ::= SEQUENCE { | ||
transferId UInt128, | ||
amount UInt64, | ||
condition UInt256, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why rename executionCondition
to condition
?
based on comments from @michielbdejong in #251
based on comments from @michielbdejong in #251
Superseded by #271 |
Binary protocol to replace RPC API. Resolves #249.
I did not pick the name.
Created with @emschwartz and @sharafian.