Ledger Plugin Interface v2

[justmoon]

Please note the changes below are in roughly chronological order, later changes may revise earlier ones.

Ledgers are a type of connector (Interledger enlightenment)

@sharafian and I realized a while ago that ILP could be simplified by treating ledgers as another type of connector. This later came to be known as "Interledger enlightenment".

Plugins that represent ledgers have to do a simple form of routing. If the ILP destination address starts with the ledger prefix, they should forward the transfer to the account mentioned in the destination address (usually the first segment after the ledger prefix). If it doesn't, they should forward the payment to the default connector. This new behavior no longer allows a connector to reach multiple other connectors through one plugin. If that is desired, the connector should create a separate plugin per peer connector.

• Remove connectors: All payments to a plugin go through the default connector if the ILP destination is not on the ledger
• Remove account/from/to: It's the plugin's responsibility to determine the destination
• Remove prefix: Ledger prefixes are now handled internally by the plugin

Ephemeral transfers

Fulfillments and rejections are simply returned by sendTransfer calls. Now that we envision Interledger transfers to always be relatively small, we can slightly relax some of the guarantees around the individual transfer. Based on work by @michielbdejong and @emschwartz.

• Change sendTransfer to return Promise<FulfillmentInfo> or reject with InterledgerRejectionError
• Add registerTransferHandler and deregisterTransferHandler (if no handler is registered, plugin should reject all incoming transfers)
• Remove noteToSelf: We can now use local state in the sendTransfer caller function
• Remove fulfillCondition: Instead, return FulfillmentInfo
• Remove rejectIncomingTransfer: Instead, throw an InterledgerRejectionError
• Remove getFulfillment: With short-lived transfers, the need to retrieve a fulfillment for the purpose of retrying later is no longer relevant

LPI is not a different layer than ILP

The ILP packet has become so simple that it no longer makes sense to encapsulate it with the transfer object. Based on #270 by @adrianhopebailie and further work by @emschwartz.

• Remove ilp from Transfer, FulfillmentInfo
• Add destination:String, data:Buffer to Transfer
• Add data:Buffer to FulfillmentInfo

No optimistic mode

As suggested by @emschwartz, optimistic mode can be simulated using Optimistic-over-Universal (OoU).

• Remove *_transfer events
• Make executionCondition and expiresAt mandatory for transfers

Middleware provides better extensibility

As discovered by @emschwartz, many use cases of plugins can be better represented using middleware.

• Remove *_{prepare,fulfill,reject,cancel} events: These can be provided by a middleware that emits such events, if desired
• Remove *_{request,response} events

Quote by probing

Instead of a dedicated quoting protocol, we quote by sending a probe payment. See #309.

• Remove sendRequest: We no longer need a dedicated request/response feature

Bandwidth instead of balances

@sharafian made the point that ILP participants only care about their balances insofar as they affect their ability to send payments. In fact, balances are just a special case of a function that takes a payment and returns whether the payment is allowed or not. We can model this more generically by treating it similarly to IP bandwidth. IP doesn't track how much bandwidth is available but instead returns an error when packets are coming through too quickly.

• Remove getBalance
• Remove minBalance/maxBalance

Cleanup LPI errors

ILP/LPI errors are a bit of a mess right now.

• Define an InterledgerRejectionError to be a proper JavaScript error object
• Change forwardedBy to be an array to match the ILP packet
• Make message a local property since it does not exist in the ILP error packet and can therefore not be forwarded

Fixes #332.

ILDCP replaces LedgerInfo

It's not a plugin concern to configure the client. This can be done via messages sent on the Interledger layer.

• Remove getInfo, getAccount and LedgerInfo
• Remove info_change event

Conditional transfers are an Interledger concern

See #359. We don't need to enforce conditions on the ledger layer. This can be done on the Interledger layer.

• Remove all mentions of executionCondition, expiresAt
• Replace transfer: Transfer with amount: string
• Replace sendTransfer(transfer: Transfer): Promise<FulfillmentInfo> with sendMoney(amount: string): Promise<void>
• Replace sendRequest(message: Message): Promise<Message> with sendData(data: Buffer): Promise<Buffer>
• Replace [de]registerRequestHandler and [de]registerTransferHandler with [de]registerDataHandler and [de]registerMoneyHandler

Interledger rejections should be emitted on the Interledger layer only

The plugin previously emitted some Interledger rejections. The plugin shouldn't even know what Interledger is. All errors emitted by the plugin are now plain JavaScript exceptions.

• Remove InterledgerRejectionError

Other changes

• Add lpiVersion = 2 static variable
• Change most optional fields to required fields to reduce ambiguity
• Clarify that errors should be distinguished by name (fixes Clarify how to recognize different error types in from LPI #302)

[justmoon]

Created a module that can turn an LPI1 plugin into an LPI2 plugin, see interledgerjs/ilp-compat-plugin.

[justmoon]

LPI is not a different layer than ILP

Having some second thoughts about this one. It seems like we're losing extensibility in a pretty major way. In ILPv1, we can add things to the ILP packet which some connectors understand in such a way that it is ignored by the receiver and other connectors that don't support it.

For example, in ILPv1 we could have added a hop limit in a non-breaking way. With this, we can't because there is no way to attach ILP layer information beyond the two natively supported fields (destination and data).

That's fine if we think we never need to extend ILP. But that's a hell of a thing to say.

Edit: OTOH, less extensibility also means less potential for shenanigans.

[michielbdejong]

Why not use http calls instead of JavaScript function calls?
I think previously the answer was "because fulfillments travel in the opposite direction and so we need to listen for events".

[justmoon]

Why not use http calls instead of JavaScript function calls?

What do you mean by HTTP calls? You mean the plugin should expose a local HTTP server and the connector would call it? That would add a lot of overhead for no (?) benefit?

[adrianhopebailie]

Some editorial comments but nothing blocking

[adrianhopebailie]

Message -> FulfillmentInfo

[adrianhopebailie]

It's not clear to me how this differs from data if the fields MUST be passed to the receiver.

My assumption is that the plugin will know what needs to go to the receiver and what doesn't based on the underlying connection type.

[emschwartz]

It sounds like custom is either for the ledger plugin itself, or for the next connector, whereas the data is end-to-end data that should be treated as opaque by the intermediaries and must be passed on.

[justmoon]

It's not clear to me how this differs from data

@emschwartz is correct.

data is passed on by connectors and plugins. custom is only passed on by plugins (generally speaking), but not by connectors. Although there are exceptions. (For example, I'm using custom.ilpLegacyAmount to pass on the ILP amount in my connector refactor.)

I'll see if I can improve the wording.

Edit: Primarily, this seems to be an issue with overloading the terms "sender" and "receiver". When I say receiver of the transfer, I mean the local receiver, i.e. the next hop. When I say receiver of the payment, I generally mean the ILP receiver, i.e. the last hop. But I can see how that's superconfusing. Perhaps we should have different terms for the local receiver. Maybe just "local receiver"?

[adrianhopebailie]

I worry about having something that is intended for the next connector (local receiver) mixed in with something for the plugin.

This implies the plugin must know how to differentiate between the two (and only send the data intended for the next connector) and that the caller must have some knowledge of the underlying plugin (i.e. does it support X in custom).

This also suggests that we've simply moved the implementation challenges of sendMessage into sendTransfer. If I have a ledger that doesn't support messaging I need to establish a communications channel specifically for sending this data in parallel to the transfers.

Could we split custom into pluginData and peerData?

As a caller, if I have no idea what plugin I am talking to, I can still safely put anything serializable into peerData and know it will get to my peer.

I will only use pluginData if I know what plugin I am speaking to or if some of the data in there becomes standard and I expect all plugins to understand it.

[adrianhopebailie]

Replace: "All custom fields that were passed to sendTransfer MUST be passed to the transfer handler by the plugin on the receiving side"

With: "All custom fields that were set in FulfillmentInfo#custom on the receiving side MUST be set in FulfillmentInfo#custom when it is resolved by the promise on the sending side."

[adrianhopebailie]

If the ILP destination address starts with the ledger prefix, they should forward the transfer to the account mentioned in the destination address (usually the first segment after the ledger prefix). If it doesn't, they should forward the payment to the default connector.

I think this could be restated as:

If the ILP destination address is the address of the receiver, they should forward the transfer to the receiver's account. If it isn't, the connector should attempt to route the payment via a new outgoing transfer in the direction of the destination ILP address.

[adrianhopebailie]

p.s. I think the build fail is because you have used folder xxxx- and it's matching on (\d\d\d\d)

[michielbdejong]

the plugin should expose a local HTTP server and the connector would call it?

you can short-circuit that if it's within the same process. Instead of calling plugin.listen(LOCALHOST_PORT) and the connector then using node-fetch or superagent, the connector can call plugin.handleRequest directly

[emschwartz]

Couple of comments but overall I like this direction

[emschwartz]

nitpick: "lpi" is indistinguishable from "Ipi" in many fonts, what about just calling this version?

[emschwartz]

Should ledger plugins try to reconnect themselves or should that be up to the user of the plugin? If the plugin will try to reconnect itself, what's the point of having this event?

[adrianhopebailie]

Should ledger plugins try to reconnect themselves or should that be up to the user of the plugin?

-1 - That seems like functionality that should be written once in whatever consumes the plugin.

Also, a plugin CAN try to reconnect silently if it is able to buffer requests in the interim. That would be a feature of a good plugin but seems like a high bar for all plugins.

Finally, that makes some assumptions about the underlying transport that will be used which I think is a mistake.

[justmoon]

Currently, the connector listens to the disconnect event to know when it should stop advertising a certain route. In fact, it actively informs peers that it can no longer service this route.

For this use case alone, I think this is an important feature.

[emschwartz]

Can't you just call connect and wait for the promise to resolve instead of having a separate event for this?

[adrianhopebailie]

It's useful for monitoring to have events even if they are just pub-sub type notifications. i.e. You likely won't react to them other than to log them.

I was tempted to suggest we should still keep the transfer and fulfillment events for this reason

[justmoon]

I think we can use @emschwartz' idea of middleware to serve the transfer events use case. That's what convinced me to take them out.

class EventMiddleware extends EventEmitter
{
  constructor (next) {
    super()
    this.next = next
  }

  getTransferHandler() {
    return async (transfer) => {
      this.emit('prepare', transfer)

      try {
        const fulfillInfo = await this.next(transfer)
        this.emit('fulfill', transfer, fulfillInfo)
      } catch (err) {
        this.emit('reject', transfer, err)
      }
    }
  }
}

And you could hook that into sendTransfer to get events for outgoing transfers and info registerTransferHandler for incoming transfers.

const incomingEvents = new EventMiddleware(myActualTransferHandler)
plugin.registerTransferHandler(incomingEvents.getTransferHandler())

incomingEvents.on('prepare', (transfer) => {
  console.log('An incoming transfer for %s was prepared', transfer.amount)
})

[emschwartz]

What's the difference between NoSubscriptionsError and UnreachableError?

[emschwartz]

Do we have a recommended maximum? Should it fit into a UInt64?

[justmoon]

Don't seem like we need a maximum. We just need an error that tells you what the maximum is.

A connector should:

try {
   plugin.sendTransfer(...)
} catch (err) {
  if (err.name === 'MaxAmountExceeded') {
    // Get maximum amount from err.maxAmount and trigger a
    // "payment too large" ILP error to tell the sender to send a
    // smaller amount.
  }
}

[emschwartz]

It sounds like custom is either for the ledger plugin itself, or for the next connector, whereas the data is end-to-end data that should be treated as opaque by the intermediaries and must be passed on.

[emschwartz]

This line should be removed

[emschwartz]

Why not make it a buffer?

[adrianhopebailie]

They are fixed length. Isn't a Buffer uneccessary over-head.

[emschwartz]

Is the value assumed to be JSON? Could you store a Buffer instead (that's what the LevelUP API assumes unless you're using the encoding-down wrapper)?

[emschwartz]

Couldn't this just be handled by the caller (with a Promise.race)?

[justmoon]

Hmm, how would you tell the plugin to stop trying to connect? Call disconnect?

[justmoon]

Ironically, I just ran into an issue in ilp that's very similar to this. ilp uses Promise.race to timeout connecting, but Promise.race doesn't do anything to stop the thing that "lost" the race.

In my case what this did is that the connection timeout was keeping the tests from exiting. (Mocha 4 no longer force-exits unlike Mocha 3, so you have to clean everything up properly.)

I did a quick fix by setting a shorter timeout. But imo, the proper fix is to remove Promise.race and replace it with setTimeout and call clearTimeout once the connection completes successfully. If you really wanted to use promises, you could use bluebird and it's idea of cancellation, but I think that's actually less clean.

General lesson: When using Promise.race, be aware that the thing that loses the race may keep doing stuff that's no longer necessary and/or that you don't want it to be doing.

[justmoon]

p.s. I think the build fail is because you have used folder xxxx- and it's matching on (\d\d\d\d)

Yes, I'm aware. I'll assign an RFC number when we're closer to being ready to merge.

[adrianhopebailie]

Yes, I'm aware. I'll assign an RFC number when we're closer to being ready to merge.

What I meant is that you could use 0000- until you take a number

[emschwartz]

I think we need more details on what value types are allowed in here and how you're supposed to use it. Do we only support JavaScript JSON values? What's the range on supported numbers?

Also, is every implementation free to use its own key names? Can connectors modify this to add more details? What if they want to add a property but the key is already used by another application?

Overall, I'm not sure that having an object here is much more helpful than having a single data property that transport protocols can define the encoding for, or is otherwise assumed to be a string.

[emschwartz]

How about adding a data property that is specifically for Transport Layer protocols and then specifying that additionalInfo can contain key-value pairs as UTF-8 strings?

[michielbdejong]

I don't think it's useful to have field names that differ from the 4 headers used in http-over-ilp (not to be confused with http-ilp).

The mapping is as follows, only the field names are different:

expiresAt -> ILP-Expiry
amount -> ILP-Amount
destination -> ILP-Destination
executionCondition -> ILP-Condition

We could also keep the names proposed here, but then I would propose changing them in #349.

[michielbdejong]

In the light of #349 I would change this to:

{
  headers: {
    'ILP-Destination': 'g.crypto.bitcoin.1XPTgDRhN8RFnzniWCddobD9iKZatrvH4.~asdf1234',
    'ILP-Condition': 'x73kz0AGyqYqhw_c5LqMhSgpcOLF3rBS8GdR52hLpB8',
    'ILP-Expiry': '2017-12-07T18:47:59.015Z',
    'ILP-Amount': '1000'
 },
 body: <body>
}

[michielbdejong]

Or maybe

{
  destination: 'g.crypto.bitcoin.1XPTgDRhN8RFnzniWCddobD9iKZatrvH4.~asdf1234',
  condition: 'x73kz0AGyqYqhw_c5LqMhSgpcOLF3rBS8GdR52hLpB8',
  expiry: '2017-12-07T18:47:59.015Z',
  amount: '1000'
  body: <body>
}

[sappenin]

-1 for the first transfer proposal, but +1 for something like the 2nd transfer proposal.

• The prefixes (i.e., ILP-... in ILP Over HTTP #349 are there as an artifact of using HTTP headers to carry the data (to distinguish between other headers that might be arbitrarily added to the request as part of a normal and valid HTTP request). Conversely, In this interface, the prefixes are not needed because there's less of a risk of "other data fields" colliding with these fields names -- mainly because we're defining the field names here.
• body is an artifact of HTTP. We should call it what it really is here (I like Adrian's suggestion above of peerData and pluginData), and just make new attributes for the data we need.

[justmoon]

Regarding whether to keep the ILP packet as an opaque blob or not:

Randomly reread chapters 1 and 2 of ASN.1 Complete recently, which talk about the exact problem we've run into. Should you separate your protocol into layers (as TCP/IP does) or should you merge them into one (as most telecommunication protocols did at the time)? It's a trade-off of simplicity vs extensibility. If you layer your protocol, each layer can evolve independently. If you merge them, they become simpler (in the short run.)

It's a trade-off and therefore: Too many layers are bad. As are too few.

So the question is: Do we need the ledger layer to be able to evolve independently from the Interledger layer? Do we want to be able to think about them independently?

I would say, yes, absolutely. We already have two actively used, competing, ledger layer protocols in BTP and ILP-over-HTTP (not to be confused with HTTP-ILP) just based on preferences without our own small team. Others may wish to use protocol buffers. In the future, we may not want to have a dependency on the TCP/IP stack at all and come up with a ledger protocol that layers straight on a secure communication link.

Merging the Interledger and ledger layers seems simple, because you're getting rid of a layer, but it isn't because now it's harder to evolve your ledger layer or your Interledger layer independently since they're tied together.

If we agree that we want to have separation between the ledger/account layer and the Interledger layer, then we need to decide whether the interface is:

1. You need to transfer this small set of fields (Destination, Data, Error-Code, Error-Triggered-By, ...) or
2. You need to transfer an opaque blob (ILP packet)

The second provides a lot less opportunity for ledger protocols to get it wrong and allows new ILP packet types to be created in the future, without having to modify each ledger protocol. (Note that you wouldn't just want to update the implementations, you'd want to update the protocols.)

We (at least some of us) have collectively come to this conclusion before. That's where the ILP packet came from. So perhaps it's not surprising that thinking about it again would lead back to the same place.

So, considering this, I would strongly advocate we make the LPIv2 interface such that preparation, fulfillment and rejection each take/emit an ILP Buffer, which is treated as an opaque blob by the plugin.

[adrianhopebailie]

Personally I am in favor of us leaning more toward simplicity in this case.

I agree that the "correct" way to layer these protocols is to have distinct layers between the ledger and ILP layers but I think the cost is higher than we estimate and that the extensibility requirement is overstated.

The interface between the Ledger and ILP layers is unique (compared to other inter-layer interfaces) because the majority of the data elements are standardized by the Interledger standard yet in the current model they are passed both at the ledger and ILP layers.

Ledger layer:

• Condition
• Expiry
• Fulfillment

ILP layer:

• ILP Address
• ILP Error Code/Name
• ILP Error Source (and intermediaries)
• ILP Error Timestamp

I think there is a strong argument that these should all be at the same layer.

I have made the case previously that the Condition and Fulfillment should be in the ILP layer and I think we have agreed that this is theoretically correct but practically makes no sense because they will be mostly used at the ledger layer. We compromised for simplicity.

In my opinion the same argument could be made for moving the other fields to the ledger layer.

There are a lot of benefits:

1. No need to define a new packet format and encoding.
2. The end-to-end data is just transported as the payload of whatever Application Communication Protocol is used for the ledger layer protocol (eg: HTTP, CoAP, JSON-RPC, custom).
3. We can leverage existing methods for securing and integrity-checking this data end-to-end (otherwise they are only usable at the ledger layer)
4. No need for a fulfillment packet that is just an envelope around a BLOB
5. No need for a payment packet that simply adds an address to the data

In summary, re-use a lot of what already exists and invent less ourselves making implementation and understanding simpler.

I find it very difficult to justify the complexity we have currently to cater for extensibility that seems very unlikely to ever be required.

[sappenin]

I concur 100% with @adrianhopebailie -- the benefits he outlined are really compelling (I had a bunch of feedback, but I think Adrian captured nearly all of it).

Only thing I would underscore is developer adoption -- implementing a packet with codecs in different languages might seem trivial, but it is a significant barrier to adoption, IMHO, and it's not clear that we couldn't accomplish the same things using HTTP and a well defined spec.

[dappelt]

On a different note: If getBalance is removed, how does a user know what his balance is? Or is the idea that the user is not aware of his balance at all?

Knowing how much funds a user has available is essential for many purchase decisions. For example, let's say a user has 5€ left in his wallet. He would like to buy a coffee and a newspaper, both cost 3€. If the user is aware of his balance, he would know that he can afford either the coffee or the newspaper, but not both, and make a conscious purchase decision. On the other hand, if the balance is unknown to the user, he would buy one item and only when he tries to pay for the second item he would figure out that he has not enough funds for both.

[michielbdejong]

Removing plugin.sendRequest only makes sense if we pivot to pennies, if we want Interledger to keep supporting the slightly wider variety of ledgers which it supports now, then the choice between on-ledger messaging and side-channel messaging is a plugin-level decision, and then IMHO we should keep that method as it is.

[emschwartz]

@justmoon While working on the PSKv2 implementation using the LPI2 I realized we also want to have a way to ignore an incoming transfer. This is necessary if you have two receivers listening on one account and want to make sure they don't interfere with one another.

[sharafian]

actually @justmoon and I were briefly discussing this yesterday. It might be a best practice to have each listener get its own sub-account. This means your ILP address would change every time you reconnect (which is OK because they're ephemeral anyways), but all your listeners would still have a shared balance. The ILP-DCP protocol would just have to take into account which listener is asking, and then send out to that same listener when notifications come in.

[michielbdejong]

I think we will eventually want to add the to field back as an optional parameter on sendData and sendMoney so that you can address multiple peers on one ledger. If you would split it into one plugin per peer, then their balances would go up and down in unison, which would be less helpful than if we just have one plugin per ledger, with a to field. After all, this is still called the ledger plugin interface, not the Peer Interface. But for now I guess it's fine to remove to, because for the forseeable future, we will mainly be working with one-to-one relationships, not one-to-many.

[emschwartz]

Nitpick: sendMoney or sendValue?

[sharafian]

sendValue would be a confusing name, because it sounds like it takes a value and communicates that value to your counterparty. sendMoney is technically not correct in all circumstances, but I think it makes more sense

[sappenin]

sendAsset?

[emschwartz]

If you specifically wanted to implement a ledger plugin that did send ILP packets over conditional ledger transfers, would you just implement one that looks at the data being sent and handles it in a different way?

[michielbdejong]

I think it would be good to clarify two things:

• how do sendData and sendMoney affect the account's balance
• how can you use on-ledger transfers or paychan claim updates to rebalance?

For example, think about how this would work in ilp-plugin-xrp-paychan:

Both parties open a payment channel to the other party, but don't send any claims yet. Balances start at zero.

Using plugin.sendData, the Sender sends a Prepare packet for 10 drops, and the Receiver fulfills it, so now the Receiver's balance is 10, and the Sender's balance is -10. Right? This is the unsecured balance.

Now in order not to hit their peer's maxUnsecuredBalance, the sender signs a paychan claim for 10 drops. How can they send this claim to their peer, and make their own balance go back up to 0?

Option A) the Sender calls plugin.sendMoney with a negative amount, and attaches the claim
Option B) the Sender sends the claim out-of-band (it used to be in the Fulfill Response), and then the Receiver calls plugin.sendMoney with a positive amount
Option C) instead of plugin.sendMoney, we define plugin.claimMoney, such that plugin.claimMoney with a positive amount is equivalent to plugin.sendMoney with a negative amount.

I think maybe there's a confusion about what we consider the "balance" of each peer? In any case, it would be good to clarify this - the word "balance" does not occur in the text now.

[sharafian]

Using plugin.sendData, the Sender sends a Prepare packet for 10 drops, and the Receiver fulfills it, so now the Receiver's balance is 10, and the Sender's balance is -10. Right? This is the unsecured balance.

Now in order not to hit their peer's maxUnsecuredBalance, the sender signs a paychan claim for 10 drops. How can they send this claim to their peer, and make their own balance go back up to 0?

I would look at this a different way. When the sender calls sendData, they are giving nothing to their peer (a sendData call carries no money), but the peer is forwarding an interledger payment on their behalf, which they will later have to settle. This means the peer is out 10 units. This is what brings the sender's balance to -10. They've effectively fronted you the money.

Now when the sender wants to pay off their debt (i.e. bring the balance back up), they call sendMoney. This call actually transmits 10 units of money to the peer, and the peer keeps this settlement without forwarding anything out. This means the peer has gained 10 units. This brings the sender's balance back to 0.

TL;DR for other people following this: sendData increases debt because your peer gets nothing in and sends a payment out for you. sendMoney decreases debt because the peer gets money in and sends nothing out for you.

[michielbdejong]

Right, I see your point. So then I agree, for the LPI the naming does makes sense; we just say that plugin.sendData decreases the sender's trustline balance, and plugin.sendMoney decreases the sender's on-ledger or paychan-secured balance, so that both can have positive amounts and it still makes sense. So then my only issue is with the way the word "transfer" is used in #360.