Update to ILPv4 + LPI2 #401

justmoon · 2017-12-18T17:21:58Z

Depends on interledger-deprecated/ilp-compat-plugin#7

emschwartz

Overall LGTM, but I think @sentientwaffle should also review this because he's more familiar with a lot of the code this PR touches

emschwartz · 2017-12-18T17:39:11Z

package.json

    "five-bells-shared": "^25.1.0",
    "ilp": "~11.2.0",
-    "ilp-packet": "~1.3.0",
+    "ilp-packet": "^2.0.0",
    "ilp-plugin-bells": "^15.0.0",


Should we stop including plugin bells?

emschwartz · 2017-12-18T17:39:11Z

src/lib/executeSourceTransfer.js

-  const fulfillPromise = promiseRetry(retryOpts, function (retry, number) {
-    return Promise.resolve(plugin.fulfillCondition(sourceTransferID, fulfillment, fulfillmentData))
-      .catch(function (err) {
-        if (shouldRetry(err)) {


Should we add fulfillment retry logic to the ilp-compat-plugin?

I do think that fulfillment retries should be in the plugin because it depends on the ledger protocol if and how you want to retry.

Should it be in ilp-plugin-compat? I don't think it's worth the effort. Retry logic is an optimization, it's only to minimize unnecessary connector losses. I'd say we migrate to LPI2 and then add retry logic per plugin. Different plugins may want to handle retries quite differently, depending on whether they're TCP or UDP based, session or request based or to deal with some other ledger/transport-specific quirks.

emschwartz · 2017-12-18T17:39:11Z

src/lib/utils.js

@@ -22,40 +20,6 @@ function getPairs (arr) {
  }, [])
 }

-/**
- * Deterministically generate a UUID from a secret and a public input.


Right now Mojaloop depends on having the same transfer ID across all of the ledgers. Taking the transfer id out of the LPI will make it more difficult to upgrade the connector used in that project. Do we have any alternative way of attaching the transfer ID from the previous ledger and making the connector use the same one on the outgoing transfer, or do we need to just make Mojaloop not depend on having the same transfer IDs?

sentientwaffle

LGTM except for a couple minor notes

sentientwaffle · 2017-12-18T19:38:24Z

src/lib/config.js

@@ -149,6 +149,8 @@ function getLocalConfig () {
  features.debugReplyNotifications =
    Config.castBool(Config.getEnv(envPrefix, 'DEBUG_REPLY_NOTIFICATIONS'))

+  const account = Config.getEnv(envPrefix, 'ILP_ADDRESS') || 'unknown'


Add ILP_ADDRESS to the README.

sentientwaffle · 2017-12-18T19:38:24Z

src/lib/route-broadcaster.js

@@ -39,9 +39,10 @@ class RouteBroadcaster {

    this.autoloadPeers = config.autoloadPeers
    this.defaultPeers = config.peers
-    // peersByLedger is stored in the form { ledgerPrefix ⇒ { connectorAddress ⇒ true } }
+    // peersByLedger is stored in the form { ledgerPrefix ⇒ true }


s/peersByLedger/peers

michielbdejong · 2017-12-18T19:39:10Z

src/lib/route-broadcaster.js

-        // timeout the plugin.sendRequest Promise just so we don't have it hanging around forever
-        timeout: this.routeBroadcastInterval
-      })
+    const broadcastPromise = this.ledgers.getPlugin(adjacentLedger).sendRequest({


Does this.ledgers.getPlugin(adjacentLedger) return an LPI1 or an LPI2 interface? Because the sendRequest method, needed for broadcasting routes, seems to be absent in LPI2?

It's not in the spec but it is in the ilp-compat-plugin as a deprecated feature

sendRequest won't exist when compat() returns the original LPI2 plugin (https://github.com/interledgerjs/ilp-compat-plugin/blob/d8331126a92ad2b65a7e7c4513ce44c13d485340/index.js#L31-L33).

So then this PR will break route broadcasts?

LPI1 plugins will work thanks to compat; LPI2 plugins will be broken.

They should be switched to use either interledger payments or the internet

in a similar way to @justmoon 's DHCP on ILP proposal (for asking your peer to assign you an ILP address), it would be easy to send a payment to your peer for which the destination is something like peer.broadcast, and the condition is either trivially fulfillable (if you're paying to send broadcasts) or unfulfillable (if you're not paying to send broadcasts). I don't see why that's worse/harder than using messaging.

Sure, that's what plugin-xrp-escrow already does. And if a BTP connection is available between the connectors then they can just use a BTP Message packet. But that's what plugin.sendRequest already nicely abstracts, and we agree that ilp-connector needs it, so then instead of removing code from the stack, we have just split the ledger plugin into a separate transfer plugin and messaging plugin, which is just moving code around.

We don't actually need a messaging plugin anymore though, because transfers are expected to be just as fast as messages. And setting the amount to 1 means that the impact on your liquidity is negligible. This just lets us keep plugins as simple as possible, which should make them less buggy and more secure

Ben and I discussed this on Slack and reduced this question to interledger/rfcs#357


        refactor: use lpi2

codecov-io · 2017-12-19T16:26:35Z

Codecov Report

Merging #401 into master will decrease coverage by 10.27%.
The diff coverage is 74.54%.

@@             Coverage Diff             @@
##           master     #401       +/-   ##
===========================================
- Coverage   84.82%   74.54%   -10.28%     
===========================================
  Files          22       51       +29     
  Lines         336     1756     +1420     
  Branches       49      278      +229     
===========================================
+ Hits          285     1309     +1024     
- Misses         51      447      +396

Impacted Files	Coverage Δ
src/errors/invalid-packet-error.ts	`100% <100%> (ø)`
src/services/routing-table.ts	`100% <100%> (ø)`
src/errors/invalid-amount-specified-error.ts	`100% <100%> (ø)`
src/errors/unacceptable-amount-error.ts	`100% <100%> (ø)`
src/errors/unreachable-error.ts	`100% <100%> (ø)`
src/errors/unacceptable-expiry-error.ts	`100% <100%> (ø)`
src/lib/ilp-errors.ts	`100% <100%> (ø)`
src/errors/no-route-found-error.ts	`100% <100%> (ø)`
src/common/log.ts	`100% <100%> (ø)`
src/errors/invalid-fulfillment-error.ts	`100% <100%> (ø)`
... and 92 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 25963d7...bb8f737. Read the comment docs.


        refactor: new routing/quoting engine


        refactor: turn config into a class


        refactor: use reduct for dependency injection


        refactor: use latest lpi2 version (ilp4)

emschwartz

Looks awesome to me. @sentientwaffle should definitely review this again

emschwartz · 2017-12-27T08:28:19Z

src/controllers/ildcp-host.js

+    writer.writeVarOctetString(Buffer.from(sourceAccount, 'ascii'))
+    writer.writeUInt8(info.currencyScale)
+    writer.writeVarOctetString(Buffer.from(info.currency, 'utf8'))
+    return writer.getBuffer()


This encoding should probably be added to the ilp-packet module

Yeah, I wasn't sure where to put it. It's not ilp-packet layer, so I don't think ilp-packet is actually the right place unless we turn that into ilp-formats. Plus - is it really worth modularizing four lines of code?

Fair enough

emschwartz · 2017-12-27T08:28:19Z

src/controllers/ilp-prepare.js

+      const { nextHop, nextHopPacket } = await this.routeBuilder.getNextHopPacket(sourceAccount, parsedPacket)
+
+      log.debug('sending outbound ilp prepare. destination=%s amount=%s', destination, nextHopPacket.amount)
+      const result = await this.sendDataToPeer(nextHop, IlpPacket.serializeIlpPrepare(nextHopPacket))


Shouldn't we modify the ILP packet in place rather than re-encoding? (That could also be a separate PR though)

Imho, that's premature optimization. Not long ago, we were ok going with a text-based format that would have been an order of magnitude slower. It's just not worth worrying about trying to get zero-copy performance at this stage. There are much lower hanging fruit.

emschwartz · 2017-12-27T08:28:19Z

src/errors/remote-quote-error.js

+
+    // TODO: Is this the right error code? Maybe we should pass on the error we
+    //   we received where possible?
+    this.ilpErrorCode = codes.T00_INTERNAL_ERROR


I'm not sure these errors will be temporary if other connectors just stop supporting ILQP

Good point - what's the right error code here? Or should we add one?

(I ran into a few cases where I was like: might be good to add a code for this - or make an existing code less specific.)

emschwartz · 2017-12-27T08:28:19Z

src/lib/plugin-store.js

+      logging: log.debug,
+      omitNull: true,
+      // All transactions should be done with isolation level SERIALIZABLE
+      // TOOD:


What's this TODO for?

This "TOOD" was going to say: So we still need SERIALIZABLE isolation level. (My guess is no.)

emschwartz · 2017-12-27T08:28:19Z

src/services/balances.js

+  }
+
+  handleMoney (amount) {
+    // TODO: Implement balance logic


This should be implemented ASAP after this PR is merged

Yeah. I think that's a job for @sharafian - he's written that code about 15 times already since it used to be in each plugin.

emschwartz · 2017-12-27T08:28:19Z

src/services/route-builder.js

+
+  // Make sure that neither amount exceeds 15 significant digits.
+  if (rate.gt(1)) {
+    return new LiquidityCurve([ [0, 0], [ PROBE_AMOUNT / rate, PROBE_AMOUNT ] ])


Shouldn't this be using bignumber.js' math functions?

Copied the existing code to make sure the results are exactly the same - I still like that idea so we don't have to update the tests as much. But we'll see once we get to updating the integration tests how it'll play out.


        chore: update deps


        fix: avoid timeout preventing node from exiting


        fix: parse CONNECTOR_DB_URI correctly


        chore: fix linting issues

sentientwaffle · 2017-12-28T22:26:33Z

README.md

  }, {
    "targetPrefix": "cny.",
-    "connectorLedger": "ilpdemo.red."
-    "connectorAccount": "lpdemo.red.cny_connector"
+    "peerAddress": "ilpdemo.red."


I think the peerAddress for this route should be ilpdemo.blue. (or something other than ilpdemo.red.), otherwise it will be caught by the targetPrefix: "" route anyway.

sentientwaffle · 2017-12-28T22:26:33Z

src/services/route-broadcaster.js

+
+  async start () {
+    try {
+      await this.reloadLocalRoutes()


this.reloadLocalRoutes is sync, no need to await.

sentientwaffle · 2017-12-28T22:26:33Z

src/services/route-broadcaster.js

+    await new Promise(resolve => setTimeout(resolve, this.config.routeBroadcastInterval))
+
+    try {
+      await this.reloadLocalRoutes()


this.reloadLocalRoutes is sync, no need to await.

sentientwaffle · 2017-12-28T22:26:33Z

src/backends/fixerio/index.js

-   * @param {String} params.destination_ledger The URI of the destination ledger
-   * @param {String} params.source_currency The source currency
-   * @param {String} params.destination_currency The destination currency
+   * @param {String} sourceAccount The URI of the source ledger


These are ILP addresses, not ledger URIs (the code is correct; the comment is wrong).

sentientwaffle · 2017-12-28T22:26:33Z

src/backends/fixerio/index.js

-   * @param {String} params.destination_ledger The URI of the destination ledger
-   * @param {String} params.source_amount The amount of the source asset we want to send
-   * @param {String} params.destination_amount The amount of the destination asset we want to send
+   * @param {String} params.sourceAccount The URI of the source ledger


sourceAccount and destinationAccount are ILP addresses.

Not anymore after config refactor.

sentientwaffle · 2017-12-28T22:26:33Z

src/controllers/ilp-prepare.js

+          })
+      }
+
+      this.backend.submitPayment({


Should the submitPayment call be inside of the result[0] === TYPE_ILP_FULFILL block? Otherwise it might be submitted even if sendDataToPeer replied with a rejection.

sentientwaffle · 2017-12-28T22:26:33Z

src/lib/utils.js

+
+      if (nextSegmentEnd === -1) {
+        prefix = address
+        return false


No need to return false, just return.

sentientwaffle · 2017-12-28T22:26:33Z

src/routing/peer.js

+const PrefixMap = require('./prefix-map')
+const log = require('../common').log.create('routing-peer')
+
+const PEER_PROTOCOL_PREFIX = 'peer'


ilp-prepare.js declares this as 'peer.'.

sentientwaffle · 2017-12-28T22:26:33Z

src/routing/peer.js

+    const newRoutes = routes.filter(route => route.epoch > this.epoch && route.nextHop !== this.address).map(route => ({
+      source_ledger: this.address,
+      destination_ledger: route.prefix,
+      min_message_window: 1,


This should reference the configured minMessageWindow.

Removed for now.

sentientwaffle · 2017-12-28T22:26:33Z

src/routing/peer.js

+        data: {
+          new_routes: newRoutes,
+          hold_down_time: holdDownTime,
+          unreachable_through_me: unreachableAccounts,


unreachableThroughMe, not unreachableAccounts.


        feat: pluggable store, switch default from sqlite to leveldown


        feat: connector should log its address on startup


        refactor: config system overhaul


        refactor: routing fixes and simplified format


        feat(store): make memdown its own store type.


        feat(config): add missing store config options


        refactor: reduce dependencies

sentientwaffle · 2018-01-02T19:39:18Z

src/services/accounts.js

+
+    const store = this.store.getPluginStore(accountAddress)
+
+    creds.options.prefix = accountAddress


Is this line necessary (see line 93 prefix: accountAddress + '.',)? If it is, shouldn't the prefix be set with a + '.' suffix?

sentientwaffle · 2018-01-02T21:09:09Z

src/services/config.js

+    features.debugReplyNotifications =
+    Config.castBool(Config.getEnv(envPrefix, 'DEBUG_REPLY_NOTIFICATIONS'))
+
+    this.address = Config.getEnv(envPrefix, 'ILP_ADDRESS') || ''


This parameter should be documented in the README.

Yeah, README needs a major rewrite...


        refactor: port to typescript


        fix: correctly generate store config object


        feat: add middleware framework


        perf: lazily create plugin store


        style: fix linting issues

sentientwaffle · 2018-01-16T17:19:48Z

README.md

+| -------------- | ------- | ------------------------------------------------------------------------ |
+| `[]`           | object  | Object describing middleware instance.                                   |
+| `[].type`      | string  | NPM module that should be `require`d to load the middleware constructor. |
+| `[].priority`  | integer | Priority at which this middleware should be inserted. Must be unique.    |


Is this field implemented? I don't see it used in the code.

Nope, it was from a previous version, I'll take it out!


        fix(types): plugins are event emitters


        fix(middleware): always pass an object as the options


        fix(config): remove unused config options


        chore(npm): rebuild schemas on npm run build


        feat: warn user about unrecognized configuration variables


        chore(docker): update docker file - use node 8


        fix: handle "integer" config; relay broadcasts

Both of these changes are needed to get the integration tests passing.

sentientwaffle · 2018-01-18T19:08:34Z

README.md

+| `*.assetCode`                 | string  | Currency code or other asset identifier that will be passed to the backend to select the correct rate for this account.                                                                                                                                                                                     |
+| `*.assetScale`                | integer | Interledger amounts are integers, but most currencies are typically represented as fractional units, e.g. cents. This property defines how many Interledger units make up one regular units. For dollars, this would usually be set to 9, so that Interledger amounts are expressed in nanodollars.         |
+| `*.balance`                   | object  | _Optional_ Defines whether the connector should maintain and enforce a balance for this account. This setting is enforced by the built-in `balance` middleware.                                                                                                                                             |
+| `*.balance.maximumDebt`       | string  | How much money (in atomic units) this counterparty is allowed to owe us before we start rejecting their packets. Each incoming ILP prepare packet increases the amount owed, each incoming settlement lowers it. Provided as an integer in a string.                                                        |


should be balance.maximum (or the balance middleware needs to change).


        chore: use nyc for code coverage


        test: fix tests


        test: remove references to missing classes


        test: fix tests hanging


        docs: update README with latest config schema

emschwartz · 2018-01-19T19:26:06Z

Woohoo! What a monster PR. Nicely work @justmoon

emschwartz approved these changes Dec 18, 2017

View changes

sentientwaffle approved these changes Dec 18, 2017

View changes

michielbdejong reviewed Dec 18, 2017

View changes

michielbdejong mentioned this pull request Dec 18, 2017

Update to LPI2 interledgerjs/ilp#122

Merged

refactor: use lpi2

Loading status checks…

6279cac

justmoon force-pushed the refactor/st-lpi2 branch from 4cc5e35 to 6279cac Dec 19, 2017

justmoon added 4 commits Dec 23, 2017

refactor: new routing/quoting engine

12b6a33

refactor: turn config into a class

21f9d3f

refactor: use reduct for dependency injection

9021056

refactor: use latest lpi2 version (ilp4)

Loading status checks…

b23fe4a

justmoon force-pushed the refactor/st-lpi2 branch from 9d17bb5 to b23fe4a Dec 26, 2017

emschwartz approved these changes Dec 27, 2017

View changes

justmoon added 4 commits Dec 27, 2017

chore: update deps

910d77c

fix: avoid timeout preventing node from exiting

954fad9

fix: parse CONNECTOR_DB_URI correctly

Loading status checks…

c191939

chore: fix linting issues

Loading status checks…

3988c85

sentientwaffle requested changes Dec 28, 2017

View changes

justmoon added 7 commits Dec 29, 2017

feat: pluggable store, switch default from sqlite to leveldown

Loading status checks…

45421ff

feat: connector should log its address on startup

Loading status checks…

daeedaa

refactor: config system overhaul

427d785

refactor: routing fixes and simplified format

e7e6dfd

feat(store): make memdown its own store type.

f04711d

feat(config): add missing store config options

9b80835

refactor: reduce dependencies

49efb2a

sentientwaffle reviewed Jan 2, 2018

View changes

justmoon force-pushed the refactor/st-lpi2 branch from 4bc732c to 5ced0f2 Jan 3, 2018

refactor: port to typescript

Loading status checks…

913ca83

justmoon force-pushed the refactor/st-lpi2 branch from 5ced0f2 to 913ca83 Jan 3, 2018

fix: correctly generate store config object

Loading status checks…

7b73f45

justmoon force-pushed the refactor/st-lpi2 branch 4 times, most recently from f7cf20e to 3a3cc48 Jan 16, 2018

feat: add middleware framework

Loading status checks…

5d0867d

justmoon force-pushed the refactor/st-lpi2 branch from 3a3cc48 to 5d0867d Jan 16, 2018

justmoon added 2 commits Jan 16, 2018

perf: lazily create plugin store

9b38350

style: fix linting issues

Loading status checks…

3597b98

sentientwaffle reviewed Jan 16, 2018

View changes

justmoon and others added 7 commits Jan 16, 2018

fix(types): plugins are event emitters

1aaa3cb

fix(middleware): always pass an object as the options

bed0e87

fix(config): remove unused config options

4367d43

chore(npm): rebuild schemas on npm run build

Loading status checks…

c36e4e6

feat: warn user about unrecognized configuration variables

Loading status checks…

bc8cea1

chore(docker): update docker file - use node 8

Loading status checks…

e09ec66

fix: handle "integer" config; relay broadcasts

Loading status checks…

38596f0

Both of these changes are needed to get the integration tests passing.

sentientwaffle reviewed Jan 18, 2018

View changes

justmoon added 2 commits Jan 19, 2018

chore: use nyc for code coverage

4cf3387

test: fix tests

29c1a8a

justmoon force-pushed the refactor/st-lpi2 branch from 9030e87 to 28c6873 Jan 19, 2018

test: remove references to missing classes

Loading status checks…

b33434a

justmoon force-pushed the refactor/st-lpi2 branch from 28c6873 to b33434a Jan 19, 2018

justmoon added 2 commits Jan 19, 2018

test: fix tests hanging

6ed7717

docs: update README with latest config schema

Loading status checks…

bb8f737

justmoon force-pushed the refactor/st-lpi2 branch from 8a8a992 to bb8f737 Jan 19, 2018

justmoon merged commit 6c72764 into master Jan 19, 2018
1 of 2 checks passed

1 of 2 checks passed

ci/circleci CircleCI is running your tests
Details

licence/cla Contributor License Agreement is signed.
Details

justmoon mentioned this pull request Feb 21, 2018

Phase 0: Minimum Viable Network (Pong) interledger-deprecated/interledger#7

Open

5 of 9 tasks complete


		const store = this.store.getPluginStore(accountAddress)

		creds.options.prefix = accountAddress

interledgerjs / ilp-connector

Join GitHub today

Update to ILPv4 + LPI2 #401

Update to ILPv4 + LPI2 #401

Conversation

justmoon commented Dec 18, 2017

emschwartz left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

sentientwaffle left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

codecov-io commented Dec 19, 2017 • edited

Codecov Report

emschwartz left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

emschwartz commented Jan 19, 2018

codecov-io commented Dec 19, 2017 •

edited