Delegated Peer and Content Routing #242

jacobheun · 2018-08-22T11:06:17Z

This is ready!

This PR adds the ability to configure delegate routing modules, #120.

I've included an example of the delegated routing working.

Note: until ipfs/kubo#4595 is released, running a compatible local node for the example requires building go-ipfs from the feat/delegated-routing branch and running that locally. This will also not work in production until a Gateway node is deployed with delegated routing support: ipfs/infra#426.

This also provides libp2p users with the ability to create and supply their own peer and content routing modules.

jacobheun · 2018-08-22T11:25:34Z

src/index.js

-    this.contentRouting = contentRouting(this)
+    // If peer or content routing modules have been provided, use those, otherwise use the dht
+    this.peerRouting = this._modules.peerRouting || peerRouting(this)
+    this.contentRouting = this._modules.contentRouting || contentRouting(this)


@diasdavid my original thought was to either use the provided peer and content routing modules, or the dht. After finishing up the core of the work and thinking more about the example and usage I feel like it would be better to use both if they are enabled. The thought being that leveraging a delegate is a good way to bootstrap a node into the network while it grows its own network, but that overtime its network would have a lot of the information. We could also change peer and content routing to allow for multiple modules and in the future potentially introduce some priority logic for ordering the calls to prevent flooding the network, if that becomes an issue.

Think of it as a Stream Muxer or a Transport, you can use one, many, all the same time, disable some, you name it :)

jacobheun · 2018-09-27T14:42:02Z

Delegated routing is ready for review! The IPFS updates aren't finished or deployed yet, but this is ready for when it is!

cc: @lgierth

vasco-santos

@jacobheun the PR seems good! The example rocks 💪

Added just some small notes.

Note, some of them are the same problem but in different files. I reported all of them (I hope), to guarantee that you do not forget any.

vasco-santos · 2018-09-27T16:50:05Z

examples/delegated-routing/src/libp2p-bundle.js

+import WebRTCStar from 'libp2p-webrtc-star';
+import MPLEX from 'libp2p-mplex';
+import SECIO from 'libp2p-secio';
+import KadDHT from 'libp2p-kad-dht';


Not problematic, but since it is disabled, we could not have it has a dependency and have null in the config.

vasco-santos · 2018-09-28T09:13:21Z

src/content-routing.js

+      if (typeof options === 'function') {
+        callback = options
+        options = {}
+      } else if (typeof options === 'number') {


Can you just add a comment here for future deprecation?

vasco-santos · 2018-09-28T09:22:16Z

src/content-routing.js

+
+          // If we don't have any results, we need to provide an error to keep trying
+          if (!results || Object.keys(results).length === 0) {
+            return cb(Object.assign(new Error('not found'), {


use err-code module here

Good catch, I completely forgot to switch that over!

vasco-santos · 2018-09-28T09:28:15Z

src/content-routing.js

+     */
+    findProviders: (key, options, callback) => {
+      if (routers.length === 0) {
+        return callback(new Error('No content routers available'))


add err-code here?

vasco-santos · 2018-09-28T09:30:52Z

src/content-routing.js

+     * @returns {void}
+     */
+    findProviders: (key, options, callback) => {
+      if (routers.length === 0) {


I would simply use

if (!routers.length)

But it is a detail. You have this in several places if you agree please change them all, otherwise leave like this

vasco-santos · 2018-09-28T09:35:10Z

src/peer-routing.js

+        if (err && err.code !== 'NOT_FOUND') {
+          return callback(err)
+        }
+        results = results || null


shouldn't this be?

results = results || []

vasco-santos · 2018-09-28T09:37:35Z

test/content-routing.node.js

+      })
+
+      it('nodeE.contentRouting.findProviders for existing record', (done) => {
+        nodeE.contentRouting.findProviders(cid, 5000, (err, providers) => {


Use themaxTimeout instead of 5000, as we intend to deprecate it in the future.

vasco-santos · 2018-09-28T09:37:46Z

test/content-routing.node.js

+      it('nodeC.contentRouting.findProviders for non existing record (timeout)', (done) => {
+        const cid = new CID('QmTp9VkYvnHyrqKQuFPiuZkiX9gPcqj6x5LJ1rmWuSnnnn')
+
+        nodeE.contentRouting.findProviders(cid, 5000, (err, providers) => {


Use themaxTimeout instead of 5000, as we intend to deprecate it in the future.

vasco-santos · 2018-09-28T09:39:09Z

test/content-routing.node.js

-        expect(err).to.not.exist()
-        expect(providers).to.have.length(0)
-        done()
+        nodeA.contentRouting.findProviders('a cid', 5000, (err, results) => {


Use themaxTimeout instead of 5000, as we intend to deprecate it in the future.

vasco-santos · 2018-09-28T09:39:33Z

test/content-routing.node.js

+        const dhtStub = sinon.stub(nodeA._dht, 'findProviders').callsArgWith(2, null, [])
+        const delegateStub = sinon.stub(delegate, 'findProviders').callsArgWith(2, null, results)
+
+        nodeA.contentRouting.findProviders('a cid', 5000, (err, results) => {


Use themaxTimeout instead of 5000, as we intend to deprecate it in the future.

jacobheun · 2018-09-28T11:13:33Z

@vasco-santos I pushed updates from your feedback. I also needed to re-work some of the code in the docs to fix linting there. The logic is unchanged though.

vasco-santos · 2018-09-28T13:39:31Z

Looks good to me!

Replaces [err-code](https://github.com/IndigoUnited/js-err-code/blob/master/index.js) with [CodeError](libp2p/js-libp2p-interfaces#314) Related: [js-libp2p#1269](libp2p#1269) Changes - removes err-code from dependencies - adds @libp2p/interfaces@3.2.0 to dependencies - uses CodeError in place of err-code

Increases default inactivity timeout to 5 minutes. closes libp2p#239

## [6.0.9](libp2p/js-libp2p-tcp@v6.0.8...v6.0.9) (2023-01-17) ### Bug Fixes * increase default socket close timeout ([libp2p#242](libp2p/js-libp2p-tcp#242)) ([a64ba41](libp2p/js-libp2p-tcp@a64ba41)), closes [libp2p#239](libp2p/js-libp2p-tcp#239) ### Trivial Changes * replace err-code with CodeError ([libp2p#240](libp2p/js-libp2p-tcp#240)) ([5c44562](libp2p/js-libp2p-tcp@5c44562)), closes [js-libp2p#1269](libp2p#1269)

## [7.1.2](libp2p/js-libp2p-mplex@v7.1.1...v7.1.2) (2023-03-21) ### Trivial Changes * replace err-code with CodeError ([libp2p#242](libp2p/js-libp2p-mplex#242)) ([8d58a3b](libp2p/js-libp2p-mplex@8d58a3b)), closes [js-libp2p#1269](libp2p#1269) * Update .github/workflows/semantic-pull-request.yml [skip ci] ([54de88d](libp2p/js-libp2p-mplex@54de88d)) * Update .github/workflows/semantic-pull-request.yml [skip ci] ([df03e8d](libp2p/js-libp2p-mplex@df03e8d)) * Update .github/workflows/semantic-pull-request.yml [skip ci] ([9c3f235](libp2p/js-libp2p-mplex@9c3f235)) ### Dependencies * **dev:** bump typescript from 4.9.5 to 5.0.2 ([libp2p#256](libp2p/js-libp2p-mplex#256)) ([a3590af](libp2p/js-libp2p-mplex@a3590af)) * **dev:** Upgrade aegir to 38.1.7 ([libp2p#257](libp2p/js-libp2p-mplex#257)) ([e0bf45a](libp2p/js-libp2p-mplex@e0bf45a))

ghost assigned jacobheun Aug 22, 2018

ghost added the status/in-progress In progress label Aug 22, 2018

jacobheun commented Aug 22, 2018

View reviewed changes

jacobheun force-pushed the feat/peer-content-routing branch from ae3d5b9 to 31df87d Compare August 23, 2018 11:48

This was referenced Sep 19, 2018

⚡️ 0.24 RELEASE 🚀 #249

Closed

feat: add timeout defaults libp2p/js-libp2p-delegated-peer-routing#6

Merged

feat: add timeout defaults libp2p/js-libp2p-delegated-content-routing#6

Merged

jacobheun force-pushed the feat/peer-content-routing branch 2 times, most recently from a0ae173 to 018a571 Compare September 24, 2018 20:58

jacobheun requested a review from vasco-santos September 27, 2018 14:39

jacobheun changed the title ~~WIP: Feat/peer content routing~~ Delegated Peer and Content routing Sep 27, 2018

jacobheun changed the title ~~Delegated Peer and Content routing~~ Delegated Peer and Content Routing Sep 27, 2018

jacobheun mentioned this pull request Sep 27, 2018

gateway: expose commands for js-libp2p delegated routing ipfs/kubo#4595

Closed

vasco-santos suggested changes Sep 28, 2018

View reviewed changes

vasco-santos approved these changes Sep 28, 2018

View reviewed changes

jacobheun force-pushed the feat/peer-content-routing branch 2 times, most recently from bfb527c to f05e612 Compare October 1, 2018 15:29

jacobheun added 10 commits October 4, 2018 15:32

feat: allow for configuring content and peer routing

58900a5

test: clean up the peer routing tests

870d675

test: add tests for delegate peer routing

50b30a0

test: add jenkins module override

0ba0a64

fix: linting and bad test

e9900db

test: module change

df374a0

test: add tests for content routing

9285e26

test: add error case tests

86ab1a0

feat: support multiple peer and content routing modules

59bc0a0

fix: linting

fb4054a

jacobheun added 8 commits October 4, 2018 15:32

chore: update routing modules and test params

ef9961a

chore: update deps

194511d

feat: add timeout support to peer and content routing

f56b888

docs: update readme

8d5f410

fix: update router/dht timeout param to be options

d8d3eaa

docs: add delegated routing example

03dc652

fix(docs): linting

b84ba05

feat: add err-code and cleanup logic

8128ae0

jacobheun force-pushed the feat/peer-content-routing branch from f05e612 to 8128ae0 Compare October 4, 2018 13:34

refactor: remove unneeded line

19d8cdf

jacobheun merged commit a95389a into master Oct 19, 2018

ghost removed the status/in-progress In progress label Oct 19, 2018

jacobheun deleted the feat/peer-content-routing branch October 19, 2018 14:28

maschad pushed a commit to maschad/js-libp2p that referenced this pull request Jun 21, 2023

fix: increase default socket close timeout (libp2p#242)

a64ba41

Increases default inactivity timeout to 5 minutes. closes libp2p#239

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Delegated Peer and Content Routing #242

Delegated Peer and Content Routing #242

jacobheun commented Aug 22, 2018 •

edited

Loading

jacobheun Aug 22, 2018

daviddias Aug 22, 2018

jacobheun commented Sep 27, 2018

vasco-santos left a comment

vasco-santos Sep 27, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

jacobheun Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

vasco-santos Sep 28, 2018

jacobheun commented Sep 28, 2018

vasco-santos commented Sep 28, 2018

Delegated Peer and Content Routing #242

Delegated Peer and Content Routing #242

Conversation

jacobheun commented Aug 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacobheun commented Sep 27, 2018

vasco-santos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacobheun commented Sep 28, 2018

vasco-santos commented Sep 28, 2018

jacobheun commented Aug 22, 2018 •

edited

Loading