feat: Circuit Relay #1063

dryajov · 2017-11-08T10:08:49Z

Remaining tasks:

Fix Node.js tests
update ipfsd-ctl to support spawning js and go nodes
add ability to ipfsd-ctl to spawn remote js and go nodes to enable browser testing
Add a test for Relay On without having any transport attached. Caught a big hairy bug here #1072 (comment)
Complete interop tests in ipfs/interop#6
Add documentation on how to enable Relay

dryajov · 2017-11-09T17:06:13Z

Consolidate the 3 ways in which a daemon can be spawned

I did a review of all the different ways we have to spawn go and js daemons as well as past conversations on IRC

[2017-09-04 08:38:04] <daviddias> dryajov: the issue on ipfsd-ctl suggests that we were already unhappy with the fragmentation. Why not evolve ipfsd-ctl to support spawning js-ipfs nodes as well?

and it seems like, what we're looking for is actually extending the ipfsd-ctl daemon to spawn js nodes as well?

Create browser interop tests

It seems like the best course of action to get tests in browsers across the board is to port the code from http-api/test/ipfs-factory which allows spawning nodes remotely (using a hapi backend).

Given the above notes, I believe the tasks should be expanded to:

Do the required modifications to ipfsd-ctl to support spawning both go and js nodes
Extract and generalize the code under http-api/test/ipfs-factory. This will allow us to run our tests seamlessly in several environments - node, browser (electron?), etc, as well as reusing them across other projects.

@diasdavid @victorbjelkholm

daviddias · 2017-11-09T18:27:08Z

upgrading ipfsd-ctl

Yes! Let's do that

port the code from http-api/test/ipfs-factory

Learn how https://github.com/ipfs/js-ipfs-api/tree/master/test/ipfs-factory is doing it. Ideally this should be all ipfd-ctl

dryajov · 2017-11-09T18:29:17Z

@diasdavid woot! Awesome, I'll do that.

richardschneider · 2017-11-09T23:25:36Z

@dryajov I was just looking at ipfs-ctl. As usual, it does not run on windows. I'll be happy to get it running on windows, once you do your consolidation.

dryajov · 2017-11-10T03:33:54Z

@richardschneider that would be great!

richardschneider · 2017-11-10T03:59:53Z

Raised windows interop issue.

daviddias · 2017-11-10T08:53:09Z

@richardschneider ipfsd-ctl does run on Windows, in fact it was one of the first modules to get support :) -- https://ci.appveyor.com/project/diasdavid/js-ipfsd-ctl-a9ywu -- thanks to @thisconnect :)

richardschneider · 2017-11-10T08:56:37Z

@diasdavid as usually you are correct. I'll close the issue.

daviddias · 2017-11-11T06:47:18Z

Compatability table

test	goD relay	jsD relay	jsB relay
goD⚡️goD	✅	✅	✅
goD⚡️jsD	✅	✅	✅
goD⚡️jsB	✅	✅	✅
jsD⚡️goD	✅	✅	✅
jsD⚡️jsD	✅	✅	✅
jsD⚡️jsB	✅	✅	✅
jsB⚡️goD	✅	✅	✅
jsB⚡️jsD	✅	✅	✅
jsB⚡️jsB	✅	✅	❌

daviddias · 2017-11-11T06:49:22Z

@dryajov brought back the compatibility table and added a column for when the browser acts as a Relay #1063 (comment)

dryajov · 2017-11-11T07:04:30Z

@diasdavid thanks!

daviddias · 2017-11-12T11:37:37Z

EDIT: moved this tasks to the first comment (dryajov)

~~- [ ] Add a test for Relay On without having any transport attached. Caught a big hairy bug here #1072 (comment)~~
~~- [ ] Add documentation on how to enable Relay~~

dryajov · 2017-11-13T10:11:08Z

Add a test for Relay On without having any transport attached. Caught a big hairy bug here #1072 (comment)

@diasdavid, I believe all we vant to do is to set the dafault flag for EXPERIMENTAL.Swarm.DisableRelay to true, I see that you changed the options and now they don't match Go's -

js-ipfs/src/core/components/libp2p.js

Lines 24 to 30 in 104ef1e

    
           relay: { 
        
             enabled: !get(config, 'EXPERIMENTAL.Swarm.DisableRelay', false), 
        
             hop: { 
        
               enabled: get(config, 'EXPERIMENTAL.Swarm.EnableRelayHop', false), 
        
               active: get(config, 'EXPERIMENTAL.Swarm.RelayHopActive', false) 
        
             } 
        
           }

I also think that in addition to this, there should be a check to skip circuit if swarm doesn't have any transports registered (that is skip it in swarm proper).

daviddias · 2017-11-13T10:36:29Z

@dryajov EXPERIMENTAL flags are not the same as things in the CONFIG file. If you want to have that information in both places that is fine but for now, js-ipfs won't have Relay enabled by default until it is properly tested (i.e this PR finished)

dryajov · 2017-11-13T10:45:26Z

@diasdavid I agree, we should disable circuit for now (i.e. enabled: false). But I don't think we should change the config flags, since they stop matching Go's at that point.

dryajov · 2017-11-13T10:59:09Z

I think I know what you're saying now - yes, we should keep the config flags and the EXPERIMENTAL flags the same and also disable relay by default with enabled: !get(config, 'EXPERIMENTAL.Swarm.DisableRelay', true),.

dryajov · 2017-11-18T19:45:42Z

@diasdavid this should address dialing when no transports are registered - libp2p/js-libp2p-switch#236

daviddias · 2017-11-24T11:56:13Z

@dryajov I know you are working on upgrading ipfsd-ctl, just wanted to suggest that it would be good to fix the issues to make the Circuit Relay tests in test/core/circuit-relay.spec.js pass first and the same goes for the Interop tests that are already there. The ipfsd-ctl upgrade is just needed for some of the tests.

dryajov · 2017-11-24T12:19:12Z

@diasdavid I agree, ipfsd-ctl will mostly help covering the browser based cases. I'll fix the tests we currently have.

daviddias · 2017-11-29T16:29:37Z

Hi @dryajov, mind rebasing master onto this branch. CI should be green (at least Travis and Circle)

daviddias · 2017-11-29T16:32:07Z

.aegir.js

@@ -18,7 +18,8 @@ function start (done) {
      (cb) => js([`${base}/10012`, `${base}/20012/ws`], true, 31012, 32012, cb),
      (cb) => js([`${base}/10013`, `${base}/20013/ws`], true, 31013, 32013, cb),
      (cb) => js([`${base}/10014`, `${base}/20014/ws`], true, 31014, 32014, cb),
-      (cb) => js([`${base}/10015`, `${base}/20015/ws`], true, 31015, 32015, cb)
+      (cb) => js([`${base}/10015`, `${base}/20015/ws`], true, 31015, 32015, cb),
+      (cb) => go([`${base}/10027`, `${base}/20027/ws`], true, 33027, 44027, cb), // we need this for circuit for now


Not true, you should move the interop test to the interop suite

Interop tests don't run in browsers right now, I thought we wanted some browser coverage?

daviddias · 2017-11-29T16:32:07Z

test/core/circuit-relay.spec.js

-  // TODO: 1) figure out why this test hangs randomly
-  // TODO: 2) move this test to the interop batch
-  it.skip('node1 <-> goRelay <-> node2', (done) => {
+  it('node1 <-> goRelay <-> node2', (done) => {


Please move this test to the interop suite. That was what was in the TODO

I can move those to the interop tests, but we'll lose any sort of browser coverage until ipfsd-ctl is ready.

daviddias · 2017-11-29T16:32:07Z

test/core/circuit-relay.spec.js

      ], done)
    })
  })

-  it('node1 <-> jsRelay <-> node2', (done) => {
+  it.skip('node1 <-> jsRelay <-> node2', (done) => {


dryajov · 2017-11-29T17:42:31Z

@diasdavid the tests in test/core/circuit-relay.spec.js are already part of the interop suit, the only reason those were added is to get some sort of browser coverage, since interop tests don't run in the browser.

daviddias · 2017-11-30T06:31:50Z

since interop tests don't run in the browser.

You can run interop tests in the browser too, it uses aegir today, it follows the same rules

This reverts commit 067c9b0.

daviddias · 2018-03-16T22:04:41Z

I'm going ahead and merge this one. Let's save the fireworks for when ipfs/interop#6 is merged though :) Anyways, great work! :D It took a while but it got done! :)

dryajov · 2018-03-16T22:13:26Z

Weeeeee!!!!! ❤️

nezzard · 2018-04-14T16:00:24Z

Why i get "Circuit not enabled" after ipfs.swarm.connect ?
repo: '.repo' + Math.random(), start: true, EXPERIMENTAL: { pubsub: true, relay: { enabled: true, hop: { enabled: true } } }

ghost assigned dryajov Nov 8, 2017

ghost added the status/in-progress label Nov 8, 2017

daviddias changed the title ~~fix: failing circuit interop tests (wip)~~ Complete the Circuit Relay Endeavour Nov 8, 2017

This was referenced Nov 8, 2017

Consolidate node starting in tests, for go and js, under one interface. #1062

Closed

Awesome Endeavour: Circuit Relay #830

Merged

richardschneider mentioned this pull request Nov 10, 2017

Windows Support #1017

Closed

dryajov force-pushed the feat/circuit-interop branch from e5906ef to af39873 Nov 24, 2017

daviddias requested changes Nov 29, 2017

View changes

daviddias and others added 19 commits Feb 15, 2018

feat: latest libp2p + other deps. Fix bugs in tests along the way

7de9de2

tutorial

11e3b4f

tutorial lint

5dbe97c

feat: read relay config from options and config file

8af1f78

docs: tutorial

64fc128

docs: clarifying specific relay addrs

9973578

docs: more clarifications

2425ae4

docs: more clarifications

0483091

docs: more clarifications

1fa52f8

docs: changes based on robs review

80a0b7d

Add minor grammar/spelling fixes to relay doc

3b47e5d

docs: small change

e0edd66

docs: expanded closing section

d1b0180

docs: more suggestions from Mr0grog

c9705b1

docs: more improvements and typos

5cee7e6

doc: minor corrections

1a99d65

Revert "docs: adding circuit relay tutorial (wip)"

4dba67c

This reverts commit 067c9b0.

wip

9f51067

docs: tutorial fixes

Loading status checks…

f265445

dryajov force-pushed the feat/circuit-interop branch from e10fccc to f265445 Mar 16, 2018

daviddias approved these changes Mar 16, 2018

View changes

ghost removed the status/in-progress label Mar 16, 2018

daviddias deleted the feat/circuit-interop branch Mar 16, 2018

ya7ya mentioned this pull request Apr 4, 2018

activating ipfs relay Paratii-Video/paratii-js#182

Merged

norzak mentioned this pull request Apr 15, 2018

WebTorrent DHT integration webtorrent/webtorrent#1152

Open

ipfs / js-ipfs

feat: Circuit Relay #1063

feat: Circuit Relay #1063

dryajov commented Nov 8, 2017 •

edited by daviddias

dryajov commented Nov 9, 2017

daviddias commented Nov 9, 2017

dryajov commented Nov 9, 2017

richardschneider commented Nov 9, 2017 •

edited

dryajov commented Nov 10, 2017

richardschneider commented Nov 10, 2017

daviddias commented Nov 10, 2017

richardschneider commented Nov 10, 2017

daviddias commented Nov 11, 2017 •

edited by dryajov

daviddias commented Nov 11, 2017

dryajov commented Nov 11, 2017

daviddias commented Nov 12, 2017 •

edited by dryajov

dryajov commented Nov 13, 2017

daviddias commented Nov 13, 2017

dryajov commented Nov 13, 2017

dryajov commented Nov 13, 2017

dryajov commented Nov 18, 2017

daviddias commented Nov 24, 2017

dryajov commented Nov 24, 2017

daviddias commented Nov 29, 2017

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

dryajov commented Nov 29, 2017

daviddias commented Nov 30, 2017

daviddias commented Mar 16, 2018

dryajov commented Mar 16, 2018

nezzard commented Apr 14, 2018 •

edited

ipfs / js-ipfs

feat: Circuit Relay #1063

feat: Circuit Relay #1063

Conversation

dryajov commented Nov 8, 2017 • edited by daviddias

dryajov commented Nov 9, 2017

daviddias commented Nov 9, 2017

dryajov commented Nov 9, 2017

richardschneider commented Nov 9, 2017 • edited

dryajov commented Nov 10, 2017

richardschneider commented Nov 10, 2017

daviddias commented Nov 10, 2017

richardschneider commented Nov 10, 2017

daviddias commented Nov 11, 2017 • edited by dryajov

Compatability table

daviddias commented Nov 11, 2017

dryajov commented Nov 11, 2017

daviddias commented Nov 12, 2017 • edited by dryajov

dryajov commented Nov 13, 2017

daviddias commented Nov 13, 2017

dryajov commented Nov 13, 2017

dryajov commented Nov 13, 2017

dryajov commented Nov 18, 2017

daviddias commented Nov 24, 2017

dryajov commented Nov 24, 2017

daviddias commented Nov 29, 2017

This comment has been minimized.

daviddias Nov 29, 2017 Member

This comment has been minimized.

dryajov Nov 29, 2017 Author Member

This comment has been minimized.

daviddias Nov 29, 2017 Member

This comment has been minimized.

dryajov Nov 29, 2017 Author Member

This comment has been minimized.

daviddias Nov 29, 2017 Member

dryajov commented Nov 29, 2017

daviddias commented Nov 30, 2017

daviddias commented Mar 16, 2018

dryajov commented Mar 16, 2018

nezzard commented Apr 14, 2018 • edited

dryajov commented Nov 8, 2017 •

edited by daviddias

richardschneider commented Nov 9, 2017 •

edited

daviddias commented Nov 11, 2017 •

edited by dryajov

daviddias commented Nov 12, 2017 •

edited by dryajov

daviddias Nov 29, 2017
Member

dryajov Nov 29, 2017
Author Member

daviddias Nov 29, 2017
Member

dryajov Nov 29, 2017
Author Member

daviddias Nov 29, 2017
Member

nezzard commented Apr 14, 2018 •

edited