New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`ipfs p2p` feedback thread #3994

Open
whyrusleeping opened this Issue Jun 19, 2017 · 16 comments

Comments

Projects
None yet
7 participants
@whyrusleeping
Member

whyrusleeping commented Jun 19, 2017

For everyone trying out the ipfs p2p command, please add any and all feedback here, this includes (but is not limited to) bugs, opinions, interface critiques, potential usecases, and feature requests.

To enable the command, you will need to run:
ipfs config --json Experimental.Libp2pStreamMounting true
And restart your daemon.

Basic usage of ipfs p2p is as follows:

First, open up a p2p listener on one ipfs node: `p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101` This will create a libp2p protocol handler for a protocol named `p2p-test` and then open up a tcp socket locally on port 10101. Your application can now connect to that port to interact with the service.

On the other ipfs node, connect to the listener on node A
ipfsi 1 p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102
Now the ipfs nodes are connected to eachother. This command also created another tcp socket for the other end of your application to connect to. Once the other end connects to that socket, both sides of your application should be able to communicate, with their traffic being routed through libp2p.

The easiest way to try this out is with netcat.

by @magik6k:
The feature is getting refactored in some breaking ways. For updated guide see https://github.com/ipfs/go-ipfs/blob/4bafbf1cdef7d6d0041154ee03efb02f2ef0685b/docs/experimental-features.md#ipfs-p2p.

What changed: #3994 (comment)

@magik6k

This comment has been minimized.

Member

magik6k commented Jun 19, 2017

What I noticed missing from one of the earlier PRs is the ability to tell to which peer connected to a listener. Earlier PRs had a switch that caused ipfs to send <PeerID>\n at the beginning of each incoming stream. This should be pretty simple to implement.

@ReisenB

This comment has been minimized.

ReisenB commented Jun 28, 2017

Is this planned to be accessible through the browser with js-ipfs-api?

@whyrusleeping

This comment has been minimized.

Member

whyrusleeping commented Jun 29, 2017

@magik6k

This comment has been minimized.

Member

magik6k commented Jun 29, 2017

I'd need to look into how websockets are currently handled in go-ipfs. For dialing this should be relatively simple to do, though listen mode would probably have to use some sort of stream muxer that has js implementation. This should fit quite well into the existing code.

@jdgcs

This comment has been minimized.

jdgcs commented Jul 31, 2017

No opening port 10101, is that right?

liu@nas:~/ipfs/go-ipfs % ./ipfs p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101

{"Address":"/ip4/127.0.0.1/tcp/10101","Protocol":"/p2p/p2p-test"}

liu@nas:~/ipfs/go-ipfs % ./ipfs p2p listener ls

/ip4/127.0.0.1/tcp/10101 /p2p/p2p-test

liu@nas:~/ipfs/go-ipfs % netstat -an | grep 10101

liu@nas:~/ipfs/go-ipfs %

@magik6k

This comment has been minimized.

Member

magik6k commented Jul 31, 2017

This creates listening libp2p handler and when p2p connection is incoming it(ipfs daemon) then opens a tcp connection to a given address and proxies the data. This seems counterintutive at first, but it allows for transparent connection proxying over libp2p

@origama

This comment has been minimized.

origama commented Nov 11, 2017

It would be nice to be able to set the listeners in the config file, so that the daemon could create them at booting time. It makes sense especially if you run IPFS as an init/systemd/docker unit.

@magik6k

This comment has been minimized.

Member

magik6k commented Nov 11, 2017

I don't think using config for that is a good idea. What may work is extracting whole ipfs p2p into a separate program as it doesn't really fit ipfs (imo), this program could then talk to ipfs daemon and reuse its libp2p instance or be standalone.

@origama

This comment has been minimized.

origama commented Nov 11, 2017

And adding up to my previous comment, I noticed a different behavior between the listener and the streamer.

Considering the case in which you start a listener for the protocol ipfs-test on peerA attached to the local port 80, and then you start a streamer on peerB to connect to peerA[ipfs-test] and listening on his local port 8080.

Now, with this setup, no matter if the streamer will disconnect or if the service on port 80 on peerA will die, the listener will still be up and running.

The streamer instead dies after whatever client running on peerB disconnects from port 8080.

Maybe it's expected, but it would be nice to have the streamer keep listening for more than one connection, or at least having a parameter (like a sort of nc -k), and maybe a config entry for this too.

@whyrusleeping whyrusleeping referenced this issue Jun 3, 2018

Merged

Refactor `ipfs p2p` #4929

4 of 5 tasks complete
@magik6k

This comment has been minimized.

Member

magik6k commented Jun 3, 2018

#4929 includes api-breaking changes, it will likely be released with the new version of go-ipfs.

EDIT: This changed a bit while the PR was open, this is the current version:

For updated how-to see https://github.com/ipfs/go-ipfs/blob/b84c386124441c1ecf261d7c4dbbc72112632299/docs/experimental-features.md#ipfs-p2p

What changes:

  • internal changes:
    • listener is now referred to as 'remote listener'
    • stream(-er) is now referred to as 'local listener'
  • we don't stop listening for local connections after accepting a single connection.
  • ipfs p2p stream ls output now contains more useful output, first address is always the initiator address.
  • ipfs p2p listener ls is now ipfs p2p ls
  • ipfs p2p listener close is now ipfs p2p close
  • Protocol names have to be prefixed with /x/ and are now just passed to libp2p as handler name. Previous version did this 'under the hood' and with /p2p/ prefix. There is a --allow-custom-protocol flag which allows you to use any libp2p protocol name.
  • ipfs p2p listener open and ipfs p2p stream dial moved to ipfs p2p [listen/forward]
    • ipfs p2p forward /x/[protocolName] [listen multiaddr] [target multiaddr]
    • For listener open, instead of:
      • ipfs p2p listener open p2p-test /ip4/127.0.0.1/tcp/10101
      • you now do
      • ipfs p2p listen /x/p2p-test /ip4/127.0.0.1/tcp/10101
    • For stream dial instead of:
      • ipfs p2p stream dial $NODE_A_PEERID p2p-test /ip4/127.0.0.1/tcp/10102
      • you now do
      • ipfs p2p forward /x/p2p-test /ip4/127.0.0.1/tcp/10102 /ipfs/$NODE_A_PEERID
@DavidHuie

This comment has been minimized.

DavidHuie commented Jun 6, 2018

@magik6k what's stopping ipfs p2p from supporting UDP?

@magik6k

This comment has been minimized.

Member

magik6k commented Jun 6, 2018

With #4929 you will be able to pass streams to udp services. You still won't be able to listen for udp connections as local listener uses go-multiaddr-net.Listen directly, and it only supports listening on tcp now.

There are some problems with supporting udp properly:

  • Need to track connections if we want 2-way communication (this isn't that hard, maybe slightly harder if one wants this to be any fast)
  • Libp2p operates on streams, udp is message oriented. There are 2 ways to solve this:
    • wrap udp packets into messages (basically size+data piped into stream)
      • IMO the wrong way to do this
    • give libp2p the concept of messages
      • There are some plans for message oriented (+unreliable) transports/muxers
        • Can't find any specific issue
      • This would be ideal for udp
      • Also, useful in other parts of libp2p.
@DavidHuie

This comment has been minimized.

DavidHuie commented Jun 6, 2018

@magik6k Is that something that you guys are open to having the community implement?

I'm designing a system that uses IPFS to create a peer-to-peer VPN. All of the VPN protocols I'm considering use UDP since TCP is implemented at a higher level networking stack.

@DavidHuie

This comment has been minimized.

DavidHuie commented Jun 6, 2018

Also, net.Conn in Go implements UDP connections as streams, so it wouldn't be a terrible way to implement that...

@whyrusleeping

This comment has been minimized.

Member

whyrusleeping commented Jun 7, 2018

@DavidHuie We're definitely open to having the community help there. It's something that will be tricky to get right, adding a message oriented layer to libp2p won't be trivial. If you want to get the ball rolling, open an issue in libp2p/libp2p about it, describe use cases, and tag people. We can start to brainstorm thoughts there.

I know that @lgierth was planning on working on that soon, he's out on vacation for another week or so. When he gets back maybe you two can work on pushing that?

@Stebalien

This comment has been minimized.

Contributor

Stebalien commented Sep 14, 2018

The refactor has been merged (massive thanks to @magik6k for keeping with this). Go pull master and try it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment