Tor onion integration #37
7 participants
this would be pretty cool!
OK I've written the go-multiaddr Tor onion address decoding:
jbenet/go-multiaddr#13
This sounds like a good plan for location-anonymous tor users to have a persistent identity and announce themselves to peers, but it doesn't cover the use case of anonymously (unlinkably?) accessing data from ipfs.
For that we'll need a mode that doesn't transmit any linkable identifiers (which includes not announcing pieces to the DHT).
Today I wrote a TorDialer and OnionListener; This repo also includes examples programs which can be used as an integration test with your system Tor. However it should soon have more ways to use Tor other than just the control port protocol with the system tor:
- new ephemeral onion services via control port
- "just launch me my own tor instance for this Dialer/Listener"
https://github.com/david415/oniondialer
Although oniondialer could be improved, we can do this later now that we have the basic interface implementations working. Therefore the next step should be to see how this Dialer and Listener can fit into go-multiaddr ... Probably oniondialer will have to change slightly to get it to fit into go-multiaddr.
I've been looking closely at the source for go-multiaddr and go-multiaddr-net...
However I'm not sure if Tor onion service multiaddr string should look like this:
/onion/timaq4ygg2iegci7.onion:80
or this
/onion/fufu.onion/tcp/80
Whereas the "readonly Tor-ified" multiaddr looks like this:
/tor/example.com/tcp/80
or should the /tor/ multiaddr not encapsulate and instead look like this?
/tor/example.com:80
What do IPFS developers think of all this?
from chat:
/onion/fufu.onion/ip4/tcp/80/tor/example.com/ip6/tcp/80
Similar to /dns in jbenet/go-multiaddr#7 (comment)
A more explicit, but very bloated option for /tor: /ip4/127.0.0.1/tcp/9050/socks5/dns/example.net/ip4/tcp/80
hi. I chatted with Juan last night on #ipfs and we decided that onion multiaddrs will look like this:
- /onion/timaq4ygg2iegci7/80
Tor doesn't transport IP or TCP... but is a realiable stream transport. We don't need DNS. Tor clients use a local SOCKS proxy; therefore the remote end does the DNS resolution. I hope I've cleared up the confusion regarding this. If not then I'm afraid this ticket will grow too big with my explainations. Perhaps find me on irc instead of commenting here with queries.
For the Tor exit node case I think it should be represented like so:
- /tor/example.com/80
So I think this sort of breaks the multiaddr parser model... and so I made my multiaddr/net dev branches do this:
- /onion/timaq4ygg2iegci7:80
That is the colon separates the embedded onion port number.
we'll already have things like:
/ip4/1.2.3.4/tcp/80/http
/dns/foo.com/A/ip4/tcp/80/http
if we made the restriction that only one component be used, why not then:
/ip4:1.2.3.4/tcp:80/http
/dns:foo.com:A:ip4/tcp:80/http
o/ this is actually harder to impl and does not layer well on a FS approach like 9P or linux proc files, which the former one does
thats not quite what I was getting at, i'm not saying that we require only one component. I'm saying that we combine the onion address and port into one as they both are given meaning by the /onion path element.
Any suggestions for changes to go-multiaddr-net?
Do you not want Tor/Onion related stuff in DialArgs since it is not a thin-waist protocol?
See here:
https://github.com/jbenet/go-multiaddr-net/blob/master/convert.go#L142-L144
@david415 right, DialArgs is for net.Dial mostly.
What we should do is:
- make an onion Dialer + Listener
- make the main Dialer + Listener extensible (so users can "add" protocols they want to allow. key is user adds it because users may not want to allow any dialing, only certain protocols. in this case, regular net dialing should also be done this way as some users would not want to allow arbitrary TCP/UDP dialing...)
(i unfort dont have time to work directly on this until after Sep 15. but can CR)
yes... we have a basic tor dialer + onion listener that we can improve more later
https://github.com/david415/oniondialergo-multiaddr-net needs a "pluggable transport system"...
and i think we can make iterative progress here wherein the first phase for Tor support you can:
- remove all transport plugins
- load the tor plugin
- do stuff with tor + ipfs!!!
However... later it would be nice to have more granular policy for operating heterogeneous ipfs nodes/gateways/proxies.
I'm looking at go-multiaddr-net and go-ipfs to see how things work... and it makes sense for thin-waist protocols... but this type of thing isn't going to work for the OnionListener use case:
https://github.com/ipfs/go-ipfs/blob/master/core/corehttp/corehttp.go#L50-L55
The onion multiaddr isn't enough information to create an onion listener. We need to know the tor process control port in order to create the onion service... then after we auto-create the onion service key material the control port tells us the onion address as well.
Perhaps in this case we need an onion + multiaddr helper function to create a new onion service and it's respective multiaddr? But how to plug it into the normal usage of go-multiaddr-net where the API user expects a multiaddr Listener to be created by only feeding it the multiaddr?
@david415 maybe the listen onion multiaddr could be different, could include more information, like the proxy:
/ip4/127.0.0.1/tcp/<control-port>/onion/<onion-address>
but it still should be advertised to others as:
/onion/<onion-address>
Perhaps in this case we need an onion + multiaddr helper function to create a new onion service and it's respective multiaddr?
yeah that's definitely doable. harder to make things plug in nicely. the other option is to create a "listener-maker" and mount that onto multiaddr, like:
c := maonion.NewController(NewMultiaddr("/ip4/127.0.0.1/tcp/<control-port>"))
c.NewListener(NewMultiaddr("/onion/<onion-addr>"))
// mount onto a multiaddr-net
manet.Listen("/onion/<onion-addr>") // doesn't work
manet.AddProtocol("onion", c)
manet.Listen("/onion/<onion-addr>") // now works
//
but this is kind of "magical" in a way i dont like (i.e. static), probably makes much more sense to rework how multiaddr-net works into something like this:
n := manet.NewNetwork()
n.Add(manet.IP4)
n.Add(manet.IP6)
n.Add(manet.TCP)
n.Add(manet.UDP)
n.Add(onion.Onion)
l, err := n.Listener("/ip4/1.2.3.4/tcp/1234")
c, err := n.Dial("/onion/<onionaddr>")
// with shortcuts like:
n := manet.NewThinWaistNetwork()
n.Add(onion.Onion)"network" is a bit odd name, but does give the idea that these networks are stacked and makes more sense why mulriaddrs work the way they do. "transport" is another possible name, but may not capture the full meaning of "network".
Sorry, let's move multiaddr pluggable-ness discussion to jbenet/go-multiaddr-net#7
hey @david415 -- @whyrusleeping has made transport changes to go-ipfs libp2p and to https://github.com/jbenet/go-multiaddr-net/ -- chances are we can resume the tor work!
does running an ipfs server as a tor hidden service work by default?
I guess there are quite a few dimensions to the intersection of tor and ipfs, and its difficult to tell what works out of the box and what needs development
No. IPFS does not "work by default" as a Tor onion service.
It is not difficult to tell the state of the IPFS + Tor integration because it is utterly non-existent; That's why we have this ticket!
To reiterate what @leif posted: It might work for dialing an IPFS node from a censored location but you better have a custom ipfs build that doesn't scream your identity to everybody in it's vicinity. I hope we can see different flavors of ipfs nodes before 1.0. A privacy preserving client is much needed otherwise every content you share and spread becomes a tracking cookie. I personally already have that barbed-wire in my head stopping me from using it to share music with my family and that's only a 1st world problem.
chatting with Juan Bennet at c-base, Berlin about Tor onion service integration
here's what we've identified as necessary for proper Tor integration:
1. adding
/onionto go-multiaddr -/onion/<onion-key>/ipfs/<ipfs-key>2. adding
/oniondialing to go-multiaddr-net3. make a build of go-ipfs that:
/onionI know plenty of people who'd be willing to run some Tor onion bootstrap nodes.
(edited by @jbenet for links)