hidservd: Solution to the TBB vs. system Tor problem

[micahflee]

OnionShare is a simple app, but it turns out there are some gnarly issues related to picking what tor to use, how to package it for Windows and OSX, what permissions need to be required, etc.

Right now OnionShare requires Tor Browser Bundle to be running in the background, so it can use the tor process that's included there. This is great for Windows and OSX because there isn't currently an officially supported way to install a system Tor on those platforms where you get auto-updates, etc. And OSX and Windows don't have sane dependency systems, so you can't just mark "tor" as a dependency of "onionshare" like you can in Linux.

This is annoying though because you need to install a second piece of software and open it separately. And in Linux, where system Tors are easy and officially supported, this isn't necessary. Requiring TBB might also prevent OnionShare from being included in Debian's main repo (will be included in contrib instead). And requiring TBB isn't practical in Tails, where the user must use the system Tor, or in Whonix or Qubes AnonVMs, where the Tor process is running in a different VM than the OnionShare process. (Related issues: #130, #27)

I describe the problem with just relying on a system Tor in Linux in this debian bug:

Unfortunately OnionShare doesn't work with the system tor without changing a bunch of stuff around, due to limitations in tor. Instead it's recommended that you install Tor Browser Bundle (perhaps using the torbrowser-launcher Debian package) and run it before running OnionShare.

When you connect to the control port, you start a hidden service by setting the values HiddenServiceDir and HiddenServicePort. The control port doesn't respond with the .onion address, which is an important thing to know. The only way to learn that is to look in HiddenServiceDir for a file called "hostname" and read its contents. If you're using a system tor, that file is only readable by the debian-tor user, which means an unprivileged user can't learn the .onion address.

Also (at least I believe) in the default Debian torrc file the control port isn't enabled. So users would have to edit their torrc before being able to use OnionShare.

If the user has Tor Browser open, it will be running its own separate Tor process as the current user. So when it writes a hostname file to HiddenServiceDir, that file is readable by the current user.

One way of solving this problem would be to make OnionShare launch its own, separate tor process and use that (this has the added benefit of bypassing #87 too). I don't want to do this for Windows and OSX though, because I don't want to bundle a copy of Tor with OnionShare, for a variety of reasons. But it could totally work for Linux -- except not for the Tails, Whonix and Qubes AnonVM use-cases listed above. Also, if I did this I'd have to create a UX for configuring Tor with bridges, etc., where if you depend on TBB then the user has already configured it to their needs.

Another way of solving this, that would solve the problem for all Linux use-cases including normal desktop distros like Debian, Ubuntu, etc., as well as the high security edge cases like Tails, Whonix, and Qubes AnonVMs, would be to make an extra daemon that runs in the background (hidservd) that's responsible for starting and stopping hidden services (and would have the added benefit of bypassing #87).

It could work like this. There can be a hidservd-server package that listens on either a unix socket or a TCP socket. Commands it accept would include:

• start_hs(port): starts a hidden service on that port and returns the .onion address (and changes iptables rules to let tor connect to that port, if necessary, like in Tails)
• stop_hs(port): stop the hidden service (and change iptables rules back, if necessary)

There can be a python package hidservd-client, that can connect to a hidservd-server either through a unix socket (in the case where tor and onionshare are run in the same VM, like in Debian, Ubuntu, Fedora, etc.) or a TCP socket (in the case of Whonix or Qubes).

In this way, OnionShare in OSX and Windows would require TBB, like it currently does. OnionShare in desktop Linux distros would depend on a system Tor and hidservd-server running in the background, but from a UX perspective it would all just work, without requiring TBB open or having to elevate privileges to root. This would also be the case for Tails. And Whonix and Qubes AnonVM users would just need to have their OnionShare desktop shortcuts open something like onionshare-gui --hidservd=[IP]:[port] and then it, too, would just work.

This also would have the added benefit of solving this problem for similar projects that will inevitably run into it, such as Ricochet https://github.com/ricochet-im/ricochet, because they could depend on hidservd-server too.

[ThomasWaldmann]

just as an additional note:

if you have torbrowser from TBB running AND also a system tor service, then onionshare gets confused and chooses to work with the system tor and then stumbles over permission issues:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/onionshare_gui/onionshare_gui.py", line 123, in start_server
self.app.start_hidden_service(gui=True)
File "/usr/lib/python2.7/dist-packages/onionshare/onionshare.py", line 104, in start_hidden_service
controller.authenticate()
File "/usr/lib/python2.7/dist-packages/stem/control.py", line 733, in authenticate
stem.connection.authenticate(self, _args, *_kwargs)
File "/usr/lib/python2.7/dist-packages/stem/connection.py", line 414, in authenticate
raise auth_exc
stem.connection.UnreadableCookieFile: Authentication failed: '/var/run/tor/control.authcookie' doesn't exist

(it does exist, but is only accessible for debian-tor user)

Stopping the system tor service helps, then it chooses to work with the tor from TBB.

[u451f]

Hi Thomas,

I cannot confirm the issue you are reporting. I run both TBB, onionshare (latest release) and a system tor on Debian Wheezy/Jessie (tested on both systems). Which version of OnionShare did you use?

[ThomasWaldmann]

I cloned the git repo some hours ago and built the deb package and installed it.
The package dependencies I installed from ubuntu 14.04.

[adrelanos]

In future, hidden services can be created without writing files to the file system using Tor's ControlPort only.

For reference, see:
Ephemeral Hidden Services via the Control Port:

• https://lists.torproject.org/pipermail/tor-dev/2015-February/008279.html
• https://trac.torproject.org/projects/tor/ticket/6411

Would that help? Could that avoid a hidservd-server package?

For the Whonix, perhaps Tails [...] use case, onionshare [and others] could then just connect to an already existing Tor ControlPort port that is supposed to be defined by the distribution (Whonix, Tails [...]).

[burdges]

Just fyi, Pond tries the system Tor on port 9050 first and then tries the TBB on port 9150 :
https://github.com/agl/pond/blob/master/client/client.go#L690
I'd consider this approach sufficient for most Tor applications that do no require a hidden service.

In the hidden service cases like here, either (a) the debian-tor approach should be corrected by a config option that lets users create hidden services, or else (b) the control port should be come the official way for users to create hidden services.

[micahflee]

I'm happy to say that we don't need a hidservd anymore, thanks to ephemeral hidden services (#178)