net: Automatically create hidden service, listen on Tor #6639

Merged
merged 5 commits into from Nov 12, 2015

Conversation

Projects
None yet
10 participants
@laanwj
Member

laanwj commented Sep 4, 2015

Discussion in #6586.

Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket API, to create and destroy 'ephemeral' hidden services programmatically.
https://stem.torproject.org/api/control.html#stem.control.Controller.create_ephemeral_hidden_service

This means that if Tor is running (and proper authorization is available), bitcoin automatically creates a hidden service to listen on, without user manual configuration. This will positively affect the number of available .onion nodes.

  • When the node is started, connect to Tor through control socket
  • Send ADD_ONION command
  • First time:
    • Make it create a hidden service key
    • Save the key in the data directory for later usage
  • Make it redirect port 8333 to the local port 8333 (or whatever port we're listening on).
  • Keep control socket connection open for as long node is running. The hidden service will
    (by default) automatically go away when the connection is closed.

Adds command line options:

  • -listenonion Automatically create Tor hidden service (default: 1)
  • -torcontrol=<ip>:<port> "Tor control port to use if onion listening enabled (default: 127.0.0.1:9051)

TODO:

  • SAFECOOKIE authentication support (see discussion in #6586)
  • HASHEDPASSWORD authentication: manually provide Tor credentials
  • Logging: most of the more specific logging should go into a debug category

@laanwj laanwj added the P2P label Sep 4, 2015

@jgarzik

This comment has been minimized.

Show comment
Hide comment
@jgarzik

jgarzik Sep 15, 2015

Contributor

I like the feature - concept ACK

Trying to think through whether there are edge cases that persuade us to default this 'off' Leaning towards default-on as presented.

Contributor

jgarzik commented Sep 15, 2015

I like the feature - concept ACK

Trying to think through whether there are edge cases that persuade us to default this 'off' Leaning towards default-on as presented.

@dcousens

This comment has been minimized.

Show comment
Hide comment
@dcousens

dcousens Sep 16, 2015

Contributor

concept ACK

Contributor

dcousens commented Sep 16, 2015

concept ACK

@btcdrak

This comment has been minimized.

Show comment
Hide comment
@btcdrak

btcdrak Sep 23, 2015

Member

concept ACK

Member

btcdrak commented Sep 23, 2015

concept ACK

@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Sep 26, 2015

Member

2015-09-26 00:03:11 [tor] Using COOKIE authentication, reading cookie authentication from /var/run/tor/control.authcookie
2015-09-26 00:03:11 [tor] Authentication cookie not found, is inaccessible, or is not exactly 64 bytes

I have ControlPort 9051 and CookieAuthentication in torrc.

Member

sipa commented Sep 26, 2015

2015-09-26 00:03:11 [tor] Using COOKIE authentication, reading cookie authentication from /var/run/tor/control.authcookie
2015-09-26 00:03:11 [tor] Authentication cookie not found, is inaccessible, or is not exactly 64 bytes

I have ControlPort 9051 and CookieAuthentication in torrc.

@pstratem

This comment has been minimized.

Show comment
Hide comment
@pstratem

pstratem Sep 26, 2015

Contributor

Check the permissions
On Sep 25, 2015 5:07 PM, "Pieter Wuille" notifications@github.com wrote:

2015-09-26 00:03:11 [tor] Using COOKIE authentication, reading cookie
authentication from /var/run/tor/control.authcookie
2015-09-26 00:03:11 [tor] Authentication cookie not found, is
inaccessible, or is not exactly 64 bytes

I have ControlPort 9051 and CookieAuthentication in torrc.


Reply to this email directly or view it on GitHub
#6639 (comment).

Contributor

pstratem commented Sep 26, 2015

Check the permissions
On Sep 25, 2015 5:07 PM, "Pieter Wuille" notifications@github.com wrote:

2015-09-26 00:03:11 [tor] Using COOKIE authentication, reading cookie
authentication from /var/run/tor/control.authcookie
2015-09-26 00:03:11 [tor] Authentication cookie not found, is
inaccessible, or is not exactly 64 bytes

I have ControlPort 9051 and CookieAuthentication in torrc.


Reply to this email directly or view it on GitHub
#6639 (comment).

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Oct 2, 2015

Member

@sipa the check was broken, the cookie is 32 bytes not 64. That said, we should have a more specific error message.

Member

laanwj commented Oct 2, 2015

@sipa the check was broken, the cookie is 32 bytes not 64. That said, we should have a more specific error message.

@theuni

View changes

src/torcontrol.cpp
+ if (f == NULL)
+ return false;
+ if (fwrite(data.data(), 1, data.size(), f) != data.size())
+ return false;

This comment has been minimized.

@theuni

theuni Oct 2, 2015

Member

false case needs fclose too

@theuni

theuni Oct 2, 2015

Member

false case needs fclose too

This comment has been minimized.

@laanwj

laanwj Oct 2, 2015

Member

oops good catch

@laanwj

laanwj Oct 2, 2015

Member

oops good catch

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Oct 2, 2015

Member

Can the shutdown detection be changed to work like the httpserver? ie Init calls InterruptTorControl() which starts the immediate shutdown process?

Member

theuni commented Oct 2, 2015

Can the shutdown detection be changed to work like the httpserver? ie Init calls InterruptTorControl() which starts the immediate shutdown process?

+
+bool TorControlConnection::Disconnect()
+{
+ if (b_conn)

This comment has been minimized.

@theuni

theuni Oct 2, 2015

Member

Need to undo the AddLocal here?

@theuni

theuni Oct 2, 2015

Member

Need to undo the AddLocal here?

+ this->disconnected = disconnected;
+
+ // Finally, connect to target
+ if (bufferevent_socket_connect(b_conn, (struct sockaddr*)&connect_to_addr, connect_to_addrlen) < 0) {

This comment has been minimized.

@theuni

theuni Oct 2, 2015

Member

Don't we want logic to retry this every n seconds if it fails? If I launch Bitcoin and Tor a minute later, I would expect that it would be detected and used after a few minutes.

@theuni

theuni Oct 2, 2015

Member

Don't we want logic to retry this every n seconds if it fails? If I launch Bitcoin and Tor a minute later, I would expect that it would be detected and used after a few minutes.

This comment has been minimized.

@laanwj

laanwj Oct 2, 2015

Member

That's what RECONNECT_TIMEOUT_START / RECONNECT_TIMEOUT_EXP is supposed to do. When I last checked, it was working.

@laanwj

laanwj Oct 2, 2015

Member

That's what RECONNECT_TIMEOUT_START / RECONNECT_TIMEOUT_EXP is supposed to do. When I last checked, it was working.

This comment has been minimized.

@theuni

theuni Oct 2, 2015

Member

It looks like that only works on disconnection, though. If the first attempt fails on socket connection and the callbacks never start, I don't see how the reconnect logic kicks in. Am I misreading?

@theuni

theuni Oct 2, 2015

Member

It looks like that only works on disconnection, though. If the first attempt fails on socket connection and the callbacks never start, I don't see how the reconnect logic kicks in. Am I misreading?

This comment has been minimized.

@theuni

theuni Oct 2, 2015

Member

Nevermind, I misremembered this function's return meaning. I thought it was possible to get an immediate failure in some cases, but a quick test shows that's not the case.

@theuni

theuni Oct 2, 2015

Member

Nevermind, I misremembered this function's return meaning. I thought it was possible to get an immediate failure in some cases, but a quick test shows that's not the case.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Oct 2, 2015

Member

(Not for this PR) Looks like it'd be pretty simple to add support for connecting via Unix socket (/var/run/tor/control) as well.

Member

theuni commented Oct 2, 2015

(Not for this PR) Looks like it'd be pretty simple to add support for connecting via Unix socket (/var/run/tor/control) as well.

@theuni

This comment has been minimized.

Show comment
Hide comment
@theuni

theuni Oct 2, 2015

Member

Concept ACK. This looks like a great feature.

Member

theuni commented Oct 2, 2015

Concept ACK. This looks like a great feature.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Oct 3, 2015

Member

@cfields

Can the shutdown detection be changed to work like the httpserver? ie Init calls InterruptTorControl() which starts the immediate shutdown process?

Yes, that sounds better. I have not done this before because it means making the libevent instance thread-safe - Interrupt happens from a different thread. But as we need multi-threaded libevent anyway from the http server that's a bad reason.

(Not for this PR) Looks like it'd be pretty simple to add support for connecting via Unix socket (/var/run/tor/control) as well.

Good idea.

Member

laanwj commented Oct 3, 2015

@cfields

Can the shutdown detection be changed to work like the httpserver? ie Init calls InterruptTorControl() which starts the immediate shutdown process?

Yes, that sounds better. I have not done this before because it means making the libevent instance thread-safe - Interrupt happens from a different thread. But as we need multi-threaded libevent anyway from the http server that's a bad reason.

(Not for this PR) Looks like it'd be pretty simple to add support for connecting via Unix socket (/var/run/tor/control) as well.

Good idea.

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Oct 3, 2015

Member

Ok: addressed all of @theuni's comments (apart from the UNIX socket, which we can add later).
Still have SAFECOOKIE auth support on my TODO list above, it is superior to COOKIE (no potential exposure of data), and COOKIE will eventually be deprecated.
This does require use of HMAC-SHA256. Luckily we have an implementation of that for BIP32, so I think it's doable.

Edit: done

Member

laanwj commented Oct 3, 2015

Ok: addressed all of @theuni's comments (apart from the UNIX socket, which we can add later).
Still have SAFECOOKIE auth support on my TODO list above, it is superior to COOKIE (no potential exposure of data), and COOKIE will eventually be deprecated.
This does require use of HMAC-SHA256. Luckily we have an implementation of that for BIP32, so I think it's doable.

Edit: done

laanwj and others added some commits Aug 25, 2015

net: Automatically create hidden service, listen on Tor
Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket
API, to create and destroy 'ephemeral' hidden services programmatically.
https://stem.torproject.org/api/control.html#stem.control.Controller.create_ephemeral_hidden_service

This means that if Tor is running (and proper authorization is available),
bitcoin automatically creates a hidden service to listen on, without user
manual configuration. This will positively affect the number of available
.onion nodes.

- When the node is started, connect to Tor through control socket
- Send `ADD_ONION` command
- First time:
    - Make it create a hidden service key
    - Save the key in the data directory for later usage
- Make it redirect port 8333 to the local port 8333 (or whatever port we're listening on).
- Keep control socket connection open for as long node is running. The hidden service will
  (by default) automatically go away when the connection is closed.
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Nov 10, 2015

Member

Also mentioned in release notes now.

Member

laanwj commented Nov 10, 2015

Also mentioned in release notes now.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Nov 10, 2015

Contributor

ACK

Tests done:

  • Restart reuses cached onion private key
  • SAFECOOKIE auth
  • HASHEDPASSWORD auth
  • Onion advertisement stops with tor disconnects
  • Onion advertisement starts when tor restarted
Contributor

petertodd commented Nov 10, 2015

ACK

Tests done:

  • Restart reuses cached onion private key
  • SAFECOOKIE auth
  • HASHEDPASSWORD auth
  • Onion advertisement stops with tor disconnects
  • Onion advertisement starts when tor restarted
@sipa

This comment has been minimized.

Show comment
Hide comment
@sipa

sipa Nov 11, 2015

Member

No code review, did a weak test (it correctly reports my Tor version is too old).

Member

sipa commented Nov 11, 2015

No code review, did a weak test (it correctly reports my Tor version is too old).

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 11, 2015

Member

That's great! I will try to look into this soon.

Member

MarcoFalke commented Nov 11, 2015

That's great! I will try to look into this soon.

@petertodd

This comment has been minimized.

Show comment
Hide comment
@petertodd

petertodd Nov 11, 2015

Contributor

utACK squashme commits

Contributor

petertodd commented Nov 11, 2015

utACK squashme commits

+ service = CService(service_id+".onion", GetListenPort(), false);
+ LogPrintf("tor: Got service ID %s, advertizing service %s\n", service_id, service.ToString());
+ if (WriteBinaryFile(GetPrivateKeyFile(), private_key)) {
+ LogPrint("tor", "tor: Cached service private key to %s\n", GetPrivateKeyFile());

This comment has been minimized.

@MarcoFalke

MarcoFalke Nov 11, 2015

Member

Nit: "Wrote service key..."?

@MarcoFalke

MarcoFalke Nov 11, 2015

Member

Nit: "Wrote service key..."?

This comment has been minimized.

@MarcoFalke

MarcoFalke Nov 11, 2015

Member

Also, mind to pass this file through

contrib/devtools/clang-format.py clang-format src/torcontrol.cpp?

@MarcoFalke

MarcoFalke Nov 11, 2015

Member

Also, mind to pass this file through

contrib/devtools/clang-format.py clang-format src/torcontrol.cpp?

This comment has been minimized.

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 11, 2015

Member

Somehow I can set up a -onlynet=onion node and connect to it (sometimes) but it never established outgoing connections:

2015-11-11 22:05:45 tor: ADD_ONION succesful
2015-11-11 22:05:45 tor: Got service ID moheur3skn7jbca4, advertizing service moheur3skn7jbca4.onion:18333
2015-11-11 22:05:45 tor: Cached service private key to /home/test/.bitcoin/testnet3/onion_private_key
2015-11-11 22:05:45 AddLocal(moheur3skn7jbca4.onion:18333,4)
2015-11-11 22:05:46 Cannot connect to t6xj6wilh4ytvcs7.onion:18333: unsupported network
2015-11-11 22:05:46 Cannot connect to 4azvkpo55qaskhjl.onion:18333: unsupported network
2015-11-11 22:05:47 Cannot connect to nkf5e6b7pl4jfd4a.onion:18333: unsupported network
2015-11-11 22:05:48 Cannot connect to 4zhkir2ofl7orfom.onion:18333: unsupported network
Member

MarcoFalke commented Nov 11, 2015

Somehow I can set up a -onlynet=onion node and connect to it (sometimes) but it never established outgoing connections:

2015-11-11 22:05:45 tor: ADD_ONION succesful
2015-11-11 22:05:45 tor: Got service ID moheur3skn7jbca4, advertizing service moheur3skn7jbca4.onion:18333
2015-11-11 22:05:45 tor: Cached service private key to /home/test/.bitcoin/testnet3/onion_private_key
2015-11-11 22:05:45 AddLocal(moheur3skn7jbca4.onion:18333,4)
2015-11-11 22:05:46 Cannot connect to t6xj6wilh4ytvcs7.onion:18333: unsupported network
2015-11-11 22:05:46 Cannot connect to 4azvkpo55qaskhjl.onion:18333: unsupported network
2015-11-11 22:05:47 Cannot connect to nkf5e6b7pl4jfd4a.onion:18333: unsupported network
2015-11-11 22:05:48 Cannot connect to 4zhkir2ofl7orfom.onion:18333: unsupported network
@gits7r

This comment has been minimized.

Show comment
Hide comment
@gits7r

gits7r Nov 11, 2015

you are mistaking. it's -onlynet=tor . you confuse onion with -onion=127.0.0.1:9050 where you instruct bitcoin to use that socks5 proxy to connect to onion peers (from tor network). While -onlynet=tor will make sure you connect only to onion (Tor) peers, -onion=xxx will only provide a gateway for these peers but connect to other kind of peers also.

gits7r commented Nov 11, 2015

you are mistaking. it's -onlynet=tor . you confuse onion with -onion=127.0.0.1:9050 where you instruct bitcoin to use that socks5 proxy to connect to onion peers (from tor network). While -onlynet=tor will make sure you connect only to onion (Tor) peers, -onion=xxx will only provide a gateway for these peers but connect to other kind of peers also.

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 12, 2015

Member

Then, why does -? doesn't even mention tor?

  -onlynet=<net>
       Only connect to nodes in network <net> (ipv4, ipv6 or onion)
Member

MarcoFalke commented Nov 12, 2015

Then, why does -? doesn't even mention tor?

  -onlynet=<net>
       Only connect to nodes in network <net> (ipv4, ipv6 or onion)
@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Nov 12, 2015

Member

onlynet=onion should work. But is unrelated to this pull. Did you pass a proxy?

Member

laanwj commented Nov 12, 2015

onlynet=onion should work. But is unrelated to this pull. Did you pass a proxy?

@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 12, 2015

Member

I tried with and without proxy:
bitcoin/src/bitcoind -debug=tor -onlynet=onion -proxy=127.0.0.1:9050 -listen=1 -torpassword=bar

But maybe my VPS provider is blocking tor...

Member

MarcoFalke commented Nov 12, 2015

I tried with and without proxy:
bitcoin/src/bitcoind -debug=tor -onlynet=onion -proxy=127.0.0.1:9050 -listen=1 -torpassword=bar

But maybe my VPS provider is blocking tor...

@laanwj

This comment has been minimized.

Show comment
Hide comment
@laanwj

laanwj Nov 12, 2015

Member

They could, but that whouldn't result in errors like

2015-11-11 22:05:46 Cannot connect to t6xj6wilh4ytvcs7.onion:18333: unsupported network

Instead you'd get proxy errors. It's not using the proxy, why is that (again, this is seemingly unrleated to this pull, maybe file a new issue)

Member

laanwj commented Nov 12, 2015

They could, but that whouldn't result in errors like

2015-11-11 22:05:46 Cannot connect to t6xj6wilh4ytvcs7.onion:18333: unsupported network

Instead you'd get proxy errors. It's not using the proxy, why is that (again, this is seemingly unrleated to this pull, maybe file a new issue)

laanwj added some commits Sep 8, 2015

torcontrol improvements and fixes
- Force AUTHCOOKIE size to be 32 bytes: This provides protection against
  an attack where a process pretends to be Tor and uses the cookie
  authentication method to nab arbitrary files such as the
  wallet
- torcontrol logging
- fix cookie auth
- add HASHEDPASSWORD auth, fix fd leak when fwrite() fails
- better error reporting when cookie file is not ok
- better init/shutdown flow
- stop advertizing service when disconnected from tor control port
- COOKIE->SAFECOOKIE auth
doc: update docs for Tor listening
- add new data directory files for 0.12 to doc/files.md
- mention torcontrol in doc/tor.md

@laanwj laanwj merged commit 58ef0ff into bitcoin:master Nov 12, 2015

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

laanwj added a commit that referenced this pull request Nov 12, 2015

Merge pull request #6639
58ef0ff doc: update docs for Tor listening (Wladimir J. van der Laan)
68ccdc4 doc: Mention Tor listening in release notes (Wladimir J. van der Laan)
09c1ae1 torcontrol improvements and fixes (Wladimir J. van der Laan)
2f796e5 Better error message if Tor version too old (Peter Todd)
8f4e67f net: Automatically create hidden service, listen on Tor (Wladimir J. van der Laan)
@MarcoFalke

This comment has been minimized.

Show comment
Hide comment
@MarcoFalke

MarcoFalke Nov 12, 2015

Member

tested ACK 58ef0ff

Member

MarcoFalke commented Nov 12, 2015

tested ACK 58ef0ff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment