Activate TOR Network on Setup #592

drunkenclam · 2019-05-10T13:27:58Z

Most Raspiblitz users are running their node from home, which makes them prone to privacy and security failures.

When setting up a new Raspiblitz, it should default to run only on TOR network. Clearnet should only be possible if the user wants to actively opt-in to advertise his node IP.

Advantages:

more private, because the node IP isn't advertised
more secure, because people can't see the BTC value connected to an advertised IP address
more comfortable for users with dynamic IPs, as they don't have to handle IP changes and port forwardings
help growing the LN network on TOR

Disadvantages:

a node behind TOR won't probably route as much payments as a node available on clearnet
?

More info: https://github.com/lightningnetwork/lnd/blob/master/docs/configuring_tor.md

openoms · 2019-05-11T15:42:54Z

I agree with Tor on by default. It also helps with port forward issues.

Would go as far as experimenting with running a Tor relay node as well to strengthen the anonymous side of the network. Would help a lot with Initial Block Download through Tor.

See as example: https://github.com/seth586/guides/blob/master/FreeNAS/extras.md#tor-relay-node

Guide for Debian/Ubuntu: https://trac.torproject.org/projects/tor/wiki/TorRelayGuide/DebianUbuntu

RolfHodl · 2019-05-11T20:47:23Z

Go for TOR as default! Great suggestion.

rootzoll · 2019-05-13T13:40:29Z

Not sure with TOR as a forced default - I like to let the user start with the mininum and then add to it by option. If TOR is a forced default, then there is no way for a user to start with the RaspiBlitz without TOR/hiddenService.

Let me paint a dramatic picture: Think of a country where its punishable by law to run TOR. Every outed RaspiBlitz user could been accused of having run TOR/hiddenService at least once, just because of the use of RaspiBlitz.

Or maybe a more paranoid picture: Someone who belives that TOR is run/subverted by a gov agency and is a honey pot would be forced joined.

BUT I think TOR on or off can be a OPTION DURING THE SETUP and so running TOR can be the first/default option here to foster its use. I think that could be the best compromise here.

openoms · 2019-05-13T14:02:00Z

BUT I think TOR on or off can be a OPTION DURING THE SETUP and so running TOR can be the first/default option here to foster its use. I think that could be the best compromise here.

💯% agree, yes can keep clearnet as the default, but give the option during the initial setup to have everything on Tor from the very start. This way the user's IP address and LN node public key does not get associated at all. That will be a huge improvement in privacy.

drunkenclam · 2019-05-13T14:28:26Z

I'd still recommend to default to TOR during initial setup with the user being able to opt-out of TOR if she wants to do so. Maybe a short explainer why to run with or without TOR.

IMHO if a user don't know what to pick here, the privacy risk by exposing the IP and the funds tied to it is far more severe than running TOR (and afaik only exit nodes had been problematic, which we are not using).

openoms · 2019-05-13T14:55:00Z

An argument for clearnet as the default is that the Raspiblitz is also for people who are completely new to the space and just want to try out having a full node. They might feel being forced into participating in Tor as well which is again another thing to digest and accept. Might repel some people and decrease adoption (not even going as far the potential legal /trust issues mentioned by @rootzoll).

Regarding the funds I would not suggest anyone keeping significant amounts on a "cheapest as possible" Raspberry Pi node so the "tainting" is minimal. Also I think running the bitcoin software would probably flag one up less than running Tor, but this can change and is very different between jurisdictions and threat models.

As a more advanced/paranoid user or running on an alternative platform one would build their own SD card anyway. In that we could make an option to install and activate Tor already before the first reboot.

In that case by building their own SD card one could have the option to have Tor on by default and others could choose to opt-in during the initial setup.

openoms · 2019-05-20T12:44:20Z

Made a branch where introduced a third parameter Tor to build the SD. When used will download only Bitcoin and Lightning and the first connections of bitcoind and LND will be through Tor.
Tor is installed after the HDD is mounted, but before bitcoind starts for the first time (or presync).

Command to test building the SD card (https://github.com/rootzoll/raspiblitz#build-the-sd-card-image):
wget https://raw.githubusercontent.com/openoms/raspiblitz/Tor_by_default/build_sdcard.sh && sudo bash build_sdcard.sh Tor_by_default openoms Tor

openoms · 2019-05-25T17:26:18Z

User reported bug in Telegram channel:
after updating (from 1.1 to 1.2) it seems that my bitcoin core node is stuck on block 577602 whereas we are now on (according to blockstream.info) 577661, as such it seems to be (i think?) preventing LND from starting

looking at the debug log for bitcoind in /home/bitcoin/.bitcoin/debug.log it seems to be related to tor. All the messages are things like:

2019-05-25T16:52:45Z Cannot create socket for toguvy5upyuctudx.onion:8333: unsupported network
2019-05-25T16:52:45Z Cannot create socket for ndndword5lpb7eex.onion:8333: unsupported network
2019-05-25T16:52:46Z Cannot create socket for 6m2iqgnqjxh7ulyk.onion:8333: unsupported network
2019-05-25T16:52:46Z Cannot create socket for 5tuxetn7tar3q5kp.onion:8333: unsupported network

sudo systemctl restart tor

command did not work: Failed to restart tor.service: Unit tor.service not found

The user was updating a node which had Tor on before.

Could replicate the error:
_bootstrap.provision.sh attempts to install Tor if restoring a HDD previously run behind Tor.

# TOR
if [ "${runBehindTor}" = "on" ]; then
    echo "Provisioning TOR - run config script" >> ${logFile}
    sudo sed -i "s/^message=.*/message='Setup TOR (takes time)'/g" ${infoFile}
    sudo /home/admin/config.scripts/internet.tor.sh on >> ${logFile} 2>&1
else 
    echo "Provisioning TOR - keep default" >> ${logFile}
fi

The sudo /home/admin/config.scripts/internet.tor.sh on fails beacuse the HDD is not mounted yet, the service cannot be created.

Solution:

proceed as if SD was built with Tor parameter.
Introducing TorBydefault=on to /home/admin/raspiblitz.info and install Tor after HDD is mounted in 40addHDD.sh

openoms · 2019-05-27T19:34:22Z

PR incoming with this one after I have done a couple of test builds.

drunkenclam · 2019-05-29T16:16:59Z

LN Tor usage getting more attention lately:

https://twitter.com/n1ckler/status/1133671925299982337

https://twitter.com/BtcpayServer/status/1131361209947570176

rootzoll · 2019-06-13T12:29:18Z

OK implemented for v1.3 release .. needs testing if working within setup process.

openoms · 2019-06-13T17:33:38Z

I must admit I had trouble with my implementation so will be excited to try a working one!

rootzoll · 2019-06-18T01:32:19Z

First test during setup shows that TOR gets installed and bitcoin syncs on it ... but bitcoin seems quite slow on catching up on sync when running behind TOR. Will see if it will catchup over night.

LND configuration looks correct for TOR after setup. Showing no URIs yet ... will wait after it has finished initial scan.

openoms · 2019-06-18T04:08:16Z

I have just finished testing an SSD with an Odroid XU4. Initial Block Download took ~3 days with some Internet/router troubles one night. Will try syncing on Tor now.

If syncing behind Tor (I presume) depends more on the network speed rather than processing power, running a Tor relay should be considered to strengthen the network.

It only takes as much as modifying the configuration file /etc/tor/torrc:

#change the nickname "myNiceRelay" to a name that you like 
Nickname myNiceRelay 
ORPort 443 
ExitRelay 0 
SocksPort 0
ControlSocket 0 
# Change the email address bellow and be aware that it will be published 
ContactInfo tor-operator@your-emailaddress-domain

Could be activated from the service menu after initial sync. What do you think @rootzoll ?

rootzoll · 2019-06-18T15:11:36Z

@openoms about activating TOR as relay node I added to this to a later issue here: #659

I am considering if the user chooses to activate TOR on setup to just apply this to the Lightning Node first (so that bitcoind is not getting that slow on ctachup syncing) and then (once full synced) you can switch also your bitcoind behind TOR ... not sure if practical possible, but from a theoretical view it should be OK as long as you dont associate your Lightning NodeID with your IP. Any concerns with that?

rootzoll · 2019-06-18T18:33:32Z

Catchup of bitcoind behind TOR is slow but seems to work. Will test a bit longer.

openoms · 2019-06-18T20:02:01Z

I am considering if the user chooses to activate TOR on setup to just apply this to the Lightning Node first (so that bitcoind is not getting that slow on ctachup syncing) and then (once full synced) you can switch also your bitcoind behind TOR ... not sure if practical possible, but from a theoretical view it should be OK as long as you dont associate your Lightning NodeID with your IP. Any concerns with that?

I agree that the most important option is to not advertise the IP of the new Lightning Node being set up, but that is an option already if port forwarding is simply not set up for 9735.
The IP only reaches the network together with the public key if port forwarding 9735 is active. In the case when the node is behind a firewall the public key is only advertised on the network when Tor is switched on. Because of this I don`t think it would worth to work on decoupling the Tor settings of bitcoind and lnd.

If someone wants to hide the fact that he is using Bitcoin and LN, Tor should be switched on for both before the Initial Sync knowing that it will affect the performance.

drunkenclam · 2019-06-18T21:54:46Z

Yep, that's probably the best choice: Just let the user decide if he wants to use clearnet for faster initial sync with less privacy. When finished, switch back to everything over tor.

Thanks for implementing this, unfortunately I don't have time for testing currently...

openoms · 2019-06-19T05:41:05Z

I have just finished testing an SSD with an Odroid XU4. Initial Block Download took ~3 days with some Internet/router troubles one night. Will try syncing on Tor now.

Started the IBD behind Tor last last night with the same config. ~20% after 10 hours. Expecting to finish roughly the same within 2-3 days. Will update this comment as it goes.

rootzoll · 2019-06-24T00:03:40Z

TODO: Info that torrent download is not done thru TOR (advise to copy)
TODO: Test re-rorrent download when TOR is on

openoms · 2019-06-25T19:42:11Z

I have just finished testing an SSD with an Odroid XU4. Initial Block Download took ~3 days with some Internet/router troubles one night. Will try syncing on Tor now.

Started the IBD behind Tor last last night with the same config. ~20% after 10 hours. Expecting to finish roughly the same within 2-3 days. Will update this comment as it goes.

This node of mine got inaccessible the second time now. First I needed to do a hard restart (pulling the plug) as it seemed to freeze and on restart it started to sync the blockchain again from 0%. Now it is again lost connection, but I am away so cannot look into it for a couple of days.

Issue: the 00raspiblitz.sh is too radical deleting all the blockchaindata in the case when there is a restart during IBD.

UPDATE: the XU4 SBC was in an off-state. Froze or failed to to power the SSD (Samsung EVO 860 rated to 1.2A). Anyway it does not seem to be problem related to the Tor network.

rootzoll · 2019-07-01T16:05:07Z

TODO: Test on RP3 was soo slow I will testest on a RP4.

@openoms there might even be faster way todo this I wil test on speed now - please let me know if you see any "danger" there:

Init RaspiBlitz while on public IP (syncing Blockchain and creating a LND wallet)
Once everything is running - switch on TOR
Then go into new REPAIR menu and RESET LND and create a new wallet
Now let this new wallet scan the blockchain and you should be good.

openoms · 2019-08-05T16:55:49Z

Same RPi4 node. Building the SDcard manually with wget https://raw.githubusercontent.com/rootzoll/raspiblitz/v1.3/build_sdcard.sh && sudo bash build_sdcard.sh v1.3 rootzoll worked. Tor installed successfully at first boot.
Based on Raspbian Buster with Desktop.
RTL still not working, the issue is open with the team: https://github.com/ShahanaFarooqui/RTL/issues/173
Will make an 1.3 RC2 for testing. Couple of people struggling to start their the RPi4-s currently.

rootzoll · 2019-08-08T16:43:00Z

Choosing TOR will be a two step if needed: During Setup user has to choose between PublicIP and TOR in general. And if user chooses to self-validate/sync with the Blitz it will ask again if the IBD (just for Bitcoind) can be PublicP or should also be behind TOR. Here is the info text on that second choice the Blitz will show:

Sync Blockchain from behind TOR? 
You decided to run your node behind TOR and validate the blockchain with your RaspiBlitz - thats good. But downloading the complete blockchain thru TOR can add some extra time (maybe a day) to the process and adds a heavy load on the TOR network.
Your RaspiBlitz can just run the initial blockchain download with your public IP (Public-Sync) but keep your Lighting node safe behind TOR. It would speed up the self-validation while not revealing your Lightning node identity. But for most privacy choose (TOR-Sync).

fluidvoice · 2019-08-12T18:59:27Z

@openoms I've played around with various settings and I think it's quite possible the problem, which appears to be rights related is due to "Apparmor" and this I think was installed by default with Debian 10 by defailt. It's also running in Raspbian, right?

rootzoll · 2019-08-12T23:32:42Z

Testing with TOR looks good so far on my Raspberries with Debian Buster.

fluidvoice · 2019-08-13T10:52:59Z

Testing with TOR looks good so far on my Raspberries with Debian Buster.

What configuration and kind of test(s) was performed so far and observed working?

openoms · 2019-08-13T11:00:54Z

Choosing TOR will be a two step if needed: During Setup user has to choose between PublicIP and TOR in general. And if user chooses to self-validate/sync with the Blitz it will ask again if the IBD (just for Bitcoind) can be PublicP or should also be behind TOR. Here is the info text on that second choice the Blitz will show:
Sync Blockchain from behind TOR? 
You decided to run your node behind TOR and validate the blockchain with your RaspiBlitz - thats good. But downloading the complete blockchain thru TOR can add some extra time (maybe a day) to the process and adds a heavy load on the TOR network.
Your RaspiBlitz can just run the initial blockchain download with your public IP (Public-Sync) but keep your Lighting node safe behind TOR. It would speed up the self-validation while not revealing your Lightning node identity. But for most privacy choose (TOR-Sync).

This is awesome! Agree that most users will only want to obscure their IP in relation to their LND node. Running the IBD behind Tor in every occasion would put an unnecessarily heavy load on the network. Now everyone has the choice to do as they like.

rootzoll · 2019-08-13T11:08:02Z

@fluidvoice with RC5 (release today). I did both options like you can see from the dialog above and they run without problem so far. Maybe give the RC5 a try as soon as its out.

fluidvoice · 2019-08-14T19:02:35Z

@fluidvoice with RC5 (release today). I did both options like you can see from the dialog above and they run without problem so far. Maybe give the RC5 a try as soon as its out.

yes, will try latest now. Btw really looks like in non-Raspbian OS it's related to the apparmor profile configuration. See issue tracking here. Looks like the problem for VM's is the apparmor service is installed and running by default in Ubuntu and Debian Buster but not in Raspbian. I will try to disable it and test.

fluidvoice · 2019-08-14T23:52:57Z

Looks like Tor works in my Debian Buster x86 VM now.
Just needed to do sudo systemctl disable apparmor.
Also, opened a research issue to track potential future inclusion of the apparmor service in Raspbian/Armbian.

openoms · 2019-08-15T22:07:35Z

IBD on Clearnet with Tor activated for Lightning went well, in 2days 2 hours on my RPi4 4gb + SSD: https://twitter.com/openoms/status/1162118831965446145.

Only concern was that, when the IBD finished the LND was restarted and waited to finish the blockchain scan until unlocked again. This added hours to have LND ready because it needed to scan the last ~5 percent of the blockchain. Ideally both bitcoin and LND sync should finish before restarting the services to put bitcoin behind Tor too or maybe wait for a manual trigger and keep in sync until looked at?

fluidvoice · 2019-08-17T18:57:04Z

edit: moved to revisiting fast-sync subject

woeisme · 2019-08-19T01:28:55Z

@fluidvoice with RC5 (release today). I did both options like you can see from the dialog above and they run without problem so far. Maybe give the RC5 a try as soon as its out.

@rootzoll what's the best way to backup an existing installation to try out RC5 ?
Is there any method for cutting down on startup time?

fluidvoice · 2019-08-19T14:59:20Z

@fluidvoice with RC5 (release today). I did both options like you can see from the dialog above and they run without problem so far. Maybe give the RC5 a try as soon as its out.

@rootzoll what's the best way to backup an existing installation to try out RC5 ?
Is there any method for cutting down on startup time?

https://github.com/rootzoll/raspiblitz#backup-for-on-chain---channel-funds

rootzoll · 2019-08-27T15:13:47Z

Only concern was that, when the IBD finished the LND was restarted and waited to finish the blockchain scan until unlocked again. This added hours to have LND ready because it needed to scan the last ~5 percent of the blockchain. Ideally both bitcoin and LND sync should finish before restarting the services to put bitcoin behind Tor too or maybe wait for a manual trigger and keep in sync until looked at?

I will check on this before final release - but maybe such optimization will get pushed to next released to not further delay the v1.3 release.

woeisme · 2019-08-31T04:22:06Z

@fluidvoice with RC5 (release today). I did both options like you can see from the dialog above and they run without problem so far. Maybe give the RC5 a try as soon as its out.

@rootzoll what's the best way to backup an existing installation to try out RC5 ?
Is there any method for cutting down on startup time?

https://github.com/rootzoll/raspiblitz#backup-for-on-chain---channel-funds

Is this the preferred method for switching between 1.3rc3 -> 1.3rc5 without losing funds?

rootzoll · 2019-09-01T16:01:52Z

@woeisme on v1.3rc3 you can use the "UPDATE" option in the SSH main menu and just use "continue anyway" if reported that no new version is available.

rootzoll · 2019-09-03T15:55:21Z

@openoms pushed optimization to future release: see here #750 so that I can release v1.3 this week and dont need re-testing a 2 day long process. It should work good enough for now.

codeoholic · 2020-06-04T19:09:42Z

Hey @rootzoll @openoms post v1.5.1 update. I am seeing this in TOR logs.

/run/tor is not owned by this user (bitcoin, 1002) but by debian-tor (111).
Before Tor can create a control socket in "/run/tor/control", the directory "/run/tor" needs to exist, and to be accessible only by the user.

Can you help me understand what exactly is going here and a possible solution.

rootzoll · 2020-06-04T20:16:04Z

@codeoholic those warnings are not critical - see issue: #402

The idea is to clean this up, when we make TOR by default in an upcoming version.

codeoholic · 2020-06-05T06:18:03Z

Okay. I was thinking this could be the reason for the LND REST service being unreachable via TOR. If this is normal, could you point me how to debug it?

rootzoll · 2020-06-05T09:50:07Z

To monitor TOR connections check the MAINMENU > TOR ... thats the nyx TOR Monitor:
https://nyx.torproject.org

rootzoll added this to the 1.3 Release milestone May 13, 2019

DaReaLDeviL mentioned this issue Jun 8, 2019

UPNP open port in router #400

Closed

rootzoll added a commit that referenced this issue Jun 12, 2019

#592 prepare tor default config

14fdc27

rootzoll changed the title ~~Run on TOR network by default~~ Activate TOR Network on Setup Jun 13, 2019

rootzoll added a commit that referenced this issue Jun 13, 2019

#592 TOR on Setup

b9afd19

rootzoll self-assigned this Jun 13, 2019

rootzoll added the final testing was fixed - needs testing label Jun 13, 2019

rootzoll mentioned this issue Jun 18, 2019

Activate TOR as relay node #659

Closed

openoms mentioned this issue Aug 5, 2019

Experimental v1.3 RC2 image for the RPi4 #708

Closed

rootzoll added a commit that referenced this issue Aug 6, 2019

#592 fixing Tor during setup

36575c2

rootzoll added a commit that referenced this issue Aug 8, 2019

#592 user choice to IBD behind TOR or IP

0ecf144

rootzoll closed this as completed Aug 27, 2019

rootzoll reopened this Aug 27, 2019

rootzoll mentioned this issue Sep 3, 2019

Optimize selfsync/validation process #750

Closed

rootzoll closed this as completed Sep 3, 2019

nyxnor mentioned this issue Jan 2, 2021

Socks5() connect to onionadress.onion :8333 failed: general failure Potential stale tip detected, will try using extra outbound peer (last tip update number seconds ago) #1917

Closed

rootzoll removed the final testing was fixed - needs testing label Dec 15, 2022

Activate TOR Network on Setup #592

Activate TOR Network on Setup #592

Comments

drunkenclam commented May 10, 2019

openoms commented May 11, 2019

RolfHodl commented May 11, 2019

rootzoll commented May 13, 2019 • edited

openoms commented May 13, 2019 • edited

drunkenclam commented May 13, 2019

openoms commented May 13, 2019 • edited

openoms commented May 20, 2019 • edited

openoms commented May 25, 2019 • edited

Solution:

openoms commented May 27, 2019

drunkenclam commented May 29, 2019

rootzoll commented Jun 13, 2019

openoms commented Jun 13, 2019

rootzoll commented Jun 18, 2019

openoms commented Jun 18, 2019

rootzoll commented Jun 18, 2019

rootzoll commented Jun 18, 2019

openoms commented Jun 18, 2019

drunkenclam commented Jun 18, 2019

openoms commented Jun 19, 2019

rootzoll commented Jun 24, 2019

openoms commented Jun 25, 2019 • edited

rootzoll commented Jul 1, 2019

openoms commented Aug 5, 2019 • edited

rootzoll commented Aug 8, 2019

fluidvoice commented Aug 12, 2019 • edited

rootzoll commented Aug 12, 2019

fluidvoice commented Aug 13, 2019 • edited

openoms commented Aug 13, 2019

rootzoll commented Aug 13, 2019

fluidvoice commented Aug 14, 2019 • edited

fluidvoice commented Aug 14, 2019 • edited

openoms commented Aug 15, 2019

fluidvoice commented Aug 17, 2019 • edited

woeisme commented Aug 19, 2019

fluidvoice commented Aug 19, 2019

rootzoll commented Aug 27, 2019

woeisme commented Aug 31, 2019

rootzoll commented Sep 1, 2019

rootzoll commented Sep 3, 2019

codeoholic commented Jun 4, 2020

rootzoll commented Jun 4, 2020

codeoholic commented Jun 5, 2020

rootzoll commented Jun 5, 2020

rootzoll commented May 13, 2019 •

edited

openoms commented May 13, 2019 •

edited

openoms commented May 13, 2019 •

edited

openoms commented May 20, 2019 •

edited

openoms commented May 25, 2019 •

edited

openoms commented Jun 25, 2019 •

edited

openoms commented Aug 5, 2019 •

edited

fluidvoice commented Aug 12, 2019 •

edited

fluidvoice commented Aug 13, 2019 •

edited

fluidvoice commented Aug 14, 2019 •

edited

fluidvoice commented Aug 14, 2019 •

edited

fluidvoice commented Aug 17, 2019 •

edited