Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

thunderhub: connect to the external APIs through Tor #1819

Merged
merged 2 commits into from Mar 18, 2021

Conversation

openoms
Copy link
Collaborator

@openoms openoms commented Nov 29, 2020

As described in: Connect to the external APIs through Tor

To test on a running RaspiBlitz installation with Tor on edit the systemd service file:
sudo nano /etc/systemd/system/thunderhub.service

Enter torify after Execstart= so the line becomes:

ExecStart=torify /usr/bin/npm run start -- -p 3010

Restart the Thunderhub service:
sudo systemctl daemon-reload
sudo systemctl restart thunderhub

@openoms openoms changed the title thunderhub: connect to the external APIs through Tor WIP thunderhub: connect to the external APIs through Tor Nov 29, 2020
@openoms openoms changed the title WIP thunderhub: connect to the external APIs through Tor [WIP - do not merge] thunderhub: connect to the external APIs through Tor Nov 29, 2020
@openoms
Copy link
Collaborator Author

openoms commented Nov 29, 2020

Unfortunately the process fails using the Tor proxy:

$ sudo journalctl -fu thunderhub
systemd[1]: Started ThunderHub daemon.
torify[29035]: 1606641240 WARNING torsocks[29035]: [syscall] Unsupported syscall number 397. Denying the call (in tsocks_syscall() at syscall.c:605)
torify[29035]: 1606641241 WARNING torsocks[29056]: [syscall] Unsupported syscall number 397. Denying the call (in tsocks_syscall() at syscall.c:605)
torify[29035]: Error: listen EPERM: operation not permitted :::3010
torify[29035]:     at Server.setupListenHandle [as _listen2] (net.js:1318:16)
torify[29035]:     at listenInCluster (net.js:1366:12)
torify[29035]:     at Server.listen (net.js:1452:7)
torify[29035]:     at /home/thunderhub/thunderhub/node_modules/next/dist/server/lib/start-server.js:2:62
torify[29035]:     at new Promise (<anonymous>)
torify[29035]:     at start (/home/thunderhub/thunderhub/node_modules/next/dist/server/lib/start-server.js:1:431)
torify[29035]:     at nextStart (/home/thunderhub/thunderhub/node_modules/next/dist/cli/next-start.js:19:125)
torify[29035]:     at /home/thunderhub/thunderhub/node_modules/next/dist/bin/next:26:341 {
torify[29035]:   code: 'EPERM',
torify[29035]:   errno: -1,
torify[29035]:   syscall: 'listen',
torify[29035]:   address: '::',
torify[29035]:   port: 3010
torify[29035]: }
torify[29035]: npm ERR! code ELIFECYCLE
torify[29035]: npm ERR! errno 1
torify[29035]: npm ERR! thunderhub@0.10.4 start: `next start "-p" "3010"`
torify[29035]: npm ERR! Exit status 1
torify[29035]: npm ERR!
torify[29035]: npm ERR! Failed at the thunderhub@0.10.4 start script.
torify[29035]: npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
torify[29035]: npm ERR! A complete log of this run can be found in:
torify[29035]: npm ERR!     /home/thunderhub/.npm/_logs/2020-11-29T09_14_01_948Z-debug.log
systemd[1]: thunderhub.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: thunderhub.service: Failed with result 'exit-code'.

cat /home/thunderhub/.npm/_logs/2020-11-29T09_14_01_948Z-debug.log | nc termbin.com 9999 : https://termbin.com/adoc

@Kixunil
Copy link

Kixunil commented Dec 2, 2020

I think you could get it working with some modifications of torsocks.conf

Specifically:

  • AllowIbound 1 - solves the error above
  • AllowOutboundLocalhost 1 - allow connecting to lnd
  • I recommend copying the system-wide file and editing the copy, then setting TORSOCKS_CONF_FILE=/path/to/your/modified/copy before calling torify

@openoms
Copy link
Collaborator Author

openoms commented Dec 6, 2020

can test this solution with pasting this:

source /mnt/hdd/raspiblitz.conf

   # torify service if Tor is used
    if [ "${runBehindTor}" = "on" ]; then
      echo "# Connect to the external APIs through Tor"
      proxy="torify"
      echo "# set up torsocks"
      sudo cp /etc/tor/torsocks.conf /etc/tor/torsocks-thunderhub.conf
      sudo sed -i "s/^#AllowInbound 1/AllowInbound 1/g" /etc/tor/torsocks-thunderhub.conf
      sudo sed -i "s/^#AllowOutboundLocalhost 1/AllowOutboundLocalhost 1/g" /etc/tor/torsocks-thunderhub.conf
      env="Environment=TORSOCKS_CONF_FILE=/etc/tor/torsocks-thunderhub.conf"
    else
      echo "# Connect to the external APIs through clearnet"
      proxy=""
      env=""
    fi

    echo "# Install ThunderHub systemd for ${network} on ${chain}"
    echo "
# Systemd unit for thunderhub
# /etc/systemd/system/thunderhub.service

[Unit]
Description=ThunderHub daemon
Wants=lnd.service
After=lnd.service

[Service]
WorkingDirectory=/home/thunderhub/thunderhub
$env
ExecStart=$proxy /usr/bin/npm run start -- -p 3010
User=thunderhub
Restart=always
TimeoutSec=120
RestartSec=30
StandardOutput=null
StandardError=journal

[Install]
WantedBy=multi-user.target
" | sudo tee /etc/systemd/system/thunderhub.service

sudo systemctl daemon-reload
sudo systemctl restart thunderhub

@Kixunil
Copy link

Kixunil commented Dec 6, 2020

That script looks correct.

@openoms openoms changed the title [WIP - do not merge] thunderhub: connect to the external APIs through Tor thunderhub: connect to the external APIs through Tor Dec 6, 2020
@openoms
Copy link
Collaborator Author

openoms commented Dec 7, 2020

ok tested this and works on 32 and 64bit (#1833) too.
Ready to merge.

@Kixunil
Copy link

Kixunil commented Dec 7, 2020

Happy to hear it works! I'll keep it in mind for my projects. :)

@rootzoll rootzoll mentioned this pull request Mar 14, 2021
8 tasks
@openoms
Copy link
Collaborator Author

openoms commented Mar 16, 2021

Thunderhub now has a built in option to proxy over Tor: https://github.com/apotdevin/thunderhub#tor-requests

Will review this with the changed config.

@Kixunil
Copy link

Kixunil commented Mar 17, 2021

Using config definitely looks cleaner but I wonder if it's actually the best thing to do. torify seems to have a nice property of also protecting against accidental leakage. A bug in ThunderHub causing a connection to not respect proxy settings would be harmless in case of torify. Same goes for all other apps.

If you want to take it one step further, systemd supports network namespaces and it should be possible to block all outgoing traffic except for Tor on system level. That way even if the (compromised) app deliberately attempted to bypass torify (which is possible) it wouldn't be able to access clearnet. Although in the specific case of ThunderHub, compromised apps creates much bigger problems than privacy leaks.

I'm not sure how paranoid Raspiblitz is supposed to be.

@openoms
Copy link
Collaborator Author

openoms commented Mar 17, 2021

@Kixunil thanks for your comment, I agree and will keep this as it is.

It is a good option to build this in Thunderhub, but we don't need to rely on it.

As for how paranoid we supposed to be: I am happy to go deep until it does not limit the functionality.
RaspiBlitz is going Tor by default from the next release: #2056 which would open the possibility to apply do this (and use namespaces) for all the systemd services going forward.

The question here is that how important it is to be able switch back to clearnet (or another proxy) easily and how realistic it is to make that an option?

re this PR: @rootzoll it is ready to be merged

@Kixunil
Copy link

Kixunil commented Mar 17, 2021

The question here is that how important it is to be able switch back to clearnet (or another proxy) easily and how realistic it is to make that an option?

Modifying systemd files is kinda unwieldy, so here config in ThunderHub could help. I want to make this configurable in deb repo because clearnet is more reliable which may be more important to some people than their IP being logged somewhere.

@rootzoll
Copy link
Collaborator

@openoms can you rebase the PR on the dev branch?

@openoms openoms changed the base branch from v1.6 to dev March 18, 2021 15:33
@openoms openoms force-pushed the torify-thunderhub-service branch 2 times, most recently from c743a75 to 3e84aae Compare March 18, 2021 15:44
modify torsocks to allow torify
update to v0.11.0
@openoms
Copy link
Collaborator Author

openoms commented Mar 18, 2021

ok, rebased to dev

@rootzoll rootzoll merged commit aa2cf8a into raspiblitz:dev Mar 18, 2021
nyxnor added a commit to nyxnor/raspiblitz that referenced this pull request May 2, 2021
* BTCPay Server does not work on TESTNET (raspiblitz#1966)

Just a mention referring to the issue so that newbies don't waist time with this and get disappointed.

* Additional little note regarding Channels Autopilot (raspiblitz#1954)

Little note warning that Channels Autopilot triggers a reboot which can be confusing for newbies and frustrating for demos.

* Update of Autopilot section (raspiblitz#1949)

I feel the update makes it clearer, feel free to accept/reject whichever way you think is best.

* Update FAQ.md with spelling & grammar changes, & fresh links (raspiblitz#2005)

* Fixed a typo (raspiblitz#2040)

Changed waist of time to waste of time in line 528

* Remove typo in echo (raspiblitz#2034)

partion -> partition

* Fix "unkown" typo (raspiblitz#2029)

* Fix Typos

* Fix Typos

* Update blitz.copyblockchain.sh

* build_sdcard.sh: prepare for the 64bit RaspberryOS (raspiblitz#1858)

* build_sdcard.sh: prepare for the 64bit RaspberryOS

* build_sdcard: don't ping and trust microsoft

* build_sdcard: make sure the pi user is present

* build_sdcard: add the archive.raspberrypi.org repo

* build: improve recognition of the base image

* build: don't stop at installing dependencies

* build: fix locales on all raspi images

* Refactor parameters on sd crad build script with FATPACK (raspiblitz#2044)

* Remove btrfs-tools as it's a transitional/virtual package. (raspiblitz#1854)

* New 64-bit sd card image with btc/nd update, fatpack & tor by default (raspiblitz#2060)

* raspiblitz#1549 faraday update 0.2.3 (raspiblitz#2061)

* electrs update to v0.8.8 and comment cleanup (raspiblitz#2059)

* miner edits on new electrs update (raspiblitz#2063)

* Check if /dev/fb1 exists and select /dev/fb0 when not (NEW) (raspiblitz#2064)

* create temp file in shared memory (/dev/shm/) (raspiblitz#2058)

* Tested app updates (raspiblitz#2043)

* btcpay: add sqlitefile and network options (raspiblitz#2067)

* Small corrections for upadted Apps raspiblitz#2066 (raspiblitz#2072)

* add the Lightning Terminal 0.4.1-alpha (raspiblitz#1994)

* raspiblitz#2019 updtae RTL to v0.10.1

* fix the lit aliases

As discussed in:
raspiblitz#1993 (comment)

* Pool: Remove stale symlink on cleanup (raspiblitz#1957)

* mempool: drop database if exists (raspiblitz#1960)

* Bump mempool to v2.1.2, use SSD for mempool cache data (raspiblitz#2014)

* Always set maxmempool to 300MB in bitcoin.conf (raspiblitz#2007)

When the mempool fills up to around 500MB, this causes the mempool.space
backend to crash on RPI due to OOM. Additionally, raising this above
300MB on a local RPI node arguably isn't a good idea anyway, because:

* If you have a 512MB mempool and all your peers have 300MB, you'll broadcast transactions that fit in your mempool but are rejected by all of your peers
* If you connect an SPV wallet to your node, your wallet will think the TX is "in the mempool" despite actually being purged from the entire Bitcoin P2P network, and never attempt to rebroadcast the purged TX
* In any case, the RPI has very limited RAM, so no point in wasting RAM
on having such a large sized mempool for bitcoind

* lnd.conf: add tls settings, improve startup time (raspiblitz#1943)

* raspiblitz#1993 Adding Lightning Terminal Menu integration (raspiblitz#2077)

* txindex: only restart bitcoind when running (raspiblitz#1831)

* raspiblitz#2074 adding docker install script (raspiblitz#2079)

* thunderhub: connect to external APIs through Tor (raspiblitz#1819)

modify torsocks to allow torify
update to v0.11.0

* add install script for Channel Tools (raspiblitz#1962)

* Update Specter Desktop info (raspiblitz#2088)

* Fixed a typo (raspiblitz#2086)

Changed "detailes" to "details"

* raspiblitz#2089 Improve & Update Sphinx-Relay (raspiblitz#2091)

* add metric collection via telegraf as new bonus service (improved version) (raspiblitz#1616)

* raspiblitz#1360 CHANGES note

* raspiblitz#1988 changed add nodes to TorV3

* adds service and command to download whitepaper (raspiblitz#2081)

* raspiblitz#2092 whitepaper additions (raspiblitz#2093)

* Tor: use CookieAuthentication (raspiblitz#2095)

* remove passwordB from lnd config (raspiblitz#2094)

* Tor: add CookieAuthFileGroupReadable 1

"If this option is set to 0, don’t allow the filesystem group to read the cookie file. 
If the option is set to 1, make the cookie file readable by the default GID. [Making
 the file readable by other groups is not yet implemented; let us know if you need 
 this for some reason.] (Default: 0)"
from: https://2019.www.torproject.org/docs/tor-manual.html.en

* use "netstat -an" to prevent IP resolution (raspiblitz#2006)

* Update blitz.notify.sh (raspiblitz#2016)

* Update 00infoBlitz.sh (raspiblitz#1828)

* btc-rpc-explorer: update to version 2.2.0 (raspiblitz#1846)

* raspiblitz#2096 update CHANGES

* Validate TCP port (raspiblitz#1848)

* Shellcheck fixes (raspiblitz#1840)

* raspiblitz#1840 changed grep uses

* raspiblitz#2080 Add chantools to commands, menu & services (raspiblitz#2097)

* raspiblitz#2042 change wording (raspiblitz#2098)

* 1973 lndconnect refactor & zeus tor by default (raspiblitz#2100)

* sphinx-relay: remove sudo from status

fixes the fresh install as in:
raspiblitz#2089

* raspiblitz#1654 adding circuitbreaker to SETTINGS menu (raspiblitz#2105)

* raspiblitz#1817 change temp name custom script for installs

* raspiblitz#1646 remove quickfix (raspiblitz#2107)

* raspiblitz#1416 added HS default ports testnet & signet

* raspiblitz#1940 i2c & hdmi_safe to ON

* raspiblitz#1978 changed SCP instructions for Windows

* raspiblitz#2032 deactivate ZAP-iOS

* Organizing ./XXdebugLogs.sh debug logs - Change Wifi to Network and more (raspiblitz#1947)

* raspiblitz#1888 kickstart cache file (raspiblitz#2109)

* 1341 acme over tor by curl default proxy (raspiblitz#2110)

* 2089 check one channel & use wget for local test (raspiblitz#2112)

* fix sphinx chet connection test

* raspiblitz#1341 deactivate .curlrc tor proxy for now

* raspiblitz#2048 deactivate UASP (raspiblitz#2113)

* 2089 onechannel (raspiblitz#2114)

* raspiblitz#2048 fix detect UASP

* update FAQ build info

* fix error check starting menu

* check raspiblitz.conf exists

* raspiblitz#2075 remove hdmi safe default

* release of v1.7RC1 image

* raspiblitz#2048 deactivate auto-reboot

* updated RC1 image

* raspiblitz#2048 deactivate USAP for RC1

* add some reboot delays

* deactivating resize for RC1

* update SHA on release

* bos: comment out link

* joininbox update to v0.3.4

* update joininbox version to v0.3.4

* fix joinmarket installation error

* joinmarket: fix version pinning

* raspiblitz#2126 fix sd card expand & raspiblitz#1888 static fallback list (raspiblitz#2132)

* remove rpcpassword related code for lnd/btc config syncing (raspiblitz#2135)

* raspiblitz#2070 basic migration impl (raspiblitz#2142)

* raspiblitz#1265 Display Refactor (raspiblitz#2149)

* SSH menu refactoring  + CONNECT,  LIGHTNING and SYSTEM menus added (raspiblitz#2119)

* Update stacking-sats-kraken version (raspiblitz#2133)

* doc of upadte to stacking-sats-kraken 0.4.2

* Typo in 05hardwareTest printed output (raspiblitz#2139)

* Add mempool to update menu (raspiblitz#2147)

* raspiblitz#2147 added a git fetch before git checkout

* Thunderhub fix and update (raspiblitz#2141)

* raspiblitz#2118 update specter to 1.3.0 (raspiblitz#2156)

* raspiblitz#2057 Recovery Window Flag `fundRecovery` (raspiblitz#2159)

* rtl: make LiT port 8443 default for Loop in  RTL

* LiT: update RTL on the end of install for Loop

* .bashrc: source fzf before opening the main menu (raspiblitz#2140)

* Run LND with a separate Tor instance to avoid restarts (raspiblitz#2148)

* raspiblitz#2165 BTCPayServer 1.0.7.2

* raspiblitz#2161 fixes on building sd card (display & no-interaction parameter) (raspiblitz#2166)

* Release v1.7RC2

* updating v1.7RC2 info

* btcpay: exit 1 if cd fails

* _commands: clarify lit aliases

* rename recover log

* raspiblitz#2075 fix HDMI switch file & clean up boot logs (raspiblitz#2175)

* raspiblitz#2048 uasp reboot log

* FN: add BITCOINRPCPORT for testnet and signet (raspiblitz#2177)

* v1.7.0 RC2 updates (raspiblitz#2171)

* CONNECT bitcoin RPC: format output and fix removal (raspiblitz#2174)

* raspiblitz#2160 Remove Bitcoind params from lnd.conf on Update & smaller update fixes   (raspiblitz#2179)

* raspiblitz#2150 fix touchscreen and calibration (raspiblitz#2181)

* Tor: look for the correct word when checking torrc

* remove 00mainMenu.sh loop from submenus (raspiblitz#2186)

* raspiblitz#2070 Implement myNode migration (raspiblitz#2187)

* raspiblitz#2185 remove hardware test (raspiblitz#2190)

* raspiblitz#1901 changing path of PID file

* improve the output in the SYSTEM menu

* raspiblitz#2157 offer http for additional apps & small fixes (raspiblitz#2196)

* raspiblitz#2173 Connect Specter fixes (raspiblitz#2197)

* set default port value
* fix port determine

* raspiblitz#2099 small fix on delete macarons

* raspiblitz#2066 small fixes for apps, updates & lcd (raspiblitz#2200)

* updated 64bit sd card image

* run apt update after adding Tor sources (raspiblitz#2203)

* BITCOIN UPDATE option (raspiblitz#2202)

* Bitcoin update RC support (raspiblitz#2205)

* raspiblitz#2167 smaller fixes for v1.7RC3 (raspiblitz#2206)

* adding spacing line

* prepare v1.7RC3

* fix lnbits macaroon write after reset

* add IP2Tor renew info

* fixing info

* remove debug info

* raspiblitz#2157 LNbits needs https for local

* more info

* reboot on macaroon reset

* fix blocking LIT install

* set correct fonts

* set font for lcd

* raspiblitz#2198 add connect Info

* fix syntax

* fix text layout

* layout text

* fix text layout

* more install info

* fix text layout

* text layout

* text layout

* fix restart lnd

* force lnd RPC/REST standard & open on boot

* raspiblitz#2198 fix fully noded connection

* raspiblitz#2198 deactivate Zap-Android from menu

* telegraf updates for v1.7.0RC3 (raspiblitz#2204)

* new path of "vcgencmd" at raspiblitz v1.7.0RC2
* whitespaces + rename raspiblitz_ip_info -> ipinfo
* add mempool info via bitcoincli
Co-authored-by: Patrick Scheich <patrick.scheich@syscovery.de>

* CHANGES: note using the base image from April 2021

* raspiblitz#2221 change hdmi mode (raspiblitz#2224)

* unlock LND after Bitcoin Core install if ready (raspiblitz#2214)

* unlock LND after Bitcoin Core install if ready
* bitcoin.update: improve output and comments

* show nginx systemd logs (raspiblitz#2220)

This is important, normally nginx fails and this is not shown.

* Update stacking-sats-kraken version (raspiblitz#2216)

* small ux thing - bigger dialog box

* raspiblitz#2070 checking version & resetting password c (raspiblitz#2226)

* Table of options on how to have blitz (raspiblitz#2219)

* Create SECURITY.md (raspiblitz#2212)

* merge updates documentation for v1.7

* upload of raspiblitz-v1.7.0-2021-04-24.img.gz

* update version 1.7.0

* upload image raspiblitz-v1.7.0-2021-04-25.img.gz

* correct signature link for v1.7.0

* SECURITY.md typos (raspiblitz#2229)

please correct this before v1.7 @rootzoll

* update README for v1.7.0

* fix typo

* List all compatible displays on Amazon (raspiblitz#2253)

Give some alternatives if the mentioned display is sold out on Amazon

* raspiblitz#2243 formatting info on additional screen vendors

* raspiblitz#2247 add HTTP code as debug info

* change force to yes on installation (raspiblitz#2254)

Co-authored-by: JayDeLux <iffrig@hotmail.com>
Co-authored-by: davterra <davterra44@gmail.com>
Co-authored-by: Nerfzooka <Nerfzooka@users.noreply.github.com>
Co-authored-by: keblek <68111648+keblek@users.noreply.github.com>
Co-authored-by: Jestopher <59354805+Jestopher-BTC@users.noreply.github.com>
Co-authored-by: openoms <43343391+openoms@users.noreply.github.com>
Co-authored-by: /rootzoll <rotzoll@googlemail.com>
Co-authored-by: FreePietje <50077459+FreePietje@users.noreply.github.com>
Co-authored-by: /rootzoll <christian@geektank.de>
Co-authored-by: openoms <oms@tuta.io>
Co-authored-by: d11n <mail@dennisreimann.de>
Co-authored-by: Christoph Stenglein <9399034+cstenglein@users.noreply.github.com>
Co-authored-by: wiz <j@wiz.biz>
Co-authored-by: PatrickScheich <50054697+PatrickScheich@users.noreply.github.com>
Co-authored-by: raulcano <raul.cano.argamasilla@gmail.com>
Co-authored-by: frennkie <mail@rhab.de>
Co-authored-by: Joseph Goulden <joegoulden@protonmail.com>
Co-authored-by: roshii <roshii@riseup.net>
Co-authored-by: adaleben <69700936+Ben10Brasileiro@users.noreply.github.com>
Co-authored-by: Nygel Lyndley <github.com@lyndley.com>
Co-authored-by: Philihp Busby <philihp@gmail.com>
Co-authored-by: Deverick <deverick@me.com>
Co-authored-by: Animalmother9000 <48626097+Animalmother9000@users.noreply.github.com>
Co-authored-by: /rootzoll <christian@fulmo.org>
@openoms openoms deleted the torify-thunderhub-service branch May 8, 2021 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants