Raspberry Pi check; fixing breaking changes when not using built-in Tor

DEVELOPMENT.md

@@ -32,29 +32,64 @@
 
 ## How to run the Application
 
-Install dependencies:
+### Install dependencies:
 
-* Ubuntu/Debian: `sudo apt install libusb-1.0-0-dev libudev-dev libffi-dev libssl-dev`
-* macOS: `brew install libusb`
+#### Ubuntu/Debian
+`sudo apt install libusb-1.0-0-dev libudev-dev libffi-dev libssl-dev`
 
-Note that `hwi-1.2.0` needs Python 3.6-3.8. If you have Python 3.9 installed then be sure to also install an old Python version and pass it to `virtualenv`, .e.g `virtualenv --python3.8 .env`.
+#### macOS
+`brew install libusb`
+
+#### Windows
+* Install python 3.8.x by downloading from [python.org](https://www.python.org/downloads/windows/)
+
+    _Do NOT install python from the Microsoft Store! It runs in a different execution environment that creates enormous headaches!_
+
+    Confirm your installation in Windows PowerShell:
+    ```
+    python --version
+    ```
+
+* Must have [Visual Studio Community Edition](https://visualstudio.microsoft.com/vs/community/) installed. Be sure to select Visual C++ during installation.
+
+* Download [libusb-1.0.dll](https://libusb.info). Use [7-Zip](https://7-zip.org) to decompress the .7z file. Copy `libusb-1.0.dll` from `VS2019/MS64/dll` to your `/Windows/System32` directory.
+
+* Configure Windows PowerShell to run scripts. See: [About Execution Policies](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1). In a PowerShell window run:
+    ```
+    Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser
+    ```
+
+
+### Set up virtualenv
+Note that `hwi-1.2.0` needs Python 3.6-3.8. If you have Python 3.9 installed then be sure to also install an old Python version and pass it to `virtualenv` (e.g. `virtualenv --python3.8 .env`).
 
 ```sh
 git clone https://github.com/cryptoadvance/specter-desktop.git
 cd specter-desktop
+pip3 install virtualenv
 virtualenv --python=python3 .env 
 source .env/bin/activate
 pip3 install -r requirements.txt --require-hashes
 pip3 install -e .
 ```
 
+_note: invoking commands in the Windows PowerShell is slightly different:_
+```
+# use 'python' instead of 'python3'
+virtualenv --python=python .env
+
+# activating virtualenv
+.env\Scripts\activate
+```
+
 Run the server:
 
 ```sh
 cd specter-desktop
 python3 -m cryptoadvance.specter server --config DevelopmentConfig
 ```
 
+
 ## How to run the tests
 _TODO: Need more thorough tests!_
 

docs/tor.md

@@ -19,6 +19,7 @@ To set this up, you'll just need to go to Specter's Settings -> Tor tab, then se
 ### Install Tor service
 Alternatively, you can setup the Tor service. Install Tor on the same server that you'll be running Specter Desktop:
 * [Debian / Ubuntu](https://2019.www.torproject.org/docs/debian.html.en)
+* Raspberry Pi OS: `sudo apt install tor`
 * [macOS](https://2019.www.torproject.org/docs/tor-doc-osx.html.en)
 
 ### Configure Specter Desktop for Remote Instance
@@ -33,19 +34,27 @@ If you're using Specter Desktop on a remote machine and would like to connect to
 
 After saving, Specter will shutdown. Start it again and you should be able to connect to your remote Specter server.
 
-### Configure Tor port
-Update your `torrc` config file (usually `/etc/tor/torrc` or `/usr/local/etc/tor/torrc` on macOS Homebrew installs) and uncomment the `ControlPort` line.
-```sh
-## The port on which Tor will listen for local connections from Tor
-## controller applications, as documented in control-spec.txt.
-ControlPort 9051
+### Configure Tor
+#### Hash Specter's `torrc_password`
+First you'll need to copy your `torrc_password` from the Specter `config.json` (found in `~/.specter` by default).
+
+Then hash the password using Tor's built-in command:
+```
+$ tor --hash-password your_torrc_password
+16:6FB92F9B361D347060D6D2E95E810604DC55A22D38492F28C51F2AACDF
 ```
 
-You may also need to add the following lines in the `torrc` file:
+Copy the `16:...` hashed password and save it for the next step
 
+#### Edit `torrc`
+Update your `torrc` config file (usually `/etc/tor/torrc`; Mac OS homebrew: `/usr/local/etc/tor/torrc` or `/opt/homebrew/etc/tor/torrc`) and uncomment the `ControlPort` and `HashedControlPassword` lines. Paste in your `16:...` hashed password here:
 ```sh
-CookieAuthentication 1
-CookieAuthFileGroupReadable 1
+## The port on which Tor will listen for local connections from Tor
+## controller applications, as documented in control-spec.txt.
+ControlPort 9051
+## If you enable the controlport, be sure to enable one of these
+## authentication methods, to prevent attackers from accessing it.
+HashedControlPassword your_hashed_password
 ```
 
 Restart the Tor service:

src/cryptoadvance/specter/bitcoind.py

@@ -82,18 +82,28 @@ class BitcoindController:
     def __init__(
         self, rpcport=18443, network="regtest", rpcuser="bitcoin", rpcpassword="secret"
     ):
-        self.rpcconn = Btcd_conn(
-            rpcuser=rpcuser, rpcpassword=rpcpassword, rpcport=rpcport
-        )
-        self.network = network
+        try:
+            self.rpcconn = Btcd_conn(
+                rpcuser=rpcuser, rpcpassword=rpcpassword, rpcport=rpcport
+            )
+            self.network = network
+        except Exception as e:
+            logger.exception(f"Failed to instantiate BitcoindController. Error: {e}")
+            raise e
 
     def start_bitcoind(
-        self, cleanup_at_exit=False, cleanup_hard=False, datadir=None, extra_args=[]
+        self,
+        cleanup_at_exit=False,
+        cleanup_hard=False,
+        datadir=None,
+        extra_args=[],
+        timeout=30,
     ):
         """starts bitcoind with a specific rpcport=18543 by default.
         That's not the standard in order to make pytest running while
         developing locally against a different regtest-instance
-        if bitcoind_path == docker, it'll run bitcoind via docker
+        if bitcoind_path == docker, it'll run bitcoind via docker.
+        Specify a longer timeout for slower devices (e.g. Raspberry Pi)
         """
         if self.check_existing() != None:
             return self.check_existing()
@@ -106,7 +116,7 @@ def start_bitcoind(
             extra_args=extra_args,
         )
 
-        self.wait_for_bitcoind(self.rpcconn)
+        self.wait_for_bitcoind(self.rpcconn, timeout=timeout)
         if self.network == "regtest":
             self.mine(block_count=100)
         return self.rpcconn
@@ -174,15 +184,15 @@ def check_bitcoind(rpcconn):
             return False
 
     @staticmethod
-    def wait_for_bitcoind(rpcconn):
-        """ tries to reach the bitcoind via rpc. Will timeout after 30 seconds """
+    def wait_for_bitcoind(rpcconn, timeout=30):
+        """ tries to reach the bitcoind via rpc. Timeout after n seconds """
         i = 0
         while True:
             if BitcoindController.check_bitcoind(rpcconn):
                 break
             time.sleep(0.5)
             i = i + 1
-            if i > 60:
+            if i > (2 * timeout):
                 raise Exception(
                     "Timeout while trying to reach bitcoind at rpcport {} !".format(
                         rpcconn
@@ -240,11 +250,20 @@ def __init__(
         rpcuser="bitcoin",
         rpcpassword="secret",
     ):
-        super().__init__(
-            rpcport=rpcport, network=network, rpcuser=rpcuser, rpcpassword=rpcpassword
-        )
-        self.bitcoind_path = bitcoind_path
-        self.rpcconn.ipaddress = "localhost"
+        try:
+            super().__init__(
+                rpcport=rpcport,
+                network=network,
+                rpcuser=rpcuser,
+                rpcpassword=rpcpassword,
+            )
+            self.bitcoind_path = bitcoind_path
+            self.rpcconn.ipaddress = "localhost"
+        except Exception as e:
+            logger.exception(
+                f"Failed to instantiate BitcoindPlainController. Error: {e}"
+            )
+            raise e
 
     def _start_bitcoind(
         self, cleanup_at_exit=True, cleanup_hard=False, datadir=None, extra_args=[]

src/cryptoadvance/specter/specter.py

@@ -597,7 +597,16 @@ def update_tor_control_port(self, tor_control_port, user):
             self._save()
             self.update_tor_controller()
 
+    def generate_torrc_password(self, overwrite=False):
+        if "torrc_password" not in self.config or overwrite:
+            self.config["torrc_password"] = secrets.token_urlsafe(16)
+            self._save()
+            logger.info(f"Generated torrc_password in {self.config_fname}")
+
     def update_tor_controller(self):
+        if "torrc_password" not in self.config:
+            # Will be missing if the user did not go through the built-in Tor setup
+            self.generate_torrc_password()
         try:
             tor_control_address = urlparse(self.proxy_url).netloc.split(":")[0]
             if tor_control_address == "localhost":

src/cryptoadvance/specter/templates/node_setup_wizard.jinja

@@ -88,7 +88,7 @@
     </div>
     <div class="card center hidden" id="step4">
         <h1><img src="{{ url_for('static', filename='img/icon.png') }}" style="width: 60px;"/> <img src="{{ url_for('static', filename='img/arrow_right.svg') }}" style="width: 60px;"/>&nbsp;{{ bitcoin_svg('main', 60) }}<br>
-        <br>Configue your node</h1>
+        <br>Configure your node</h1>
         <p>QuickSync?&nbsp;&nbsp;
             <label class="switch">
                 <input type="checkbox" name="quicksync" id="quicksync" onchange="toggleNodeTypeSelect()" checked>

src/cryptoadvance/specter/util/bitcoind_setup_tasks.py

@@ -16,6 +16,9 @@ def setup_bitcoind_thread(specter=None, internal_bitcoind_version=""):
             "Linux": "x86_64-linux-gnu.tar.gz",
             "Darwin": "osx64.tar.gz",
         }
+        # ARM Linux devices (e.g. Raspberry Pi 4 == armv7l) need ARM binary
+        if platform.system() == "Linux" and "armv" in platform.machine():
+            BITCOIND_OS_SUFFIX["Linux"] = "arm-linux-gnueabihf.tar.gz"
         bitcoind_url = f"https://bitcoincore.org/bin/bitcoin-core-{internal_bitcoind_version}/bitcoin-{internal_bitcoind_version}-{BITCOIND_OS_SUFFIX[platform.system()]}"
         packed_name = os.path.join(
             specter.data_folder,
@@ -165,15 +168,15 @@ def setup_bitcoind_directory_thread(specter=None, quicksync=True, pruned=True):
 
         specter.config["bitcoind_setup"]["stage"] = "Starting up Bitcoin Core..."
         specter._save()
-        specter.bitcoind = BitcoindPlainController(
-            bitcoind_path=specter.bitcoind_path,
-            rpcport=8332,
-            network="mainnet",
-            rpcuser=specter.config["rpc"]["user"],
-            rpcpassword=specter.config["rpc"]["password"],
-        )
+
+        # Specter's 'bitcoind' attribute will instantiate a BitcoindController as needed
+        timeout = 30
+        if platform.system() == "Linux" and "armv" in platform.machine():
+            # Raspberry Pi will need more time to spin up bitcoind
+            timeout = 60
         specter.bitcoind.start_bitcoind(
-            datadir=os.path.expanduser(specter.config["rpc"]["datadir"])
+            datadir=os.path.expanduser(specter.config["rpc"]["datadir"]),
+            timeout=timeout,
         )
         specter.set_bitcoind_pid(specter.bitcoind.bitcoind_proc.pid)
         specter.update_use_external_node(False)
@@ -191,7 +194,7 @@ def setup_bitcoind_directory_thread(specter=None, quicksync=True, pruned=True):
             specter._save()
         specter.check()
     except Exception as e:
-        logger.error(f"Failed to setup Bitcoin Core. Error: {e}")
+        logger.exception(f"Failed to setup Bitcoin Core. Error: {e}")
         specter.config["bitcoind_setup"]["error"] = str(e)
         specter._save()
     finally:

src/cryptoadvance/specter/util/tor_setup_tasks.py

@@ -1,4 +1,4 @@
-import os, time, requests, secrets, platform, tarfile, zipfile, sys, subprocess, shutil, stat, zipfile, logging
+import os, time, requests, platform, tarfile, zipfile, sys, subprocess, shutil, stat, zipfile, logging
 import pgpy
 from pathlib import Path
 from .sha256sum import sha256sum
@@ -19,6 +19,11 @@ def copytree(src, dst, symlinks=False, ignore=None):
 
 def setup_tor_thread(specter=None):
     try:
+        # There is no Tor Browser binary for Raspberry Pi 4 (armv7l)
+        if platform.system() == "Linux" and "armv" in platform.machine():
+            raise Exception(
+                "Linux ARM devices (e.g. Raspberry Pi) must manually install Tor"
+            )
         specter.config["torbrowser_setup"]["stage"] = "Starting Tor setup process..."
         specter._save()
         TOR_OS_SUFFIX = {
@@ -113,8 +118,7 @@ def setup_tor_thread(specter=None):
                             os.chmod(destination_file, st.st_mode | stat.S_IEXEC)
         os.remove(packed_name)
         if "torrc_password" not in specter.config:
-            specter.config["torrc_password"] = secrets.token_urlsafe(16)
-            specter._save()
+            specter.generate_torrc_password()
         with open(os.path.join(specter.data_folder, "torrc"), "w") as file:
             file.write("ControlPort 9051")
             file.write(