Working standup scripts #15

jodobear · 2020-09-28T14:22:16Z

Installs:

Bitcoin core using Tor - specify external data dir
C-lightning
LND
Esplora - specify external data dir
BTCPay Server needs blockchain data corruption issue resolved.

…r management of code

ChristopherA · 2020-10-01T00:53:36Z

@Fonta1n3 @shannona Can you review this PR early next week?

shannona

The scripts are looking very nice. I like the organization and the use of the config file.

I did a medium-level dive, reading the README, reading the config file, and trying out an install on a fairly fresh Debian 9 system with c-lightning. Generally looks great, but there seem to be a few discrepancies in the script and more notably c-lightning didn't install.

shannona · 2020-10-06T23:00:59Z

Scripts/scripts/ss.conf

+# if you are going to access it locally then you can just use your hostname and add .local like so: hostname.local
+FQDN=
+# the region in which the server is hosted. Required for timezone settings
+REGION=


Need to give an example of this, or tell users where to go to get the right format.

Added the following:

FQDN : e.g. myawesomedomain.com

REGION : e.g. UCT, GMT, Pacific, America/New_York, Europe/Madrid, Asia/Kolkata

debian comes with curl, so the commands below might be useful if you don't know the codes by heart

curl https://ipinfo.io/ip

curl https://json.geoiplookup.io/<Public_IP>

Although this would probably only work with a VPS (unless they have a static IP)

shannona · 2020-10-06T23:02:13Z

Scripts/scripts/ss.conf

+###########################
+# The bitcoin network you want bitcoind to start with. Valid choices:
+# mainnet, testnet or regtest
+NETWORK=mainnet


I'd suggest we default this to testnet, to force someone to purposefully change it to choose to engage with real funds.

shannona · 2020-10-06T23:02:56Z

Scripts/scripts/README.md

+
+3. Edit config for your node setup using your favourite text editor:
+# nano ss.conf
+


It'd be helpful to say. "You must change [A], and you will probably want to change [B] and [C], but look at the whole file.

Added the following:

You must modify USERPASS else, your server will be unprotected and provide values for all the variables under VPS if installing on a VPS and set to true. Do note that the script defaults to testnet full node. Make appropriate changes.

shannona · 2020-10-06T23:17:03Z

Scripts/scripts/ss.conf

+# c-lightning options
+###########################
+# http-plugin
+# CLN_HTTP_PLUGIN=flase


This hasn't been implemented, hence set to false

shannona · 2020-10-06T23:18:02Z

Scripts/scripts/README.md

+
+5. Display help:
+# source ss_00_main.sh -h
+


Shouldn't be an extra step. (This implies that you need to source the script, then get the help.)

changed to source ss_01_help.sh

shannona · 2020-10-06T23:21:21Z

Scripts/scripts/ss_00_main.sh

+# prompt user before continuing with installation
+if ! "$NOPROMPT"; then
+  read -rp  "Continue with installation? (Y/n): " confirm
+fi


This ("Y/n") implies that "Y"es is the default, but when I just hit return, it said "Entered . Exiting.." and quit.

Either this should be set to "Y" as a default (e.g., $confirm = "Y") or it should requery if you fail to enter an answer.

shannona · 2020-10-06T23:26:07Z

Scripts/scripts/ss_00_main.sh

+cp -r $SCRIPTS_DIR ~standup/
+chown standup ~standup/scripts-conf
+cd ~standup/scripts-conf
+rm -r $SCRIPTS_DIR


This seems to be the first use of ~standup/scripts-conf:

chown: cannot access '/home/standup/scripts-conf': No such file or directory
-bash: cd: /home/standup/scripts-conf: No such file or directory

We should probably tell people that if they want to review the scripts they're in ~standup/scripts

Done. Added following to README:

Move the standup-scripts directory to /home/standup/standup-scripts. If you want to review the scripts later, you can find them at this location.

shannona · 2020-10-06T23:28:54Z

Scripts/scripts/ss_06_c-lightning.sh

+  sudo -u standup git clone https://github.com/Start9Labs/c-lightning-http-plugin.git "$LIGHTNING_DIR"/plugings/
+  cd "$LIGHTNING_DIR"/plugings/c-lightning-http-plugin/
+  cargo build --release
+  chmod a+x /home/you/.lightning/plugins/c-lightning-http-plugin/target/release/c-lightning-http-plugin


/home/you ?

shannona · 2020-10-06T23:36:30Z

Scripts/scripts/ss_06_c-lightning.sh

+#!/bin/bash
+
+# standup script - install c-lightning
+


Some type of fatal error here:

-------Standup - downloading & Installing c-lightning

fatal: Could not change back to '/root/Bitcoin-Standup-Scripts/Scripts/scripts': Permission denied
-bash: cd: /home/standup/lightning: No such file or directory
error: pathspec 'v0.9.1' did not match any file(s) known to git.
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
-bash: ./configure: No such file or directory
make: *** No targets specified and no makefile found. Stop.
make: *** No rule to make target 'install'. Stop.
-bash: : command not found

Debian 9. Fresh Linode (with a bit of security added). Run as root.

That's strange. If run as root, it should be able to cd. I haven't had issues with this. What security measures might have caused this?

Did you initiate c-lightning installation after completing the installation of bitcoind?

shannona · 2020-10-06T23:51:06Z

Scripts/scripts/README.md

+
+QR Code:
+--------
+Upon completion of the script there will be a QR code saved to /qrcode.png which you can open and scan:


I don't see anything that installs a qrcode.png right now.

ls -lagh /qrcode.png

ls: cannot access '/qrcode.png': No such file or directory

Good catch. I definitely missed that one

shannona · 2020-10-06T23:53:45Z

After this settles, my next question is going to be: how can we merge this into the StackScript for Linode, since that's our preferred methodology for the Learning Bitcoin course (and at the least we'd like the automatic c-lightning install.)

jodobear · 2020-10-16T16:20:46Z

After this settles, my next question is going to be: how can we merge this into the StackScript for Linode, since that's our preferred methodology for the Learning Bitcoin course (and at the least we'd like the automatic c-lightning install.)

I can just merge relevant parts into a single script with c-lightning for the StackScript. Does that work?

shannona · 2020-11-03T23:53:17Z

After this settles, my next question is going to be: how can we merge this into the StackScript for Linode, since that's our preferred methodology for the Learning Bitcoin course (and at the least we'd like the automatic c-lightning install.)

I can just merge relevant parts into a single script with c-lightning for the StackScript. Does that work?

Definitely!

ChristopherA · 2021-07-13T18:41:58Z

@jodobear wrote

I can just merge relevant parts into a single script with c-lightning for the StackScript. Does that work?

Did this happen? Should this PR be closed, or is there still useful code for the future here?

ChristopherA · 2021-07-16T22:43:57Z

@icculp @nochiel — any desire to pick up this PR, test it, and see if we can't merged into master?

nochiel · 2021-07-21T02:22:50Z

@icculp @nochiel — any desire to pick up this PR, test it, and see if we can't merged into master?

I've begun reviewing the PR.

icculp · 2021-07-21T02:36:25Z

Funny timing Nick, I started looking at it just a little while ago as well

nochiel

Looks good to me. I've annotated some scripts with a few comments/questions.

nochiel · 2021-08-08T23:52:10Z

Scripts/scripts/ss_00.1_vps.sh

@@ -0,0 +1,60 @@
+#!/bin/bash


Assuming availability of coreutils, shouldn't the shebang be#!/usr/bin/env bash instead of hardcoding the location of bash?

nochiel · 2021-08-08T23:54:11Z

Scripts/scripts/ss_00_main.sh

+bold=$(tput bold)
+normal=$(tput sgr0)
+underline=$(tput smul)


Should the script check for ncurses before using tput? (Or can we assume that it is always install wtih the debian base system.)

Update: No need to check for ncurses because the Debian base system install contains:

# aptitude search ~pimportant ~prequired ~pimportant -F%p
adduser
apt
apt-utils
base-files
base-passwd
bash
bsdmainutils
bsdutils
coreutils
cpio
cron
dash
debconf
debconf-i18n
debian-archive-keyring
debianutils
diffutils
dmidecode
dpkg
e2fslibs
e2fsprogs
fdisk
findutils
gdbm-l10n
gpgv
grep
gzip
hostname
ifupdown
init
init-system-helpers
iproute2
iptables
iputils-ping
isc-dhcp-client
isc-dhcp-common
kmod
less
libapt-inst2.0
libapt-pkg5.0
libc-bin
libcomerr2
libpam-modules
libpam-modules-bin
libpam-runtime
libss2
libudev1
login
logrotate
mawk
mount
nano
ncurses-base
ncurses-bin
netbase
passwd
perl-base
procps
readline-common
rsyslog
sed
sensible-utils
systemd
systemd-sysv
sysvinit-utils
tar
tasksel
tasksel-data
tzdata
udev
util-linux
vim-common
vim-tiny
whiptail

nochiel · 2021-08-09T11:15:00Z

Scripts/scripts/README.md

+5. Display help:
+# source ss_00_main.sh -h
+
+This script can be installed on any Debian based system. By default this script will:


Suggested change

This script can be installed on any Debian based system. By default this script will:

This script can be installed on any Debian based system. The [Debian base system](https://www.debian.org/doc/debian-policy/ch-binary.html#base-system) installs the following packages which this script assumes are [standard](https://wiki.debian.org/tasksel#A.22standard.22_task):

<pre>

$ aptitude search ~pimportant ~prequired ~pimportant -F%p

adduser

apt

apt-utils

base-files

base-passwd

bash

bsdmainutils

bsdutils

coreutils

cpio

cron

dash

debconf

debconf-i18n

debian-archive-keyring

debianutils

diffutils

dmidecode

dpkg

e2fslibs

e2fsprogs

fdisk

findutils

gdbm-l10n

gpgv

grep

gzip

hostname

ifupdown

init

init-system-helpers

iproute2

iptables

iputils-ping

isc-dhcp-client

isc-dhcp-common

kmod

less

libapt-inst2.0

libapt-pkg5.0

libc-bin

libcomerr2

libpam-modules

libpam-modules-bin

libpam-runtime

libss2

libudev1

login

logrotate

mawk

mount

nano

ncurses-base

ncurses-bin

netbase

passwd

perl-base

procps

readline-common

rsyslog

sed

sensible-utils

systemd

systemd-sysv

sysvinit-utils

tar

tasksel

tasksel-data

tzdata

udev

util-linux

vim-common

vim-tiny

whiptail

</pre>

By default this script will:

nochiel · 2021-08-09T11:17:35Z

Scripts/scripts/ss_00.1_vps.sh

@@ -0,0 +1,60 @@
+#!/bin/bash


Suggested change

#!/bin/bash

#!/usr/bin/env bash

nochiel · 2021-08-09T11:17:55Z

Scripts/scripts/ss_00_main.sh

@@ -0,0 +1,320 @@
+#!bin/bash


Suggested change

#!bin/bash

#!/usr/bin/env bash

nochiel · 2021-08-19T15:46:27Z

Scripts/scripts/ss_03_user_ssh.sh

+if [ -n "$SSH_KEY" ] && [[ "$SSH_KEY" != "__UNDEFINED__" ]]; then
+  mkdir ~standup/.ssh
+  echo "$SSH_KEY" >> ~standup/.ssh/authorized_keys
+  chown -R standup ~standup/.ssh


Suggested change

chown -R standup ~standup/.ssh

chown -R standup ~standup/.ssh

chmod 0400 ~standup/.ssh/authorized_keys

nochiel · 2021-08-19T16:01:34Z

Scripts/scripts/ss_04_tor.sh

+"
+#  To use source lines with https:// in /etc/apt/sources.list the apt-transport-https package is required. Install it with:
+if [ -z "$(which apt-transport-https)" ]; then
+  apt-get install apt-transport-https -y


Suggested change

apt-get install apt-transport-https -y

apt-get install apt-transport-https -y

apt-get install lsb-release apt-transport-https -y

Make sure the user has lsb-release so that setting '/etc/apt/sources.list has a valid distribution.

nochiel · 2021-08-19T16:02:03Z

Scripts/scripts/ss_04_tor.sh

+fi
+
+# We need to set up our package repository before you can fetch Tor. First, you need to figure out the name of your distribution:
+DEBIAN_VERSION=$(lsb_release -c | awk '{ print $2 }')


Suggested change

DEBIAN_VERSION=$(lsb_release -c | awk '{ print $2 }')

DEBIAN_VERSION=$(lsb_release -c | awk '{ print $2 }')

if [[ -z "$DEBIAN_VERSION" ]]; then

DEBIAN_VERSION="stable"

fi

nochiel · 2021-08-19T16:51:29Z

Scripts/scripts/ss_06_c-lightning.sh

+$MESSAGE_PREFIX downloading & Installing c-lightning
+"
+# get & compile clightning from github
+sudo -u standup git clone https://github.com/ElementsProject/lightning.git ~standup/lightning


Is it preferable to compile from source instead of using pre-compiled binaries?

nochiel · 2021-08-19T16:52:08Z

Scripts/scripts/ss_06_lnd.sh

+echo "
+$MESSAGE_PREFIX getting lnd... depending on your network it can take more than an hour. With good network it usually takes about 5-10 mins.
+"
+go get -d github.com/lightningnetwork/lnd


Is it preferable to compile from source instead of using pre-compiled binaries?

ChristopherA · 2021-08-19T19:28:15Z

@shannona is your review at #15 now resolved?

EniRox001 · 2022-08-24T09:11:00Z

Scripts/scripts/ss_07_esplora.sh

+apt-get install nodejs -y
+
+#  get esplora & set electrs api url
+sudo -u standup git clone https://github.com/Blockstream/esplora "$ESPLORA_REPO"


Would like to change this repo to link to the new esplora fork at BlockchainCommons/esplora

jodobear added 15 commits July 5, 2020 04:28

added merged standup script

88312c6

fixed tor install and installing bitcoin-core via onion site

548be9f

fully functioning installation of bitcoin-core via onion site

bbe53d4

divided the script into logical parts for better management of code

d20f86a

added scripts dir and divided the script into logical parts for bette…

300ecc5

…r management of code

working lnd installation & code cleanup

1e0b216

base working scripts to develop on

ec03d0e

perfectly working lightning installation

4603577

working c-lightning, lnd and btcpayserver using local bitcoin data

d7a9b32

complete rework & working ss using config instead of cli args

818589e

removed older scripts

751fd76

removed old scripts, updated working & tested scripts

3435a64

rm duplicate user_ssh

bacc8a2

rm ss.sh

8eb8b68

rm ss.sh

d164d47

jodobear requested a review from ChristopherA as a code owner September 28, 2020 14:22

jodobear added 2 commits September 28, 2020 16:36

added README & commented out btcpay settings

fa9025f

Merge branch 'jodobear'

d5fe388

shannona requested changes Oct 6, 2020

View reviewed changes

shannona mentioned this pull request Nov 3, 2020

2020 Summer Internship Project: Lightning Support for Bitcoin Standup BlockchainCommons/Community#7

Closed

nochiel approved these changes Aug 19, 2021

View reviewed changes

shannona mentioned this pull request May 25, 2022

PROJECT: Modular Bitcoin-Standup-Scripts BlockchainCommons/Community#121

Open

EniRox001 suggested changes Aug 24, 2022

View reviewed changes

ChristopherA mentioned this pull request Dec 2, 2022

Uncomplicate btcblock alias and fix 301 error BlockchainCommons/Learning-Bitcoin-from-the-Command-Line#593

Merged


		3. Edit config for your node setup using your favourite text editor:
		# nano ss.conf

-This script can be installed on any Debian based system. By default this script will:
+This script can be installed on any Debian based system. The [Debian base system](https://www.debian.org/doc/debian-policy/ch-binary.html#base-system) installs the following packages which this script assumes are [standard](https://wiki.debian.org/tasksel#A.22standard.22_task):
+<pre>
+$ aptitude search ~pimportant ~prequired ~pimportant -F%p
+adduser
+apt
+apt-utils
+base-files
+base-passwd
+bash
+bsdmainutils
+bsdutils
+coreutils
+cpio
+cron
+dash
+debconf
+debconf-i18n
+debian-archive-keyring
+debianutils
+diffutils
+dmidecode
+dpkg
+e2fslibs
+e2fsprogs
+fdisk
+findutils
+gdbm-l10n
+gpgv
+grep
+gzip
+hostname
+ifupdown
+init
+init-system-helpers
+iproute2
+iptables
+iputils-ping
+isc-dhcp-client
+isc-dhcp-common
+kmod
+less
+libapt-inst2.0
+libapt-pkg5.0
+libc-bin
+libcomerr2
+libpam-modules
+libpam-modules-bin
+libpam-runtime
+libss2
+libudev1
+login
+logrotate
+mawk
+mount
+nano
+ncurses-base
+ncurses-bin
+netbase
+passwd
+perl-base
+procps
+readline-common
+rsyslog
+sed
+sensible-utils
+systemd
+systemd-sysv
+sysvinit-utils
+tar
+tasksel
+tasksel-data
+tzdata
+udev
+util-linux
+vim-common
+vim-tiny
+whiptail
+</pre>
+By default this script will:

	chown -R standup ~standup/.ssh
	chown -R standup ~standup/.ssh
	chmod 0400 ~standup/.ssh/authorized_keys

	apt-get install apt-transport-https -y
	apt-get install apt-transport-https -y
	apt-get install lsb-release apt-transport-https -y

Working standup scripts #15

Are you sure you want to change the base?

Working standup scripts #15

Conversation

jodobear commented Sep 28, 2020

ChristopherA commented Oct 1, 2020

shannona left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jodobear Oct 16, 2020 • edited Loading

Choose a reason for hiding this comment

matthiasdebernardini Dec 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jodobear Oct 16, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ls -lagh /qrcode.png

Choose a reason for hiding this comment

shannona commented Oct 6, 2020

jodobear commented Oct 16, 2020

shannona commented Nov 3, 2020

ChristopherA commented Jul 13, 2021

ChristopherA commented Jul 16, 2021

nochiel commented Jul 21, 2021

icculp commented Jul 21, 2021

nochiel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ChristopherA commented Aug 19, 2021

Choose a reason for hiding this comment

jodobear Oct 16, 2020 •

edited

Loading

matthiasdebernardini Dec 18, 2020 •

edited

Loading

jodobear Oct 16, 2020 •

edited

Loading