Raspberry Pi 4 and Raspbian Buster / Docker

[ndeet]

Playing around with rpi 4 with 2GB ram and a 512GB microSD card (no external drive)

Writing down notes on differences to current docs here: https://docs.btcpayserver.org/deployment/raspberrypideployment

Step 3:

the link says "Raspbian Stretch Lite" but it links to https://downloads.raspberrypi.org/raspbian_lite_latest which is now Raspbian Buster (which currently breaks btcpay-setup.sh, see Step 14 below)

Step 12+13:

maybe can be made optional with rpi4. Currently testing a 512GB Samsung EVO microSD card but only ran it for 1h now, needs more testing

Step 14:

installing docker on stable buster channel currently does not work and . ./btcpay-setup.sh -i fails with E: Package 'docker-ce' has no installation candidate

This will break also current rpi 3 I guess as it is Raspian Buster related.

Upstream issue here docker/for-linux#709

Temporary fix, should not be done on offical repo imo:
I made it work by doing the step of the setup script manually and pass the nightly channel and run setup again.

curl -fsSL https://get.docker.com -o get-docker.sh
CHANNEL="nightly" sh get-docker.sh
. ./btcpay-setup.sh -i

Step 14 (take 2):

export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-xs;opt-save-memory"
With rpi 4 with 2 or 4 GB of ram the fragment opt-save-memory is not needed anymore. Also opt-save-storage-xs;opt-save-memory is only needed if you use small usb drive with pruning. (When I run a full node at home I want to have the full chain on my rpi to tinker around, but maybe that's only me.)

off topic: document tor only mode

Add info how to run Tor only with no need for public static IP and opening the router etc. It just works if you follow the docs but maybe be more precise that eg. nginx + host is not needed etc.
see #157

[NicolasDorier]

nginx is needed in my opinion or you will miss many feature in BTCPay.

The way of doing it is using asahi, having a mdns name like btcpay.local and use that as the Host.
I think putting an IP will not work.

Ping @jmaurice

[ndeet]

re Tor only:
ok thanks for the info, mdns is preferred yes. I did add a host and added it to my /etc/hosts but that's not something that should be in the docs I guess, maybe as troubleshoot instructions if the mdns of the .local domain does not work?

re rpi 4 + sdcard:
The testnet synced in 4-5 h (~30GB) and runs fine, the sdcard has ~30MB/s (for bigger files) write speed which seems to be enough. I read a lot that sdcard should not be used because of speed (on rpi3) but also because of durability. I found no info on that that does say that for sdcard explicitly but more for all flash based memory in general which includes also usb dongles. So not sure if the usb brings any benefits on a rpi4 if the write speed is not the bottleneck anymore.
I like that I do not have a big external drive or usb dongle attached to rpi (like on raspibolt + raspiblitz), also less power consumption and ease of install (no formatting + mounting on install process).
But again, have no long-term data here on durabilty, so guess time will tell.

pruned node + FastSync:
I want a full node history locally to eventually use it to play around and or look up transactions anonymously and use the bitcoin node directly via rpc. I'm not sure if a pruned node has still full block explorer capabilities etc. I read the FastSync docs and the option to verify FastSync snapshot against another fullnode myself is somewhat ok but still does not feel "right". Are there any other drawbacks other than the ones listed here https://github.com/btcpayserver/btcpayserver-docker/tree/master/contrib/FastSync#lightning-network-routing-issues about lightning channels?

Maybe we should add a second rpi guide or additional section "non pruned full rpi node for developers"?

Also it seems that full sync can be done in 96h on a rpi 4 (with ssd though, likely sdcard slower) raspiblitz/raspiblitz#592 (comment)
Other than that seems that rpi4 is a beast and much more capable than rpi3.

[NicolasDorier]

agree, I am also interested into having an idea in how much time it took to sync for mainnet.

[ndeet]

my provider will hate me downloading 250 GB data but I will give it a try, having only 30-50/Mbit/s downstream at home will be a challenge but will try in a few days

re Tor only:
Another issue coming up due to using tor only and doing no IPv4 forwarding in local network is that the SSL issuance fails as letsencrypt can't validate. It seems that Zap only accepts connections via https it seems? The gRPC config that shows up on BTCPay is like this (no SSL + port 80):

{
"configurations": [
{
"host": "btctn1.local",
"port": 80,
"ssl": false,
"chainType": "Testnet",
"type": "grpc",
"cryptoCode": "BTC",
"certificateThumbprint": null,
.... <snip macaroons>

When trying to use this in Zap it fails with "unable to connect error, make sure TLS certi is valid..". Afaik LND requires/forces SSL connections. I guess even self signed certificate will fail here maybe but not sure if Zap only cares about the cert validity together with hostname or also the CA root etc. Did not try yet.

I tried also the .onion host and set port to 443 + ssl to true but that fails as expected because DNS tries to resolve the .onion TLD and runs into nirvana.

So not sure what the best solution is here. Either make it working using .onion or do self signed cert and import it to local trust store. Or somehow detect .local hosts and make non SSL connections work or use the LND generated default TLS for this?

[wiz]

@ndeet I sent you a draft of an article I've been writing for running BTCPay on RPI4 with 4GB ram, I was also working this past week on making a Tor only setup. But to address some of your specific concerns on this issue, I think I arrived at all the same conclusions you did I just beat you by a few days ;)

1. I agree opt-save-memory is no longer necessary and since I have 4GB I actually added a new option for opt-more-memory in Add opt-moar-memory to increase dbcache to 1GB in bitcoin.conf #168

2. I'm using a 500GB SSD on my RPI so I don't use pruning and also don't use opt-save-storage like is usually necessary with RPI installations

3. As you saw for the https://raspberrypi.local self-signed cert, Nicolas merged my PR last night so should work now if you do BTCPAY_HOST=raspberrypi.local Automatically enable self signed SSL cert if using *.local hostname #172

4. Regarding E: Package 'docker-ce' has no installation candidate error, I didn't submit a PR for this, I only sent Nicolas a DM about it on Keybase, my patch was simply to add a |sed -e 's/buster/stretch/' to the curl https://get.docker.com command in btcpay-setup.sh but I think your using the 'nightly' channel is a cleaner workaround until it gets merged to the 'stable' channel.

Let's exchange notes on mattermost chat and discuss what PR(s) we should do in addition to the ones I already made?

[NicolasDorier]

For 3. I don't think it will work @jmaurice . Zap will not trust the certificate probably. Maybe if you add the certthumbprint ? unsure Zap did it as nobody do it.

[ndeet]

@jmaurice @NicolasDorier
3. tried it with master branch and self signed cert; unfortunately need to confirm that Zap fails with error that it needs a valid cert

[wiz]

Does Zap support Tor? It's not very useful to connect to Zap over your LAN.

[ndeet]

Nah, does not work with Tor, mentioned it above, the electrum app does normal dns lookup which fails. So you would have to somehow make it go through socks to find its way to .onion

I tried also the .onion host and set port to 443 + ssl to true but that fails as expected because DNS tries to resolve the .onion TLD and runs into nirvana.

Other than that, RTL works now thanks to your PR so that is fine for me now as I mainly find it useful for channel managment. Wanted to check Zap just because of interest, tried it last time a year ago or so.

[NicolasDorier]

@jmaurice there is @mandelduck who made his wallet works with orbot and VPN mode. Work only for REST, not grpc.

[NicolasDorier]

Because grpc sucks. But I think I already covered the topic.

[NicolasDorier]

@ndeet I pushed new version of RTL, now it works even on http.

[NicolasDorier]

I think you guys should ping the library authors that rely on gRPC and make them work without HTTPS if accessing onion website.

[mandelduck]

I think its easy to make grpc work over http, I left an issue on grpc github about tor an waiting for a response

…

On Tue, Jul 23, 2019 at 4:58 PM Nicolas Dorier ***@***.***> wrote: I think you guys should ping the library authors that rely on gRPC and make them work without HTTPS if accessing onion website. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#170?email_source=notifications&email_token=AB3Y5H74S6ZBK2NX5HDAR53QA22ZTA5CNFSM4IEIIOQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2SII2Y#issuecomment-514098283>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB3Y5H2F4YG2HK35DTCN7ITQA22ZTANCNFSM4IEIIOQA> .

[NicolasDorier]

Closing this #176 fix it