Upgrade 0.5.0 LND ignores tlsextradomain and tlsextraip

[mbio16]

Hello,

I upgraded my node to 0.5.0. After that LND regenerated TLS cert and key, however it ignores lnd.conf where is tlsextradomain and ip is configured.

I remove umbrel.local from lnd.conf as tlsextradomain param and removed the TLS.key and cert files. Restart of the LND container regenerated cert and key but with same umbrel.local domain and no other tlsextradomain dns name.

[kroese]

I think it is caused by the commandline parameters now specified in docker.

They override the values in the config file, the same problem as the Bitcoin container now has.

So you can only modify settings that are not already in the commandline, this new behaviour is really annoying.

[mbio16]

After I read your issue, it was clear this is the same for lnd as well. The weird thing is that when I look in RTL to LND conf, it knows all the params from conf. However, It is ignored. It would be great to have a param in docker-compose to specifiy path to conf file and if the param is set, this conf file overides default docker-compose commandline parameters.

[lightningorb]

Same issue here. Checking for how this can be fixed with docker-compose.yaml

[lightningorb]

As much as I try not to let things like this get to me, it's true that ignoring lnd.conf may be a somewhat annoying design decision.

[lightningorb]

I'm noticing two files of interest:

./app-data/lightning/exports.sh
./repos/https---github-com-getumbrel-umbrel-apps-git/lightning/exports.sh 

(they seem to originate from https://github.com/getumbrel/umbrel-apps)

Oddy, modifying the --tlsextraip flag also doesn't seem to have the desired effect. I'm also wondering whether this flag can appear multiple times, or if the ips need to be comma seperated.

[lightningorb]

Hmm my conclusion is that this isn't the right approach. i.e shouldn't try overriding tlsextraip. Instead should connect to LND via the 'connect wallet' functionality giving via the interface.

[kroese]

@bc31164b-cfd5-4a63-8144-875100622b2d I am happy if that is a solution for you, but for many people that is not an option.

I run many extra containers, and I cannot put them in umbrel_main_network or umbrel will give an error while stopping (that it cannot delete the network). But if I put them in separate networks, it will require tlsextraip which is not possible any longer. So this issue is really a problem.

[felipehere]

I'm dealing with the same issue here... LND ignores new tlsextraip parameters. Were you able to work around this? I haven't found further discussion about this topic.

[mbio16]

no, there is no current workaround. TLS cert is renew every time when container is restarted. Use tor lnd connect to connect to your mobile app.

[lukechilds]

As @bc31164b-cfd5-4a63-8144-875100622b2d mentions:

Hmm my conclusion is that this isn't the right approach. i.e shouldn't try overriding tlsextraip. Instead should connect to LND via the 'connect wallet' functionality giving via the interface.

This is really the correct approach. @kroese the app sandbox on Umbrel provides security by limiting what apps have access to. You shouldn't expect a container running directly on the OS to be able to interface directly with a container in apps. This does happen to work today if you reference them directly by their internal IP, but this will break in the future when we lock down network sandboxing further.

You should connect over the Umbrel's local network interface, Tor, or something like Tailscale.

Alternatively if you want to keep your current setup, knowing it will break at some point in the future, and just want to get it working now. You could just pin your LND client to your SSL certificate and then disable host/ip verification. Or even just disable SSL verification all together. It's really not providing any security between two peers on an internal virtual network.

I'm interested @mbio16 @felipehere @bc31164b-cfd5-4a63-8144-875100622b2d what are your use cases for wanting to change this setting? Are you also running external containers on the OS that you want to directly interface with LND like @kroese? Or something else?

If there is some common need we aren't satisfying then we can see if there's something we can change to support it better.

[felipehere]

Hi @lukechilds, thank you for jumping in.

My goal is to set up LNbits on the clearnet so that I can more properly host Lightning wallets for friends and family. As you know, the 'connect wallet' functionality only offers Tor and local network connections.

So I've installed an LNbits instance on a VPS (which I also use to route the node's connection to the clearnet on hybrid mode) and I'm trying to allow it to use my node's wallet. In order to allow LNbits on the VPS to access the node's restwallet API, I need to set up a new tlsexternalip on lnd.conf and generate a new tls.cert file that LNbits on the VPS can use.

It would be great to learn your thoughts about what I'm trying to achieve and if Umbrel could support it somehow.

[mbio16]

Hi @lukechilds,

The reason is that I have domain name with DNSSEC setup to my LND. I have own LND rest client (on different HW). Cert has to be valid, disable verification is not right sec approach.

[TrezorHannes]

Hi @lukechilds, thank you for jumping in.

My goal is to set up LNbits on the clearnet so that I can more properly host Lightning wallets for friends and family. As you know, the 'connect wallet' functionality only offers Tor and local network connections.

So I've installed an LNbits instance on a VPS (which I also use to route the node's connection to the clearnet on hybrid mode) and I'm trying to allow it to use my node's wallet. In order to allow LNbits on the VPS to access the node's restwallet API, I need to set up a new tlsexternalip on lnd.conf and generate a new tls.cert file that LNbits on the VPS can use.

It would be great to learn your thoughts about what I'm trying to achieve and if Umbrel could support it somehow.

This is a common usecase, which is used by a number of umbrel nodes following my guide, and with 0.5 not feasible anymore.

[lukechilds]

Got it, thanks for the responses guys.

Totally get the public LNbits use case, we have some ideas on how we can support this use case better directly in Umbrel in the future. In the meantime, here are some ideas for workarounds:

1. (I would suggest this) You could run both LNbits and LND on your Umbrel, then just run a VPS that proxies clearnet traffic on the VPS to the LNbits instance on your Umbrel. There is an example for how to do this with BTCPay here: https://jorijn.com/installing-nginx-reverse-proxy-with-ssl-certificate-umbrel-btcpayserver/ You could take the same approach for LNbits.

2. If you're happy using Tailscale you could run Tailscale on both your Umbrel and your VPS. Then you can enable Tailscale MagicDNS and we can update the Lightning app to pass the Umbrel's hostname via tlsextradomain to LND. This means you can run LNbits on the VPS and connect to LND on your Umbrel over Tailscale MagicDNS at LND_REST_ENDPOINT="https://umbrel:8080" which should now have a valid SSL cert.

3. If you don't want to use Tailscale you could use a VPN or SSH tunnel to proxy the Umbrel LND ports to the VPS (like @TrezorHannes current guide). You could then set umbrel.local to resolve to 127.0.0.1 in /etc/hosts on the VPS and connect via LND_REST_ENDPOINT="https://umbrel.local:8080". So requests to umbrel.local on the VPS resolve to localhost which then hits a local port that gets tunneled to the Umbrel over VPN/SSH. It's a bit of a hack but this should satisfy the clients SSL validation logic and require minimal changes from the current guide.

[TrezorHannes]

Appreciate the constructive ideas Luke. I really do.
But I'd rather keep the guide as is, and keep suggesting to umbrel-users to just not upgrade. Nothing in it for them, just more complexity and handcuffs.

LND has the tlsextraip for the secured handshake, and LNBits is only one of many instances where the rest-wallet needs to validate a secure connection. I don't get why we take secure, established ways which are working for raspibolt, blitz, start9, mynode-users, but not supposed to work for umbrel.

[lukechilds]

Option 3 I suggested above does keep this same method but allows it to work for Umbrel. It just requires setting up the hostname on the VPS end instead of adding an IP on the LND end. It would just be a one line change on the VPS:

echo '127.0.0.1 umbrel.local' | sudo tee -a /etc/hosts

FWIW the behaviour that's preventing your current guide from working could be considered a bug in LND:

; Adds an extra ip to the generated certificate. Setting multiple tlsextraip= entries is allowed.
; (old tls files must be deleted if changed)
; tlsextraip=

https://github.com/lightningnetwork/lnd/blob/fec8fd9c63dc672dc3ea1b9be47bd1d6c4a0a54d/sample-lnd.conf#L42-L44

According to the LND docs multiple tlsextraip args are allowed to apply multiple IPs. So it seems like setting tlsextraip in lnd.conf should be merged in with the default IP we apply in Umbrel via the --tlsextraip CLI arg. Not overriding it which is what it seems to be currently doing. Let me talk to Lightning Labs and see if this is a bug and something that's possible to fix.

[TrezorHannes]

Thanks, that would be the best way forward.

I had 5 tlsextraip entries in umbrel < 0.5, and more than one in raspiblitz and raspibolt. So it's certainly working if defined in lnd.conf. But never tried adding additional via CLI.

Is there a way for me to adjust docker-compose and add additional --tlsextraip flags?
I will also leverage this for my own needs to regenerate my deleted tls.cert via --tlsautorefresh.

[TrezorHannes]

This might be something to look into

; Do not include the interface IPs or the system hostname in TLS certificate,
; use first --tlsextradomain as Common Name instead, if set.
; tlsdisableautofill=true

umbrel@umbrel:~ $ cat umbrel/app-data/lightning/exports.sh | grep tlsdisableautofill    
BIN_ARGS+=( "--tlsdisableautofill" )

[lukechilds]

OK so heard back from Lightning Labs, this is correct LND behaviour:

While I see why it would be useful in your case, I think it would be confusing in general if the override/precedence behavior is different for some of the values. And that new behavior wouldn't allow you to overwrite multi-value config file parameters anymore from the CLI.

So we'll have to handle this specially somehow in the Lightning app to get it working.

Is there a way for me to adjust docker-compose and add additional --tlsextraip flags?

You can directly edit umbrel/app-data/lightning/exports.sh, however it will get overwritten if you update the Lightning app.

This might be something to look into

; Do not include the interface IPs or the system hostname in TLS certificate,
; use first --tlsextradomain as Common Name instead, if set.
; tlsdisableautofill=true

We actually added this to prevent an issue where the SSL cert got rotated every startup due to LND seeing the Docker container hostname (which is random) not the host hostname. And then we manually passed the correct details in via the config. #318

But now I'm thinking if we remove that we could solve the SSL cert rotation issue by just spoofing the container hostname to the host hostname. I think it might grab the correct IP by default. That way we might be able to remove our tlsextradomain/tlsextraip options and users will have full control over them in the config file.
I don't think this is going to help since it'll inherit all of the Docker container hostname/IPs, not the host hostname/IPs.

[lukechilds]

@TrezorHannes this should now be resolved in the latest version of the Lightning app in the Umbrel App Store. You can now set tlsextraip/tlsextradomain in the LND config and it will be respected. Thanks for your suggestions!

[felipehere]

Hi @lukechilds , thank you for addressing it! I was able to successfully set tlsextraip & tlsextradomain in the latest version.

[mbio16]

Hi, @lukechilds. TLS cert has tlsextraip/tlsextradomain. Thanks