Bitcoin network set in one place

[optout21]

More centralized control of the bitcoin network used. Currently it's regtest, but can be switched easily to testnet4, by changing the config at the top of devenv.nix.
Fixes #58

I'm not sure if signet makes sense, as there is no POW-based mining in signet.
As for mainnet, it's a bit further down the road :)

[vnprc]

This runs as is on regtest, which is a great start, but I noticed some issues when attempting to switch it back to testnet.

Stretch goal: it would be very nice if just could detect the network via an environment variable and fail fast with an error message like error: generate-blocks is only valid in regtest. current network: testnet4.

I restated this issue more clearly below, with precise instructions on how to implement it.

[vnprc]

Ok, given that some just recipes need to know the bitcoin network I think the best way to approach this change is to set the bitcoinNetwork variable as you have already done (but change it to camelCase).

Then validate it like so:

  case "$BITCOIND_NETWORK" in
    regtest|testnet4)
      echo "BITCOIND_NETWORK=$BITCOIND_NETWORK"
      ;;
    *)
      echo "❌ BITCOIND_NETWORK must be one of: regtest, testnet4" >&2
      exit 1
      ;;
  esac

This prevents users from entering invalid networks like typos (reftest) or unsupported options (signet, mainnet (for now)).

Then set it as an env var like this:

env.BITCOIND_NETWORK = bitcoin_network;

Access it within devenv.nix like this: ${config.env.BITCOIND_NETWORK}

Use it to fail fast in the generate-blocks just recipe like this:

  if [ "$BITCOIND_NETWORK" != "regtest" ]; then
    echo "This recipe only works on regtest" >&2
    exit 1
  fi

[vnprc]

Rebase from master. I just added the var and env var to devenv.nix while refactoring regtest-setup.sh. So you don't have to do that part.

[optout21]

I've done suggested changes (camelcase naming), and also:

• set the port from the network (18443/regtest, 48332/testnet4)
• abort with error if network is not supported or just invalid

Other potential issues:

• roles/jd-server/config-examples/jds-config-local-example.toml contains hardcoded port, not addressed in this PR
• network present in job_creator.rs ("const NETWORK: Network = Network::Regtest;") -- should this take it from the env var?
• file roles/jd-client/config-examples/jdc-config-hosted-example.toml contains some Template Provider IP addresses, which may be of a specific network type (Hosted testnet TP)
• same for some other files (roles/pool/config-examples/pool-config-hosted-tp-example.toml, test/config/interop-jd-translator/jdc-config.toml, test/config/interop-jd-translator/pool-config.toml)

[vnprc]

It works with regtest but when I switch to testnet4 the cln and jd-server processes both die trying to connect to the bitcoind RPC interface.

I was able to get jd-server running by changing the port in roles/jd-server/config-examples/jds-config-local-example.toml. In order to dynamically set this value we need to pass it via command line. If that is not an option it may require a code change in the rust module to add this cli parameter.

CLN is failing because testnet4 is not the right setting. CLN expects testnet and converts it to testnet4 on the backend. Once I hard coded this fix it failed again on the bcli plugin with is also misonfigured now. I'll leave it to you to continue troubleshooting.

Please test that both regtest and testnet4 work before requesting another review. I'm happy to set up some pair coding time if you'd like to go that route.

[vnprc]

Sorry for all the back and forth. This task is deceptively difficult. That's just how it works out sometimes. ¯\_(ツ)_/¯

[optout21]

CLN is failing because testnet4 is not the right setting.

I haven't experienced this, I have tested several times with testnet4, and I have not experienced a problem with CLN. In fact I cannot reproduce the error you described. I'm not sure how to proceed with this.

I was able to get jd-server running by changing the port in roles/jd-server/config-examples/jds-config-local-example.toml. In order to dynamically set this value we need to pass it via command line.

I've mentioned in the previous comment that this is not yet done, in fact you mentioned "This file should be copied to /config and the processes definition updated, but that is outside of the scope of this PR.". Copying the file as setup time, and changing the port in it (via sed) may be another solution.

In the previous comment I mentioned concerns about the Template Providers, could you also comment on those?
#61 (comment)

[optout21]

Port in jd-server config is taken care of, config file is copied and then changed (using sed).

The CLN failure i cannot reproduce

[optout21]

Oh, you are probably running CLN v24, which does not support testnet4 yet, v25 is needed. That's why in
PR #51 devenv.lock version was bumped. Make sure CLN version is v25.02.1.

[vnprc]

Ok I have tested this PR with both networks. It works! It's rough around the edges but in the name of progress I declare this PR approved. We can roll forward and clean up the rough edges.

My issues with cln were caused by an apparent bug where it calls out to bitcoin-cli and sucks in the config file located at ~/.bitcoin/bitcoin.conf, which on my machine specified regtest when the nix daemon was running testnet4.

CLN should not behave this way because conf and dir are specified on the command line, but here we are. WYGD. I "fixed" the issue in this commit which just throws up a warning about this conflict if the config file is detected when starting up the devenv shell.

[vnprc]

To address you previous questions:

• roles/jd-server/config-examples/jds-config-local-example.toml contains hardcoded port, not addressed in this PR

addressed in one of your subsequent commits. nice!

• network present in job_creator.rs ("const NETWORK: Network = Network::Regtest;") -- should this take it from the env var?

Yeah I saw that too. It's kind of weird but this is in test code so I don't think it's an issue.

• file roles/jd-client/config-examples/jdc-config-hosted-example.toml contains some Template Provider IP addresses, which may be of a specific network type (Hosted testnet TP)
• same for some other files (roles/pool/config-examples/pool-config-hosted-tp-example.toml, test/config/interop-jd-translator/jdc-config.toml, test/config/interop-jd-translator/pool-config.toml)

These are all test and example configs. Repurposing the example configs was a quick and dirty way to get started in this repo but we should be migrating away from these to our own configs in the top level config dir.