cln container is not starting

[Czino]

The cln container was working until it didn't.

After running nigiri start --ln the cln container keeps restarting due to errors

Tried removing dockers and images and restart nigiri which yields same error.

Temporary fix is to reinstall nigiri via curl https://getnigiri.vulpem.com | bash and then start again. I am saying temporary here because sporadically the cln container stops working after a restart.

Logs

2024-03-12 15:44:02 Core-Lightning starting
2024-03-12 15:44:02 Core-Lightning started, RPC available on port 9835
2024-03-12 15:44:02 lightningd --network="${LIGHTNINGD_NETWORK}" "$@"
2024-03-12 15:43:02 lightningd: Binding rpc socket to 'lightning-rpc': Operation not supported
2024-03-12 15:44:02 Couldn't watch /root/.lightning/bitcoin: No such file or directory
2024-03-12 15:44:02 2024-03-12T14:44:02.524Z INFO    lightningd: v23.11
2024-03-12 15:44:02 2024-03-12T14:44:02.697Z DEBUG   plugin-manager: started(12) /usr/local/libexec/c-lightning/plugins/autoclean
2024-03-12 15:44:02 2024-03-12T14:44:02.698Z DEBUG   plugin-manager: started(13) /usr/local/libexec/c-lightning/plugins/chanbackup
2024-03-12 15:44:02 2024-03-12T14:44:02.700Z DEBUG   plugin-manager: started(14) /usr/local/libexec/c-lightning/plugins/bcli
2024-03-12 15:44:02 2024-03-12T14:44:02.701Z DEBUG   plugin-manager: started(15) /usr/local/libexec/c-lightning/plugins/commando
2024-03-12 15:44:02 2024-03-12T14:44:02.702Z DEBUG   plugin-manager: started(16) /usr/local/libexec/c-lightning/plugins/fetchinvoice
2024-03-12 15:44:02 2024-03-12T14:44:02.704Z DEBUG   plugin-manager: started(17) /usr/local/libexec/c-lightning/plugins/funder
2024-03-12 15:44:02 2024-03-12T14:44:02.709Z DEBUG   plugin-manager: started(18) /usr/local/libexec/c-lightning/plugins/topology
2024-03-12 15:44:02 2024-03-12T14:44:02.712Z DEBUG   plugin-manager: started(19) /usr/local/libexec/c-lightning/plugins/keysend
2024-03-12 15:44:02 2024-03-12T14:44:02.714Z DEBUG   plugin-manager: started(20) /usr/local/libexec/c-lightning/plugins/offers
2024-03-12 15:44:02 2024-03-12T14:44:02.715Z DEBUG   plugin-manager: started(21) /usr/local/libexec/c-lightning/plugins/pay
2024-03-12 15:44:02 2024-03-12T14:44:02.717Z DEBUG   plugin-manager: started(22) /usr/local/libexec/c-lightning/plugins/txprepare
2024-03-12 15:44:02 2024-03-12T14:44:02.718Z DEBUG   plugin-manager: started(23) /usr/local/libexec/c-lightning/plugins/cln-renepay
2024-03-12 15:44:02 2024-03-12T14:44:02.720Z DEBUG   plugin-manager: started(24) /usr/local/libexec/c-lightning/plugins/spenderp
2024-03-12 15:44:02 2024-03-12T14:44:02.722Z DEBUG   plugin-manager: started(25) /usr/local/libexec/c-lightning/plugins/sql
2024-03-12 15:44:02 2024-03-12T14:44:02.723Z DEBUG   plugin-manager: started(26) /usr/local/libexec/c-lightning/plugins/cln-grpc
2024-03-12 15:44:02 2024-03-12T14:44:02.724Z DEBUG   plugin-manager: started(27) /usr/local/libexec/c-lightning/plugins/bookkeeper
2024-03-12 15:44:02 2024-03-12T14:44:02.725Z DEBUG   plugin-manager: started(29) /usr/local/libexec/c-lightning/plugins/clnrest/clnrest.py
2024-03-12 15:44:02 2024-03-12T14:44:02.977Z DEBUG   lightningd: io_break: check_plugins_manifests
2024-03-12 15:44:02 2024-03-12T14:44:02.977Z DEBUG   lightningd: io_loop_with_timers: plugins_init
2024-03-12 15:44:02 2024-03-12T14:44:02.979Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_channeld
2024-03-12 15:44:02 2024-03-12T14:44:02.980Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_closingd
2024-03-12 15:44:02 2024-03-12T14:44:02.981Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_connectd
2024-03-12 15:44:02 2024-03-12T14:44:02.981Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_gossipd
2024-03-12 15:44:02 2024-03-12T14:44:02.982Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_hsmd
2024-03-12 15:44:02 2024-03-12T14:44:02.982Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_onchaind
2024-03-12 15:44:02 2024-03-12T14:44:02.983Z DEBUG   lightningd: testing /usr/local/libexec/c-lightning/lightning_openingd
2024-03-12 15:44:02 2024-03-12T14:44:02.983Z DEBUG   hsmd: pid 37, msgfd 49
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_CHECK_PUBKEY
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_ANY_DELAYED_PAYMENT_TO_US
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_ANCHORSPEND
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_HTLC_TX_MINGLE
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_SIGN_SPLICE_TX
2024-03-12 15:44:02 2024-03-12T14:44:02.984Z DEBUG   hsmd: capability +WIRE_HSMD_CHECK_OUTPOINT
2024-03-12 15:44:02 2024-03-12T14:44:02.989Z DEBUG   connectd: pid 38, msgfd 53
2024-03-12 15:44:02 2024-03-12T14:44:02.989Z DEBUG   hsmd: Client: Received message 27 from client
2024-03-12 15:44:02 2024-03-12T14:44:02.989Z DEBUG   hsmd: Client: Received message 27 from client
2024-03-12 15:44:02 2024-03-12T14:44:02.989Z DEBUG   hsmd: new_client: 0
2024-03-12 15:44:02 2024-03-12T14:44:02.994Z DEBUG   connectd: Created listener on 0.0.0.0:9935
2024-03-12 15:44:02 2024-03-12T14:44:02.994Z DEBUG   connectd: REPLY WIRE_CONNECTD_INIT_REPLY with 0 fds
2024-03-12 15:44:02 2024-03-12T14:44:02.994Z DEBUG   lightningd: io_break: connect_init_done
2024-03-12 15:44:02 2024-03-12T14:44:02.994Z DEBUG   lightningd: io_loop: connectd_init
2024-03-12 15:44:03 2024-03-12T14:44:03.003Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2024-03-12 15:44:03 2024-03-12T14:44:03.003Z DEBUG   lightningd: io_break: plugin_config_cb
2024-03-12 15:44:03 2024-03-12T14:44:03.003Z DEBUG   lightningd: io_loop_with_timers: config_plugin
2024-03-12 15:44:03 2024-03-12T14:44:03.003Z DEBUG   lightningd: All Bitcoin plugin commands registered
2024-03-12 15:44:03 2024-03-12T14:44:03.009Z DEBUG   lightningd: Adding block 88: 22d982b3c19b4d998d2381d2d6a4a07b5706518702bd82801b4911f9cc34f6fb
2024-03-12 15:44:03 2024-03-12T14:44:03.011Z DEBUG   plugin-bcli: Feerate min_acceptable raised from 500 perkb to floor of 1000
2024-03-12 15:44:03 2024-03-12T14:44:03.011Z DEBUG   lightningd: io_break: maybe_completed_init
2024-03-12 15:44:03 2024-03-12T14:44:03.011Z DEBUG   lightningd: io_loop_with_timers: setup_topology
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loaded 0 HTLC signatures from DB
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loaded 0 HTLC signatures from DB
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loaded 0 HTLC signatures from DB
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loaded 3 channels from DB
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loading in HTLCs for channel 1
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 incoming HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loading in HTLCs for channel 3
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 incoming HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Loading in HTLCs for channel 4
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 incoming HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 outgoing HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 outgoing HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.012Z DEBUG   wallet: Restored 0 outgoing HTLCS
2024-03-12 15:44:03 2024-03-12T14:44:03.013Z DEBUG   gossipd: pid 48, msgfd 52
2024-03-12 15:44:03 2024-03-12T14:44:03.013Z DEBUG   hsmd: new_client: 0
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: gossip_store_compact_offline: 0 deleted, 14 copied
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 106x1x0/0 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 106x1x0/1 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 108x1x0/0 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 108x1x0/1 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 110x1x0/0 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: Received channel_update for channel 110x1x0/1 now ACTIVE
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: total store load time: 0 msec
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: gossip_store: Read 3/6/2/0 cannounce/cupdate/nannounce/cdelete from store (0 deleted) in 2867 bytes
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: seeker: state = STARTING_UP New seeker
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   gossipd: REPLY WIRE_GOSSIPD_INIT_REPLY with 0 fds
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   lightningd: io_break: gossipd_init_done
2024-03-12 15:44:03 2024-03-12T14:44:03.016Z DEBUG   lightningd: io_loop: gossip_init
2024-03-12 15:44:03 lightningd: Binding rpc socket to 'lightning-rpc': Operation not supported

Details

[tiero]

Unfortunately, we do not maintain the CLN docker image, would be best to page here someone from Core LN team.
It has been very painful to make it work in a docker-compose scenario I recall, I had to find a comment in a buried issue on Blockstream repo, as no-one I think ever tested the docker setup.

[altafan]

Unfortunately, we do not maintain the CLN docker image

We use the latest image though. Maybe this can be solved by locking and using a specific version of cln instead?

[tiero]

Can you check again? @Czino