[BUG] signmessage hangs forever on the VLS signer and freezes the node (hsmd queue wedge)

[welliv]

[BUG] signmessage hangs forever on the VLS signer and freezes the node (hsmd queue wedge)

Repo: Blockstream/greenlight (or wherever signer/signerproxy issues live)
Severity: High — a single signmessage call makes the node unresponsive for all signing operations until it restarts.
Affects: All Greenlight nodes (reproduced on the gl-testing in-process python signer AND the production Rust VLS signer + hosted node).

Summary

Calling the node's signmessage RPC never completes. The request is forwarded to the signer via hsmd → gl-signerproxy → scheduler, and the signer (VLS) never returns a response. Because lightningd's hsmd channel is a serial queue, the stuck request blocks every subsequent signing operation — invoice creation, payments, channel ops — which then time out. The node only recovers after a restart (on hosted infra, outside the user's control).

Reproduction

Environment: testnet hosted node, lightningd v26.06gl1, glcli signer run (production Rust VLS signer), connected via cln-grpc.

1. Healthy node — invoices mint fine.
2. Call signmessage (via cln-grpc SignMessage or the hsmd path):
   • gl-testing harness: in-process python signer hangs on signmessage (documented in alby-hub-greenlight's live E2E test).
   • Production Rust VLS signer: same behavior — the request never completes (>45s, both through the hub's REST API and directly).
3. Immediately after: any subsequent signing RPC hangs:

   17:14:02  Signing message            ← signmessage dispatched (never completes)
   17:17:37  signmessage failed: context canceled    ← caller gave up
   17:18:25  invoice failed: context canceled        ← invoice creation now ALSO hangs
   

4. Restarting the local signer does not clear the wedge (the stuck request lives in the node-side hsmd queue, not the signer connection).

Expected behavior

signmessage either:

• completes with a signature (hsmd SIGN_MESSAGE handled by the VLS signer), or
• returns a clean "not supported" error immediately — without leaving the hsmd queue wedged.

Impact

• Any client that calls signmessage (e.g. lightning address / LNURL auth flows, node-identity verification, some wallets) can take a user's node offline for signing until the hosted node happens to restart.
• The wedge is silent: the node process stays up, getinfo still responds, only signing operations hang.

Notes

• The same hsmd-queue wedge was previously observed only on the gl-testing python signer; this report confirms it on the production signer path (glcli signer / VLS).
• Suggested fix areas: (a) VLS signer implements or explicitly rejects SIGN_MESSAGE; (b) signerproxy returns a timeout/error instead of leaving the request pending; (c) lightningd-side hsmd queue isolation so one stuck request cannot block all signing.

Evidence

• Timestamps/log excerpt above (hub log: Signing message → invoice failed: context canceled).
• Reproduced twice: once with a 3-hour-old attached signer, once after killing the signer and letting it re-attach (PID respawn verified) — wedge persisted both times.

[cdecker]

signmessage is not part of the allowed API, and is blocked on purpose by the signer. This is working as expected, but maybe we should remove the method from the API or return an error message.

[welliv]

Hi cdecker,

I am building a Greenlight backend for Alby Hub. Alby Hub is self-hosted wallet software that supports LDK, LND, Phoenixd, CLN and Bark. Users pick a backend and run Lightning operations through it.

I'm testing every signing path now. That is where I hit this issue.

A single signmessage call hangs on the VLS signer. Because hsmd is a serial queue, that stuck request blocks invoices, payments, and channels. The node stays frozen until restart.

I currently block SignMessage at the backend and omit sign_message from NIP-47. A health watchdog detects frozen nodes via Getinfo timeout and pump stall. This protects a user, but the root cause is upstream.

What would help from GL:

1. Fast-fail signmessage with a clean "not supported" error. This unblocks hsmd for every Greenlight client.
2. If VLS implements SIGN_MESSAGE, I can re-enable it.
3. A node-restart API would let me auto-recover instead of just detecting wedges.

I would also like your feedback on whether there is a better integration pattern for a wallet client on Greenlight.

Happy to test anything on my end.