Skip to content

feat(node): marketplace node daemon — config, credentials, control auth, inventory - #358

Merged
v0l merged 4 commits into
masterfrom
feat/node-daemon
Aug 5, 2026
Merged

feat(node): marketplace node daemon — config, credentials, control auth, inventory#358
v0l merged 4 commits into
masterfrom
feat/node-daemon

Conversation

@v0l

@v0l v0l commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

First slice of marketplace increment 2. Scope is deliberately everything the daemon can do before a tunnel or a server-side endpoint exists, so nothing here is verified against a stub.

Control plane: LNVPS dials the node, over HTTP

Commands are request/response with a result to report — start a VM, stop a VM — which is what HTTP already is. A persistent websocket would mean rebuilding correlation ids, in-flight replay and reconnect semantics to arrive back at the same thing.

The usual reason to prefer an outbound socket is NAT traversal, and it does not apply here: once WireGuard is up LNVPS can reach the node directly, and a node without a working tunnel cannot serve guests anyway. This also matches lnvps_fw, so there is one daemon operational model rather than two.

Outbound calls remain for the two things that precede or outlive the tunnel: registration and heartbeat.

Auth is a pinned public key, not a bearer token

A shared token has to be generated, delivered to the node, stored on the operator's disk, rotated and revoked — five chances to leak a secret that grants control of every guest on the machine. A public key is not a secret, so none of those steps exist, and forging a command requires LNVPS's private key.

LNVPS_CONTROL_PUBKEY is compiled in; every command carries a NIP-98 event signed by the matching key. Verification binds:

Bound Why
Signature Nothing else in the event means anything until it is authentic
Pubkey The operator owns the machine and can sign anything — but not as LNVPS
URL + method A signed status poll must not be redirected at a destructive command
Body hash (payload) Otherwise a captured header is reusable with different arguments
Timestamp ±60s A captured request stops working quickly, with room for clock skew on a machine we do not administer
Replay cache Replaying a captured stop is a second outage

A binary built without the key refuses to serve the control API rather than serving it to everyone.

HTTPS with the certificate pinned at registration

NIP-98 authenticates requests to the node. Nothing authenticated the node's replies — so anything able to answer on the tunnel address (a guest on the same machine that grabbed the IP, a route-server misconfiguration) could report that a VM started when it did not, or return another node's state.

The node self-signs, LNVPS records the SHA-256 fingerprint at registration, and every later call checks the presented certificate against that pin. No CA: a public CA would add a third party able to issue for a name we already control out-of-band.

Two consequences follow from the pin being registered once, and both are enforced rather than documented:

  • The identity is persisted. A certificate minted per restart would stop matching the pin and the node would silently go unreachable.
  • A corrupt certificate is a hard failure, not a reason to mint a new one — and the error says what to do. Regenerating quietly would change the fingerprint with nothing in the logs explaining the outage.

lnvps-node fingerprint prints the value to register, warning on stderr when it had to generate one, since that means LNVPS's copy is stale.

Verified against an independent implementation, because a SHA-256 agreeing with itself proves nothing:

lnvps-node:  848d7b290589dde8424f9e2409e0b51089c3b5dd6255def9d4d1bc8a7587da9c
openssl:     848d7b290589dde8424f9e2409e0b51089c3b5dd6255def9d4d1bc8a7587da9c

The tunnel is not by itself a trust boundary

Guests run on this machine, and a guest able to route to the node's tunnel address could otherwise stop its neighbours. Two independent defences, both required: the listener binds only the tunnel interface address — enforced at startup, not documented, because a comment in a config file does not survive being copied between machines — and every request is authenticated regardless.

lnvps_host_util becomes lib + bin

So the daemon and the lnvps-host-info pre-flight check an operator runs by hand cannot disagree about the same machine.

Its feature flags turned out to be decorative: vaapi and nvml existed in Cargo.toml, but the code gated on target_os instead, so --no-default-features did not compile — exactly the configuration a node needs, since a node has neither libva nor NVML installed. Now gated properly, and "we did not look" reports Unsupported rather than an empty feature list, which is a different answer from "we looked and found nothing".

Output is byte-identical to the pre-refactor binary, verified by building master with only the cfg fix applied and diffing.

The crate stays outside the workspace (its own lockfile and Docker context are unchanged); it is now in exclude, which is what makes the path dependency legal.

Verification

All seventeen guards were mutation-tested — each disabled individually, confirming at least one test fails:

drop pinned-pubkey check          caught ✓    drop wildcard-bind refusal          caught ✓
drop payload/body binding         caught ✓    drop tunnel-only bind check         caught ✓
drop replay guard                 caught ✓    drop credential permission check    caught ✓
drop timestamp window             caught ✓    drop signature verification         caught ✓
drop event-kind check             caught ✓    drop url binding                    caught ✓
drop method binding               caught ✓    silently regenerate a corrupt cert  caught ✓
mint a new cert per restart       caught ✓    world-readable private key          caught ✓
world-readable tls directory      caught ✓    claim a reload is newly generated   caught ✓
omit tunnel address from the SAN  caught ✓

Inventory parsers take &str/fixture trees rather than reading the host, so they are tested against captured contents rather than whatever the CI machine happens to have. Cross-checked against a real host anyway:

  • 65178272 kB MemTotal → 66742550528 bytes (the kB label is KiB; reporting it unconverted would understate the machine 1024x)
  • of 15 /sys/block entries — 8 loop devices, an optical drive, 4 empty card-reader slots at 0 sectors, 2 NVMe — exactly the 2 NVMe are reported. Both filters do real work on that machine.

Not in this PR

The listener itself, heartbeat frames, .deb packaging and GPU inventory. The listener needs a tunnel address to bind and a VmBackend to call; GPU inventory lands with increment 11a's eligibility probe so PCI address, IOMMU group cleanliness and BAR sizes are collected once, against real hardware.

Release requirement: the packaging workflow must inject LNVPS_CONTROL_PUBKEY, or released nodes will refuse control requests. Recorded in the plan as part of decision 12.

Increment 3 requirement: registration must carry the TLS fingerprint and store it on marketplace_node, with a re-registration path for rotation — a node that regenerates its certificate with no way to update the pin becomes permanently unreachable. Recorded as decision 14.

v0l added 4 commits August 5, 2026 12:37
…th, inventory

First slice of increment 2: everything the daemon can do before a tunnel or a
server-side endpoint exists, so nothing here is tested against a stub.

**Control plane direction (decision 11).** LNVPS dials the node over the
tunnel, using HTTP. Commands are request/response with a result to report —
start a VM, stop a VM — which is what HTTP already is; a websocket would mean
rebuilding correlation ids, in-flight replay and reconnect semantics to arrive
back at the same thing. The usual reason to prefer an outbound socket, NAT
traversal, does not apply: WireGuard gives direct reachability, and a node
without a working tunnel cannot serve guests anyway. Matches `lnvps_fw`, so
there is one daemon operational model rather than two.

**Control auth is a pinned key, not a token (decision 12).** A shared secret
would have to be generated, delivered, stored on the operator's disk, rotated
and revoked — five chances to leak something that grants control of every guest
on the machine. Instead LNVPS's public key is compiled into the binary and
every command carries a NIP-98 event signed by the matching private key, which
never leaves LNVPS. Verification binds URL, method, body hash and timestamp,
and keeps a bounded replay cache: replaying a captured stop is a second outage.
A binary built without `LNVPS_CONTROL_PUBKEY` refuses to serve the control API
rather than serving it to everyone.

**The tunnel is not by itself a trust boundary (decision 13).** Guests run on
this machine and could route to the node's tunnel address, so two independent
defences are required: the listener binds only the tunnel interface address,
enforced at startup rather than documented, and every request is authenticated.

`lnvps_host_util` becomes a lib + bin so the daemon and the `lnvps-host-info`
pre-flight check an operator runs cannot disagree about the same machine. Its
feature flags turned out to be decorative — `--no-default-features` did not
compile, which is exactly the configuration a node needs, since a node has
neither libva nor NVML installed. Detection is gated on the features properly
now, and "we did not look" reports `Unsupported` rather than an empty list.
Output is byte-identical to the pre-refactor binary on this host.

All eleven guards mutation-tested: each was individually disabled and at least
one test failed in every case.
Control traffic already runs inside WireGuard, so this is not about
confidentiality on the wire. It closes a different gap: NIP-98 authenticates
requests *to* the node, but nothing authenticated the node's *replies*. Without
server authentication, anything able to answer on the tunnel address — a guest
on the same machine that grabbed the IP, a route-server misconfiguration —
could report that a VM started when it did not, or return another node's state.

The node self-signs, LNVPS records the SHA-256 fingerprint at registration, and
every later call checks the presented certificate against that pin. No CA is
involved: a public CA would add a third party able to issue a certificate for a
name we already control out-of-band, which is strictly worse. Same crate and
approach as lnvps_fw's control API.

Two consequences follow from the pin being registered once, and both are
enforced rather than documented:

- The identity is persisted. A certificate minted on every restart would stop
  matching the pin, and the node would silently go unreachable.
- A certificate that cannot be parsed is a hard failure, not a reason to mint a
  new one, and the error says what to do about it. Regenerating quietly would
  change the fingerprint with nothing in the logs to explain the outage.

`lnvps-node fingerprint` prints the value to register, and warns on stderr when
it had to generate one, since that means LNVPS's copy is now stale.

Verified against `openssl x509 -noout -fingerprint -sha256`, which produces the
same digest — a SHA-256 agreeing with itself proves nothing. Six guards
mutation-tested: silent regeneration, per-restart minting, key and directory
permissions, the freshly-generated flag, and SAN coverage.
…nable

Two problems, one mine.

**Mine:** I removed `use std::path::Path` on a clippy warning, but it is used
by the VA-API probe — code that only compiles with the `vaapi` feature, which
needs libva installed. My machine does not have it, so the only configuration I
could build locally was the one that excludes those lines, and the lint was
correct for that build and wrong for the one CI runs. Now gated the same way as
the rest of the VA-API code.

**Pre-existing:** the published `lnvps-host-info` image has never been able to
start. The binary links libva for VA-API detection, but the runtime stage is
`debian:trixie-slim` with no libva installed, so every run died with
"libva.so.2: cannot open shared object file". Verified against master before
changing anything, so this is not fallout from the lib refactor. The runtime
stage now installs libva2 and libva-drm2.

That second bug is the interesting one: the build was green throughout, because
compiling the binary and being able to run it are different claims and CI only
ever checked the first. Both arch builds now load the image and run it,
asserting the output really is the JSON host registration consumes rather than
just a process that exited 0. Confirmed the assertion fails against master's
image and passes against the fixed one.
`lnvps_node` depends on `lnvps_host_util` by path, and that crate sits outside
the workspace on purpose: it ships as a standalone per-arch operator tool with
its own lockfile and Docker context, and it is that exclusion which makes the
path dependency legal in the first place.

cargo-chef builds its skeleton from workspace members only, so when cooking the
main image the path dependency was simply absent and resolution failed before a
single crate compiled — "unable to update /app/src/lnvps_host_util". Copying
the directory in before the cook step is enough. It is small and changes
rarely, so the cost to layer caching is negligible.

Verified by building the `build` target to completion locally rather than
inferring it from the resolution step passing.
@v0l
v0l merged commit d551bb2 into master Aug 5, 2026
6 checks passed
@v0l
v0l deleted the feat/node-daemon branch August 5, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant