feat(node): marketplace node daemon — config, credentials, control auth, inventory - #358
Merged
Conversation
…th, inventory First slice of increment 2: everything the daemon can do before a tunnel or a server-side endpoint exists, so nothing here is tested against a stub. **Control plane direction (decision 11).** LNVPS dials the node over the tunnel, using HTTP. Commands are request/response with a result to report — start a VM, stop a VM — which is what HTTP already is; a websocket would mean rebuilding correlation ids, in-flight replay and reconnect semantics to arrive back at the same thing. The usual reason to prefer an outbound socket, NAT traversal, does not apply: WireGuard gives direct reachability, and a node without a working tunnel cannot serve guests anyway. Matches `lnvps_fw`, so there is one daemon operational model rather than two. **Control auth is a pinned key, not a token (decision 12).** A shared secret would have to be generated, delivered, stored on the operator's disk, rotated and revoked — five chances to leak something that grants control of every guest on the machine. Instead LNVPS's public key is compiled into the binary and every command carries a NIP-98 event signed by the matching private key, which never leaves LNVPS. Verification binds URL, method, body hash and timestamp, and keeps a bounded replay cache: replaying a captured stop is a second outage. A binary built without `LNVPS_CONTROL_PUBKEY` refuses to serve the control API rather than serving it to everyone. **The tunnel is not by itself a trust boundary (decision 13).** Guests run on this machine and could route to the node's tunnel address, so two independent defences are required: the listener binds only the tunnel interface address, enforced at startup rather than documented, and every request is authenticated. `lnvps_host_util` becomes a lib + bin so the daemon and the `lnvps-host-info` pre-flight check an operator runs cannot disagree about the same machine. Its feature flags turned out to be decorative — `--no-default-features` did not compile, which is exactly the configuration a node needs, since a node has neither libva nor NVML installed. Detection is gated on the features properly now, and "we did not look" reports `Unsupported` rather than an empty list. Output is byte-identical to the pre-refactor binary on this host. All eleven guards mutation-tested: each was individually disabled and at least one test failed in every case.
Control traffic already runs inside WireGuard, so this is not about confidentiality on the wire. It closes a different gap: NIP-98 authenticates requests *to* the node, but nothing authenticated the node's *replies*. Without server authentication, anything able to answer on the tunnel address — a guest on the same machine that grabbed the IP, a route-server misconfiguration — could report that a VM started when it did not, or return another node's state. The node self-signs, LNVPS records the SHA-256 fingerprint at registration, and every later call checks the presented certificate against that pin. No CA is involved: a public CA would add a third party able to issue a certificate for a name we already control out-of-band, which is strictly worse. Same crate and approach as lnvps_fw's control API. Two consequences follow from the pin being registered once, and both are enforced rather than documented: - The identity is persisted. A certificate minted on every restart would stop matching the pin, and the node would silently go unreachable. - A certificate that cannot be parsed is a hard failure, not a reason to mint a new one, and the error says what to do about it. Regenerating quietly would change the fingerprint with nothing in the logs to explain the outage. `lnvps-node fingerprint` prints the value to register, and warns on stderr when it had to generate one, since that means LNVPS's copy is now stale. Verified against `openssl x509 -noout -fingerprint -sha256`, which produces the same digest — a SHA-256 agreeing with itself proves nothing. Six guards mutation-tested: silent regeneration, per-restart minting, key and directory permissions, the freshly-generated flag, and SAN coverage.
…nable Two problems, one mine. **Mine:** I removed `use std::path::Path` on a clippy warning, but it is used by the VA-API probe — code that only compiles with the `vaapi` feature, which needs libva installed. My machine does not have it, so the only configuration I could build locally was the one that excludes those lines, and the lint was correct for that build and wrong for the one CI runs. Now gated the same way as the rest of the VA-API code. **Pre-existing:** the published `lnvps-host-info` image has never been able to start. The binary links libva for VA-API detection, but the runtime stage is `debian:trixie-slim` with no libva installed, so every run died with "libva.so.2: cannot open shared object file". Verified against master before changing anything, so this is not fallout from the lib refactor. The runtime stage now installs libva2 and libva-drm2. That second bug is the interesting one: the build was green throughout, because compiling the binary and being able to run it are different claims and CI only ever checked the first. Both arch builds now load the image and run it, asserting the output really is the JSON host registration consumes rather than just a process that exited 0. Confirmed the assertion fails against master's image and passes against the fixed one.
`lnvps_node` depends on `lnvps_host_util` by path, and that crate sits outside the workspace on purpose: it ships as a standalone per-arch operator tool with its own lockfile and Docker context, and it is that exclusion which makes the path dependency legal in the first place. cargo-chef builds its skeleton from workspace members only, so when cooking the main image the path dependency was simply absent and resolution failed before a single crate compiled — "unable to update /app/src/lnvps_host_util". Copying the directory in before the cook step is enough. It is small and changes rarely, so the cost to layer caching is negligible. Verified by building the `build` target to completion locally rather than inferring it from the resolution step passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First slice of marketplace increment 2. Scope is deliberately everything the daemon can do before a tunnel or a server-side endpoint exists, so nothing here is verified against a stub.
Control plane: LNVPS dials the node, over HTTP
Commands are request/response with a result to report — start a VM, stop a VM — which is what HTTP already is. A persistent websocket would mean rebuilding correlation ids, in-flight replay and reconnect semantics to arrive back at the same thing.
The usual reason to prefer an outbound socket is NAT traversal, and it does not apply here: once WireGuard is up LNVPS can reach the node directly, and a node without a working tunnel cannot serve guests anyway. This also matches
lnvps_fw, so there is one daemon operational model rather than two.Outbound calls remain for the two things that precede or outlive the tunnel: registration and heartbeat.
Auth is a pinned public key, not a bearer token
A shared token has to be generated, delivered to the node, stored on the operator's disk, rotated and revoked — five chances to leak a secret that grants control of every guest on the machine. A public key is not a secret, so none of those steps exist, and forging a command requires LNVPS's private key.
LNVPS_CONTROL_PUBKEYis compiled in; every command carries a NIP-98 event signed by the matching key. Verification binds:payload)A binary built without the key refuses to serve the control API rather than serving it to everyone.
HTTPS with the certificate pinned at registration
NIP-98 authenticates requests to the node. Nothing authenticated the node's replies — so anything able to answer on the tunnel address (a guest on the same machine that grabbed the IP, a route-server misconfiguration) could report that a VM started when it did not, or return another node's state.
The node self-signs, LNVPS records the SHA-256 fingerprint at registration, and every later call checks the presented certificate against that pin. No CA: a public CA would add a third party able to issue for a name we already control out-of-band.
Two consequences follow from the pin being registered once, and both are enforced rather than documented:
lnvps-node fingerprintprints the value to register, warning on stderr when it had to generate one, since that means LNVPS's copy is stale.Verified against an independent implementation, because a SHA-256 agreeing with itself proves nothing:
The tunnel is not by itself a trust boundary
Guests run on this machine, and a guest able to route to the node's tunnel address could otherwise stop its neighbours. Two independent defences, both required: the listener binds only the tunnel interface address — enforced at startup, not documented, because a comment in a config file does not survive being copied between machines — and every request is authenticated regardless.
lnvps_host_utilbecomes lib + binSo the daemon and the
lnvps-host-infopre-flight check an operator runs by hand cannot disagree about the same machine.Its feature flags turned out to be decorative:
vaapiandnvmlexisted inCargo.toml, but the code gated ontarget_osinstead, so--no-default-featuresdid not compile — exactly the configuration a node needs, since a node has neither libva nor NVML installed. Now gated properly, and "we did not look" reportsUnsupportedrather than an empty feature list, which is a different answer from "we looked and found nothing".Output is byte-identical to the pre-refactor binary, verified by building master with only the cfg fix applied and diffing.
The crate stays outside the workspace (its own lockfile and Docker context are unchanged); it is now in
exclude, which is what makes the path dependency legal.Verification
All seventeen guards were mutation-tested — each disabled individually, confirming at least one test fails:
Inventory parsers take
&str/fixture trees rather than reading the host, so they are tested against captured contents rather than whatever the CI machine happens to have. Cross-checked against a real host anyway:MemTotal→ 66742550528 bytes (thekBlabel is KiB; reporting it unconverted would understate the machine 1024x)/sys/blockentries — 8 loop devices, an optical drive, 4 empty card-reader slots at 0 sectors, 2 NVMe — exactly the 2 NVMe are reported. Both filters do real work on that machine.Not in this PR
The listener itself, heartbeat frames,
.debpackaging and GPU inventory. The listener needs a tunnel address to bind and aVmBackendto call; GPU inventory lands with increment 11a's eligibility probe so PCI address, IOMMU group cleanliness and BAR sizes are collected once, against real hardware.Release requirement: the packaging workflow must inject
LNVPS_CONTROL_PUBKEY, or released nodes will refuse control requests. Recorded in the plan as part of decision 12.Increment 3 requirement: registration must carry the TLS fingerprint and store it on
marketplace_node, with a re-registration path for rotation — a node that regenerates its certificate with no way to update the pin becomes permanently unreachable. Recorded as decision 14.