FUG-93: per-reservation BLE HCI (btmon) capture for reservation agents - #70
Open
issuefleet[bot] wants to merge 2 commits into
Open
FUG-93: per-reservation BLE HCI (btmon) capture for reservation agents#70issuefleet[bot] wants to merge 2 commits into
issuefleet[bot] wants to merge 2 commits into
Conversation
An agent debugging BLE inside a reservation had no way to see link-level
truth — only that BleakClient.connect() timed out, not whether CONNECT_IND
went unanswered, the peripheral answered and the link dropped, or the central
never issued the request. This adds host-side btmon capture an agent can drive
and read back as a btsnoop.
btmon needs an AF_BLUETOOTH HCI monitor socket + CAP_NET_RAW/ADMIN the
unprivileged reservation container deliberately lacks, so capture runs
host-side (the daemon is already root): `btmon -w` per reservation into the
reservation's state dir, bind-mounted read-only into the container at
/run/hitl/capture. start/stop go through the daemon API; the btsnoop is read
back over the existing SSH path. No capability changes anywhere.
- Runner: StartCapture/StopCapture/CaptureStatus keyed on reservation id, a
watchdog bounding each capture by a size cap and a max-duration auto-stop
(can't fill the rig disk), SIGTERM on stop so the btsnoop flushes cleanly,
and teardown wired into Stop/Cleanup so a capture dies with its reservation.
- Queue/API/daemon: capture gated on an active reservation; three routes
(POST .../btmon/{start,stop}, GET .../btmon) + a --btmon* flag set.
- CLI: `hitl btmon start|stop|status|fetch|capture` (the last is the windowed
start→run→stop→fetch form). fetch/capture annotate the trace with the DUT
BLE MAC and print the Wireshark filter.
- nix: bluez on the daemon (btmon path) + rig; container note for `btmon -r`.
Shared-adapter caveat, documented not hidden: the rig has ONE Bluetooth
controller shared by every DUT (DESIGN.md open item), so a trace contains
other DUTs' BLE for its window. It's annotated by the DUT BLE MAC, not
isolated; docs point at the per-DUT-dongle follow-up as the real fix.
Verified offline: go build + unit tests for runner/queue/daemon via bazel
(hermetic Go SDK), nix files parse, all default targets build. The on-hardware
run (opens in btmon -r/Wireshark, teardown on release) remains for the rig.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
CI `prek` (prettier) flagged `*every*`; prettier normalizes emphasis to underscores (`_every_`), matching the rest of the file. Reproduced offline with prettier + markdownlint + nixpkgs-fmt + buildifier over the changed files; all clean now. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
An agent debugging BLE inside a reservation had no way to see link-level truth: it could observe that
BleakClient.connect()timed out, but not why — whetherCONNECT_INDwent unanswered, the peripheral answered and the link dropped, or the central never issued the request. Those have different fixes and the harness logs can't tell them apart. This is the blocker on settling the FUG-61 provisioning-flake root cause.This adds host-side
btmonHCI capture an agent holding a reservation can drive and read back as a btsnoop, with no host access and no rig redeploy — approach (1) from the ticket, refined to per-reservation scoping.Design
btmonneeds anAF_BLUETOOTHHCI monitor socket (HCI_CHANNEL_MONITOR) +CAP_NET_RAW/CAP_NET_ADMIN, which the unprivileged reservation container deliberately lacks (containers are handed to agents). So capture runs host-side — the daemon is already root — asbtmon -wper reservation into that reservation's state dir, bind-mounted read-only into the container at/run/hitl/capture. start/stop go through the daemon API (POST /reservation/{id}/btmon/{start,stop},GET …/btmon); the btsnoop is read back over the existing SSH path. No capability changes anywhere.CLI:
hitl btmon start | stop | status | fetch <out.btsnoop> [--mac] | capture [--mac] --out F -- <cmd>(the last is the windowed start→run→stop→fetch convenience).Acceptance criteria
hitl btmon start --id RES…fetch. ✓btmon -rand Wireshark — native btsnoop; stop sends SIGTERM (not SIGKILL) so the file flushes cleanly. ✓fetch --macprints thebluetooth.addr == <mac>Wireshark filter; the shared-adapter caveat is stated plainly. ✓start). ✓runner.Stop. ✓pi/hitl/README.md,pi/hitl/DESIGN.md(with the caveat and the updated per-DUT-BLE-radio open item), and a line inDEVELOPERS.md. ✓Shared-adapter caveat (documented, not hidden)
The rig has one Bluetooth controller shared by every DUT (DESIGN.md already lists "per-DUT BLE radio" as an open item). A trace therefore contains other DUTs' BLE traffic for its window. This is stated plainly; the trace is annotated by the DUT BLE MAC, not isolated. The real fix — a per-DUT USB BT dongle passed through FUG-73-style — is recorded as the eventual direction, not done here.
Verification
bazel test //pi/hitl/...— runner/queue/daemon Go units (incl. new capture tests: disabled-guard, status shape, start/stop lifecycle with a stand-in btmon, byte formatting) all pass. gofmt clean; nix files parse; all default bazel targets build (hermetic Go SDK).btmon -r/Wireshark and is torn down on release, batched into one reservation. This worktree has no network creds to reach the rig; needs a relayed run.Files
pi/hitl/internal/runner/{capture.go,podman.go,runner.go},internal/queue/queue.go,internal/api/api.go,cmd/hitl-managerd/main.go,cmd/hitl/main.go,nix/{hitl-app,container}.nix, plusREADME.md/DESIGN.md/DEVELOPERS.mdand Go unit tests.Closes-Linear: FUG-93 (https://linear.app/fughilli/issue/FUG-93/hitl-expose-btmon-hci-capture-to-agents-inside-the-reservation)