Skip to content

feat(room-node): add live diagnostics menu - #21

Merged
steipete merged 2 commits into
mainfrom
feat/room-diagnostics-menu
Aug 8, 2026
Merged

feat(room-node): add live diagnostics menu#21
steipete merged 2 commits into
mainfrom
feat/room-diagnostics-menu

Conversation

@steipete

@steipete steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

What Problem This Solves

When Talk connects but the room is silent, the firmware previously exposed no local evidence showing whether audio reached the microphone, AFE, decoded playback, renderer, or codec path. Operators had to infer health from connection state and logs.

Closes #20

Why This Change Was Made

This adds one shared diagnostics owner to the room runtime rather than board-specific debug screens or log parsing.

  • Long-press the status screen or empty Canvas background to open a modal on lv_layer_top(); the underlying screen and state remain intact.
  • Record microphone/reference levels at codec-read ownership, post-AFE levels at fetch ownership, and playback levels/results at the production render tap.
  • Show freshness, monotonic counters, AFE/capture ownership, Talk phase, Canvas state, Wi-Fi, heap, uptime, display geometry, configured topology, and volume.
  • Add a 500 ms asynchronous 1 kHz local tone through av_render -> render tap -> I2S renderer -> esp_codec_dev -> ES8388.
  • Serialize the entire tone against Talk with one media gate. The UI deliberately reports renderer accepted; software cannot prove analog speaker output.
  • Add local USB REPL controls: diagnostics open|close|tone|status. There is no new remote node command or gateway allowlist surface.
  • Fix local Canvas re-show for retained standalone images and make idle taps visible on non-animated boards.

The modal is not a third screen or a new room UI state. Cached scalar snapshots are updated by their owners under small locks; the LVGL timer performs no I²C, filesystem, JSON, command, or network work.

User Impact

A person standing at the room node can now answer:

  • Is the mic producing samples?
  • Is AFE producing output?
  • Did remote PCM reach the renderer?
  • Did the renderer accept writes or return errors?
  • Is Talk idle, starting, active, or closing?
  • Is ambient WakeNet or Talk VC AFE active?
  • Is the node/operator/Wi-Fi path connected?
  • Does a local production-path speaker tone make sound?

On non-animated boards such as Tab5, the status screen shows Hold for diagnostics so the gesture is discoverable.

Evidence

Automated

  • git diff --check
  • Host room-file validation: PASS
  • Host diagnostics meter tests with -Werror: PASS
  • Waveshare ESP32-S3 room-node build: PASS, 57% app partition free
  • Fresh Tab5 ESP32-P4 build from checked-in defaults: PASS, 51% app partition free
  • Final Codex autoreview: clean; TruffleHog clean

Physical Tab5

  • Fresh full flash verified USB Serial/JTAG as primary bidirectional console; this caught and eliminated stale local build-config drift.
  • Diagnostics modal opened asynchronously from the 4 KiB console task without stack growth or reset after an initial physical repro exposed synchronous LVGL construction as unsafe.
  • Modal remained open at 80% brightness while camera capture succeeded (160x280 JPEG), proving shared screen/privacy coexistence.
  • Local tone completed 25/25/25 requested/enqueued/renderer accepted, with zero renderer errors and RX/SPK meter 86/100.
  • External microphone recording found a dominant approximately 1 kHz acoustic component around 82x above the median 1 kHz noise floor, proving sound emerged into the room without uploading private audio.
  • Sustained room speech moved the perceptual meters (MIC 11/100, AFE 7/100) with fresh timestamps.
  • Tone requested during active Talk returned busy and enqueued zero frames; Talk then stopped cleanly.

UI geometry proof

No room-camera screenshot or audio artifact is uploaded. The physical modal contains:

  1. Diagnostics header and large Close button.
  2. Full-width MIC, AFE, and RX/SPK bars with value/freshness labels.
  3. Wrapped audio ownership/counter text.
  4. Full-width green Play test tone button.
  5. Wrapped runtime/system section beneath in one scrollable page.

The layout was exercised on the connected 1280x720 ST7123 Tab5. Waveshare remains build-proven; its 410x502 touch layout needs the next available physical-device smoke test.

Size

Production C/H: approximately +1,474 net lines. Tests: +28. Docs/build metadata: +27. The growth is the requested capability and its owner-recorded telemetry, not duplicated board policy.

@steipete
steipete marked this pull request as ready for review August 8, 2026 07:03
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. labels Aug 8, 2026
@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed August 8, 2026, 3:15 AM ET / 07:15 UTC.

ClawSweeper review

What this changes

The PR adds a local room-node diagnostics overlay, USB REPL controls, audio-path meters, and a Talk-serialized production-path speaker test.

Merge readiness

⚠️ Ready for maintainer review - 4 items remain

Keep open for maintainer product-direction review: this is a coherent, hardware-proven room-node capability, but it adds a large always-bundled diagnostics surface rather than repairing established behavior.

Priority: P2
Reviewed head: c87c535f31130e4a062a34fda71b8a3b196de983
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Strong physical proof and green target builds support a well-structured patch, pending normal core-feature sponsorship.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (terminal): A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.
Evidence reviewed 4 items Current-main check: The diagnostics implementation remains wholly outside current main: the PR head is not an ancestor of main and its 20 changed files include new diagnostics sources.
Exact-head hardware proof: The contributor’s redacted Tab5 transcript demonstrates modal opening, 25/25/25 tone delivery with zero renderer errors, live MIC/AFE values, camera coexistence, and correct busy rejection during active Talk.
Build coverage: GitHub context reports successful exact-head builds for ESP32-S3 component, ESP Box 3, ESP32 node, Tab5, and Waveshare room-node jobs.
Findings None None.
Security None None.

How this fits together

Room-node firmware connects local microphone/AFE capture, Talk playback, the display UI, and codec hardware. The new overlay reads owner-recorded runtime snapshots and gives a nearby operator local evidence about those paths.

flowchart LR
A[Microphone and codec input] --> B[AFE and capture processing]
C[Talk playback] --> D[Renderer and codec output]
B --> E[Diagnostics snapshots]
D --> E
F[Status screen or USB REPL] --> G[Diagnostics overlay]
E --> G
G --> H[Local operator evidence]
Loading

Decision needed

Question Recommendation
Should the room-node firmware permanently bundle this local diagnostics and speaker-test interface as a core capability? Sponsor the core diagnostics interface: Accept the shared local overlay and REPL controls as supported room-node functionality, with follow-up hardware smoke coverage on Waveshare.

Why: The implementation is coherent and physically demonstrated, but it introduces a substantial always-shipped UI and runtime surface that current main does not define.

Before merge

  • Resolve merge risk (P1) - The patch adds 1,528 production lines across the shared media lifecycle, capture path, Canvas, and UI; a regression there can affect Talk or local room-node availability.
  • Resolve merge risk (P1) - Physical after-fix proof covers Tab5, while the Waveshare touch layout is build-proven but has not received an equivalent device smoke test.
  • Complete next step (P2) - A maintainer must sponsor or narrow the new core diagnostics surface; no mechanical repair is currently indicated.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Code growth production +1,528 / -54; tests +28 The feature spans 20 files and materially expands the shared room-node runtime, so core ownership direction matters before merge.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #20
Summary: This PR is the explicit candidate implementation for the linked room diagnostics request.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Obtain the remaining board smoke test (recommended)
    Exercise the hold gesture, modal close, and speaker-test busy path on Waveshare before merge to reduce cross-board display and input availability risk.
  2. Accept Tab5 as the representative hardware proof
    Merge with the current exact-head Tab5 evidence and successful Waveshare build, then schedule the physical Waveshare check separately.

Technical review

Best possible solution:

If maintainers want on-device diagnostics as a core room-node capability, merge this shared-owner approach after accepting the cross-board availability risk and scheduling a Waveshare device smoke test.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug report. The contributor supplied a high-confidence exact-head physical Tab5 run for the newly added behavior.

Is this the best way to solve the issue?

Unclear pending product direction: the shared snapshot design avoids board-specific debug screens, but maintainers must choose whether this large local diagnostics surface belongs in core firmware.

AGENTS.md: not found in the target repository.

Codex review notes: model internal, reasoning high; reviewed against 1cb1a17eb28c.

Labels

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
  • remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.

Label justifications:

  • P2: This is a bounded room-node operator feature without evidence of an active user-facing outage.
  • merge-risk: 🚨 availability: The change serializes Talk and local playback while modifying shared media, capture, and UI runtime paths.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.
  • proof: sufficient: Contributor real behavior proof is sufficient. A redacted exact-head Tab5 terminal transcript shows the after-fix modal, live meters, production-path tone result, Talk contention behavior, and camera coexistence.

Evidence

What I checked:

Likely related people:

  • steipete: Introduced the shared room runtime on current main and authored the proposed diagnostics extension. (role: shared room-runtime introducer and recent area contributor; confidence: high; commits: 17837cee4852, 827972dcc567; files: components/esp-openclaw-room-node/room_media.c, components/esp-openclaw-room-node/esp_openclaw_room_node.c)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Obtain a maintainer decision on bundling this diagnostics experience in room-node firmware.
  • Run the targeted Waveshare physical smoke test when hardware is available.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-08-08T07:07:12.162Z sha c87c535 :: needs real behavior proof before merge. :: none

@steipete

steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Physical-device proof for reviewed head c87c535f31130e4a062a34fda71b8a3b196de983 on the connected M5Stack Tab5 (ESP32-P4 rev 1.3, ST7123 panel), after a fresh full flash from checked-in defaults.

The transcript below is captured from the primary USB Serial/JTAG console. ANSI cursor traffic, Wi-Fi identity/signal, addresses, authentication details, room audio, and camera imagery are omitted. No firmware lines are synthesized.

openclaw> diagnostics open
diagnostics open queued
I (24828) M5Stack Tab5: Setting LCD backlight: 80%

openclaw> diagnostics tone
I (27794) I2S_RENDER: open channel:2 sample_rate:16000 bits:16
diagnostics tone queued
I (27798) I2S_IF: STD: TX, data_bit: 16, slot_bit: 32, ws_width: 32, slot_mode: STEREO, slot_mask: 0x3
I (27801) I2S_IF: STD: TX, sample_rate_hz: 16000, mclk_multiple: 256, clk_src: 0

openclaw> diagnostics status
diagnostics=open Talk=idle tone=done (25/25/25 requested/queued/accepted)
audio mic=11@77519060us afe=7@77488552us rx=86@26333454us renderer accepted/errors=25/0

I (98664) sc202cs: Detected Camera sensor PID=0xeb52
openclaw> diagnostics status
diagnostics=open Talk=idle tone=done (25/25/25 requested/queued/accepted)
audio mic=0@98219190us afe=0@98231935us rx=86@26333454us renderer accepted/errors=25/0

I (116923) room_media: Talk owns capture; WakeNet disabled for AFE reopen
I (119603) AFE: AFE Version: (1MIC_V251128)
I (119613) AFE: AFE Pipeline: [input] -> |AEC(VOIP_LOW_COST, NLP_ON)| -> [output]

openclaw> diagnostics tone
diagnostics tone busy
openclaw> diagnostics status
diagnostics=open Talk=active tone=busy (25/0/0 requested/queued/accepted)
audio mic=2@119620669us afe=1@119623577us rx=0@119637106us renderer accepted/errors=130/0

The camera invocation during the open modal returned a 160x280 JPEG; the private room frame is intentionally not uploaded. The subsequent diagnostics=open result shows the modal remained active after camera use.

The 500 ms production-path tone was also recorded with a separate room microphone and analyzed locally. The recording remains private; this is the exact spectral-analysis output:

peak_1khz_time=0.700s ratio=81.9x peak_window_rms=1584.7
peak_frequency=1020Hz peak_to_1khz=2.16x

Together this proves the after-fix path on hardware: asynchronous modal construction without reset, live MIC/AFE movement, 25/25/25 tone delivery with zero renderer errors, analog approximately-1 kHz speaker output, camera coexistence, and tone rejection while Talk owns media. All five exact-head CI build jobs are also green.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 8, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 8, 2026
@steipete

steipete commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Maintainer decision: sponsor the shared diagnostics interface as supported room-node functionality.

Local observability is part of the room-node operator contract: when Talk is silent, the person beside the node needs an on-device answer for microphone, AFE, playback, renderer, codec-path, ownership, and runtime state. The shared owner-recorded snapshots and single modal are the canonical implementation; board-specific debug screens or a remote gateway command would be a worse ownership boundary.

I accept the exact-head Tab5 physical proof plus all five green target builds as sufficient for this change. The Waveshare 410x502 physical gesture/layout smoke test remains a worthwhile follow-up when that hardware is available, but is not a blocker for the shared implementation: the existing Waveshare firmware builds successfully on this head, retains its animated status path, and the modal/input code is common.

No code changed after the clean source review, exact-head CI, and sufficient physical proof. Proceeding to merge PR #21; issue #20 should close through the PR linkage.

@steipete
steipete merged commit cebf517 into main Aug 8, 2026
16 of 17 checks passed
@steipete
steipete deleted the feat/room-diagnostics-menu branch August 8, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add on-device room diagnostics and speaker test

1 participant