[v1.10.0] App control over WebRTC, better speaker audio, smoother face tracking

🌐 App control over WebRTC (JSON-RPC 2.0)

The daemon now relays a single JSON-RPC 2.0 control protocol over the WebRTC data channel and /ws/sdk, so any client — the TS SDK, a browser app, or a phone — can drive and observe on-robot apps through one transport. Apps expose their own methods on a small /rpc WebSocket, and the daemon routes by namespace: apps.* lifecycle calls (start/stop/status/install) are handled locally while everything else is relayed to the running app, with the app's notifications fanned out to every connected client. A phone can now steer a local app without losing its remote session, thanks to a new robot-lock mode that accepts a peer as control-only while a local app holds the robot. The TS SDK exposes this through rpcCall(method, params, {timeoutMs}) and onNotification(method, cb).

// Drive an on-robot app over the WebRTC data channel (JSON-RPC 2.0).
// The daemon handles apps.* lifecycle calls locally and relays the rest
// to the running app, fanning its notifications back to every client.
await reachy.rpcCall("apps.start", { name: "my_app" }, { timeoutMs: 5000 });
const status = await reachy.rpcCall("apps.status", {});

reachy.onNotification("apps.myapp.on_event", (params) => console.log(params));

• Expose the apps control API over WebRTC via a JSON-RPC 2.0 relay and SDK helpers by @FabienDanieau in #1266

☁️ Python WebRTC consumer for cloud backends

A new hardware-free ReachyCentralConsumer lets a cloud service (HF Space, Cloud Run, any Python backend) subscribe to a robot's camera/mic and drive it over the central signaling relay using aiortc — without the daemon, GStreamer, or the full ReachyMini stack on the backend. The robot now advertises short-lived Cloudflare TURN relay candidates so NAT-restricted remote consumers can connect, and the TS host SDK gains a connectToHostCredsOnly mode that forwards {hfToken, robotPeerId, signalingUrl} to an app's backend instead of opening a session in the iframe, avoiding the "robot busy" race when a cloud backend owns the WebRTC peer. The consumer recovers from stale/lost producers and failed peer connections, and clears the last decoded frame on teardown.

pip install "reachy_mini[central-consumer]"

from reachy_mini.media.central_consumer import ReachyCentralConsumer

async with ReachyCentralConsumer(robot_name="my-robot", hf_token=...) as consumer:
    frame = await consumer.latest_frame()  # numpy RGB

📚 Documentation: Cloud-Backend Consumer · Media architecture

• Python aiortc central consumer, Cloudflare TURN relay, and credentials-only SDK embed mode by @FabienDanieau in #1182

🔊 Better speaker audio

Two changes make Reachy Mini sound better. Audio firmware 2.1.4 raises the maximum speaker volume by 6 dB and fixes a regression where the microphone stopped emitting audio after a USB reset. On the daemon side, a new 10-band graphic EQ on every speaker branch compensates for the "boxy" resonance of the plastic head shell — cutting the low-mid bass boom and lifting the muffled 1–8 kHz presence range. The default gains were derived from a differential acoustic measurement (speaker response with vs. without the shell), and you can override or disable them in the daemon config; a reusable calibration tool ships under src/reachy_mini/tools/speaker_eq_calibration/.

{ "speaker_eq_gains": [0.0, -13.21, -5.55, -4.28, -4.32, 5.80, 4.65, 4.90, 3.41, 0.0] }

Set in ~/.config/reachy_mini/daemon_config.json (Wireless: /home/pollen/.config/reachy_mini/daemon_config.json). Set all values to 0.0 to bypass the EQ, then restart the daemon to apply.

📚 Documentation: Speaker equalization · Troubleshooting & FAQ

• Bump audio firmware to 2.1.4 for +6 dB max volume and a microphone-output fix by @FabienDanieau in #1242
• Add a daemon-side speaker EQ to correct the head-shell "boxy" resonance by @FabienDanieau in #1267

🎯 Smoother face tracking

The daemon-side face tracker now smooths the normalized face center through an adaptive filter before it reaches look_at_image_pose(), so small target jitter is damped while meaningful movement still gets a fast response. The filter combines EMA smoothing for ordinary motion, a faster alpha for large frame-to-frame jumps, and a radial dead zone for sub-threshold noise, with an explicit reset on target loss or tracking pause. State survives transient misses but resets when the tracker drops its target association, and the backend's duplicate center deadband was removed so target filtering has a single owner. The public start_head_tracking() / get_tracked_face() API is unchanged.

from reachy_mini import ReachyMini

with ReachyMini() as mini:
    mini.start_head_tracking()
    face = mini.get_tracked_face()  # detected, x, y in [-1, 1], roll
    mini.stop_head_tracking()

📚 Documentation: Head Tracking example

• Add an adaptive face tracking filter to the daemon-side tracker by @jrubiosainz in #1268

📡 Live pose stream over the WebRTC data channel

The daemon now pushes the robot's present state to subscribed WebRTC peers at ~30 Hz over a dedicated unreliable/unordered pose data channel, instead of the client polling get_state over the reliable-ordered control channel. A dropped frame is simply superseded by the next one 33 ms later rather than stalling the stream, so live 3D mirror and head-tracking views stop jittering on lossy Wi-Fi. Each pushed frame carries a monotonic seq so the client can drop stale or out-of-order frames, and get_state now delegates to the same single-source build_state_dict (including face_target and per-motor joint positions). The JS SDK exposes refcounted subscribePose() / unsubscribePose() so multiple local consumers share one daemon-side subscription, re-asserted on reconnect. Idle sessions cost nothing — the push timer only runs while at least one peer is subscribed.

• Push the live robot pose to subscribed WebRTC peers at ~30 Hz on a lossy data channel by @pierre-rouanet in #1296

🏷️ Persisted robot rename with live apply

Owners can now rename a robot from the app or BLE setup flow and see it take effect immediately — no daemon restart. A user-set display name is persisted under the config dir, wins over the --robot-name CLI default, and survives reboots. Every write path (WebRTC data channel, REST GET/POST /api/daemon/robot-name, BLE SET_NAME) converges on the same live-apply: the in-memory name and daemon status refresh, the central relay re-advertises on its next heartbeat, and the mDNS LAN record re-registers. The TS SDK exposes getRobotName() / setRobotName().

• Persist a user-set robot name and apply it live across mDNS and the central relay by @pierre-rouanet in #1298

🔄 WebRTC resilience and additive SDK APIs

The TS SDK now rides out ICE blips and network changes on its own. A bounded re-dial loop (4 attempts, ~22 s) emits sessionReconnecting / sessionReconnected before any terminal event, a data-channel silence watchdog (2.5 s nudge, 8 s re-dial) catches dead links without false positives on backgrounded tabs, and autoReconnect (now on by default) can be toggled off for flows like the daemon update so a reboot's sessionStopped isn't absorbed. Every daemon-facing addition is fail-open against older daemons — unknown commands are silently dropped and the SDK resolves null on the shared 4 s _slotRoundtrip timeout, so a new SDK calling an old daemon no longer hangs a gated UI. The public runtime surface is additive; only the type surface grows (ReachyMiniConstructor.version, new ReachyMiniInstance members), so downstream mocks may need a tsc pass.

📚 Documentation: Wake & Sleep Lifecycle Contract · JavaScript SDK

• WebRTC resilience, additive APIs, and internals refactor by @tfrere in #1315

🎬 Recorded-move playback over the data channel

The daemon now exposes play_recorded_move over the WebRTC data channel — the data-channel twin of POST /api/move/play/recorded-move-dataset. It loads the move cache-first from a HF dataset and runs it through Backend.play_move, so the sidecar sound plays on the robot speaker in lockstep with the motion. Fire-and-forget: the ack means "dispatched", not "finished", and an optional initial_goto_duration eases into the move's first frame instead of snapping. A generic preload_dataset primitive warms the cache for non-official datasets right after connecting. The TS SDK exposes playRecordedMove(moveName, { dataset?, initialGotoDuration? }) and preloadDataset(dataset).

• Port play_recorded_move and preload_dataset onto the WebRTC data channel by @tfrere in #1346

🧭 Host shell update gate, resilience UX, and staleness self-check

The host shell now gates the embed on a daemon update flow, surfaces resilience state in the picker/boot views, and runs an SDK staleness self-check that compares the vendored SDK against the installed daemon version (npm source, major-only) with a persisted dismissal. The check ships with a new lifecycle-contract doc that pins who owns waking and sleeping the robot across the app / SDK / host / daemon boundary: apps never wake or sleep the robot — the platform does — and an app's only lifecycle duty is cleaning up its own resources in onLeave.

📚 Documentation: Wake & Sleep Lifecycle Contract

• Host update gate, resilience UX, and picker/boot refresh by @tfrere in #1347
• SDK staleness self-check v2 with a persisted dismissal by @tfrere in #1348

🤖 Robot & Motion

• Stream the ReSpeaker Direction-of-Arrival in the robot state (demand-driven ~10 Hz poller, REST + pose stream) by @tfrere in #1310
• Expose the BMI088 IMU reading over the data channel (get_imu, cached from the 50 Hz control loop) by @tfrere in #1313
• Persist the first wake-up flag over the data channel so the mobile wizard shows once per robot by @tfrere in #1340
• Cut speech-tapper CPU on the CM4 by replacing the sample deque with a NumPy rolling window by @ai-ag2026 in #1259

🧩 Apps & SDK

• Add a delete_hf_token data-channel command and signOut() so an owner can unlink a robot remotely by @pierre-rouanet in #1297
• Add an idle-reset handoff grace and make wake_up a no-op on an already-awake robot by @tfrere in #1311
• Disable WebSocket compression on loopback SDK connections to cut local-daemon CPU by @ai-ag2026 in #1260
• Migrate the host UI from MUI v7 to v9 by @tfrere in #1211

🎥 Audio & Vision

• Tighten the Linux camera detection fallback for deterministic device selection by @andimarafioti in #1101

🔧 Other QoL Improvements

• Run the power-button monitor as a direct script (~1 s boot, 150→25 MB RSS) and self-heal the stale gpio systemd unit by @RemiFabre in #1307

📖 Documentation

• Pull-request template with a tested-on matrix and AI-assistance declaration, and written contribution conventions for issues, PRs, commits, and labels by @FabienDanieau in #1339
• Generate conversation-app profiles in the shared profile.md layout and drop obsolete templates by @alozowski in #1357
• Web Worker clock pattern for backgrounded tabs in the JavaScript SDK guide by @tfrere in #1370 — docs

🐛 Bug and typo fixes

• Resolve robot_name to its daemon via mDNS so a non-default name targets a specific robot by @alozowski in #1222
• Restore face tracking on macOS/Windows/Linux without the V4L2 plugin by falling back to portable GStreamer converters by @alozowski in #1289
• Skip broadcasting state when no clients are connected by @RemiFabre in #1292
• Reset the robot to idle (goto_sleep, motors disabled) when the app slot becomes free on graceful or abnormal teardown by @pierre-rouanet in #1294
• Time out _slotRoundtrip so an unknown command on an older daemon no longer hangs the SDK by @pierre-rouanet in #1295
• Fail the update job on a non-zero pip install exit code so the daemon doesn't restart on the old version by @tfrere in #1312
• Give read_jpeg a 1 s sample timeout so the one-shot encode doesn't return None on a loaded runner by @tfrere in #1328
• Route startup_check through the logger so boot results reach the log file by @RemiFabre in #1337
• Throttle minimal_demo's set_target loop to 100 Hz so the example stops commanding 13000 targets/s by @RemiFabre in #1345
• Select the newest PyPI pre-release by version parsing rather than the trailing lexicographic key, so the beta channel surfaces newer RCs by @FabienDanieau in #1351
• Match the OAuth-redirect spinner color to its busy button by @tfrere in #1354

🏗️ Internal

• Drop the removed custom_container input from the HF doc-build workflow by @FabienDanieau in #1261
• Follow up on the doc-builder CI workflow by @FabienDanieau in #1265
• Add a huggingface_hub-style release CI workflow with PyPI trusted publishing and AI-drafted release notes by @FabienDanieau in #1269
• Make the audio, camera, and media test surface runnable in CI with virtual devices by @FabienDanieau in #1271
• Normalize PEP 440 versions to semver before npm version in the publish workflow by @pierre-rouanet in #1273
• Add WebRTC test coverage for the signalling relay, client, and media server by @FabienDanieau in #1274
• Raise unit-test coverage for the daemon backend and low-coverage routers and utils by @FabienDanieau in #1275
• Fix downstream RC test deps and route release-notes billing to the org by @FabienDanieau in #1283
• Point the doc build back at upstream doc-builder by @FabienDanieau in #1329
• Make RCs visible, keep manual note edits, and ship RCs to npm by @FabienDanieau in #1349
• Cut every RC from main and run the notes tooling from main by @FabienDanieau in #1350