title	Reachy Mini Conversation App
emoji	🎤
colorFrom	red
colorTo	blue
sdk	static
pinned	false
short_description	Talk with Reachy Mini!
suggested_storage	large
tags	reachy_mini reachy_mini_python_app

Reachy Mini conversation app

Conversational app for the Reachy Mini robot combining realtime voice, vision, personality-aware tools, and choreographed motion.

Table of contents

• Overview
• Architecture
• Installation
• Configuration
• Running the app
• LLM tools
• Creating and adding tools
• Advanced features
• Contributing
• License

Overview

• Low-latency audio conversation through the Hugging Face realtime backend, using the built-in server or a local endpoint.
• Vision is handled by the realtime backend when the camera tool is used.
• Layered motion system queues primary moves (dances, emotions, goto poses, breathing) while blending speech-reactive wobble.
• Async tools integrate motion, camera capture, and MCP Tool Spaces. The optional web UI (--ui) manages conversations, personalities, tools, and settings.

Architecture

The app connects the user, AI services, and robot hardware:

Installation

Important

Install Reachy Mini's SDK before using this app.
Windows support is currently experimental and has not been extensively tested. Use with caution.

Using uv (recommended)

Set up with uv:

# macOS (Homebrew)
uv venv --python /opt/homebrew/bin/python3.12 .venv

# Linux / Windows (Python in PATH)
uv venv --python python3.12 .venv

source .venv/bin/activate
uv sync

Include dev dependencies:

uv sync --group dev

Note

Run uv sync --frozen to install the exact dependency set from uv.lock without re-resolving versions.

Using pip

python -m venv .venv
source .venv/bin/activate
pip install -e .

Install dev dependencies:

pip install -e .[dev]                   # Development tools

Configuration

The default setup uses the Hugging Face backend and does not require an API key.

Copy .env.example to .env when you want to point Hugging Face at your own local endpoint.

Variable	Description
REALTIME_TRANSCRIPTION_LANGUAGE	Optional input transcription language for the realtime backend. Defaults to en; set to a backend-supported code such as zh for Chinese.
HF_REALTIME_CONNECTION_MODE	Hugging Face connection selector: deployed uses the built-in Hugging Face server; local uses HF_REALTIME_WS_URL. Defaults to deployed.
HF_REALTIME_WS_URL	Direct websocket endpoint for your own Hugging Face backend. Accepts either a base URL like ws://127.0.0.1:8765/v1 or the full websocket URL ws://127.0.0.1:8765/v1/realtime. Used when HF_REALTIME_CONNECTION_MODE=local.
HF_TOKEN	Optional token for Hugging Face access. Local endpoints receive only this explicitly configured token.
REACHY_MINI_APP_TIMEOUT_MINUTES	Minutes of inactivity before Reachy goes to sleep and the app stops. Defaults to 1440 (one day); set to 0 to disable.

Hugging Face Connection Modes

Use the built-in Hugging Face server through the app-managed Space proxy. This is the default for a new install; set it explicitly only when you want to switch back from a saved local endpoint:

HF_REALTIME_CONNECTION_MODE=deployed

Deployed session allocation falls back to cached hf auth login credentials and reports the daemon-provided hardware ID when available. Cached credentials and the hardware ID are not sent to local endpoints.

Run your own realtime voice backend using speech-to-speech on the same machine as the conversation app:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://127.0.0.1:8765/v1/realtime

Run your own Hugging Face backend on your laptop and connect to it from Reachy Mini Wireless over the same Wi-Fi network:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://<your-laptop-lan-ip>:8765/v1/realtime

For that LAN setup, make sure the backend listens on an address reachable from the robot, not only on 127.0.0.1.

If the backend stays bound to loopback on your laptop, you can forward it into the robot over SSH instead:

ssh -N -R 8765:127.0.0.1:8765 <robot-user>@<robot-host>

Then set this on the robot:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://127.0.0.1:8765/v1/realtime

In the web UI's Settings view, the Connection section lets you choose either the built-in server or a local host:port target. The UI writes HF_REALTIME_CONNECTION_MODE for you, and the local path writes HF_REALTIME_WS_URL with a default of localhost:8765.

Running the app

Activate your virtual environment, then launch:

reachy-mini-conversation-app

Tip

Make sure the Reachy Mini daemon is running before launching the app. If you see a TimeoutError, it means the daemon isn't started. See Reachy Mini's SDK for setup instructions.

The app runs in console mode. Add --ui to serve the web interface at http://127.0.0.1:7860/.

CLI options

Option	Default	Description
--no-camera	False	Run without camera capture.
--ui	False	Serve the web UI at http://127.0.0.1:7860/, in addition to console mode.
--robot-name	None	Optional. Connect to a specific robot by name when running multiple daemons on the same subnet. See Multiple robots on the same subnet.
--debug	False	Enable verbose logging for troubleshooting.

Examples

# Audio-only conversation (no camera)
reachy-mini-conversation-app --no-camera

# Launch with the minimal web UI for personality/mic/settings control
reachy-mini-conversation-app --ui

LLM tools exposed to the assistant

The default profile exposes these tools. Use Tools → Tool access to customize any profile. Every bundled profile enables head_tracking by default; users can still disable it per personality.

Tool	Action	Dependencies
dance	Queue a dance from reachy_mini_dances_library.	Core install only.
stop_dance	Clear queued dances.	Core install only.
play_emotion	Play a recorded emotion clip via Hugging Face datasets.	Core install only. Uses the default open emotions dataset: pollen-robotics/reachy-mini-emotions-library.
stop_emotion	Clear queued emotions.	Core install only.
camera	Capture the latest camera frame and analyze it with the selected realtime backend.	Core install only. Requires the camera (disable with --no-camera).
idle_do_nothing	Explicitly remain idle during an idle turn. Not intended for normal conversation turns.	Core install only.
move_head	Queue a head pose change (left/right/up/down/front).	Core install only.
head_tracking	Follow the user's face with the head, or stop following.	Core install only. Requires a daemon with the vision extra and a camera.
go_to_sleep	Run Reachy's sleep movement and stop the current app after an explicit user request.	Core install only.
sweep_look	Sweep Reachy's head left, right, and back to center.	Shared tool, enabled by default in the default profile.
remember	Save one short, stable fact about the user for future sessions.	Core install only. Stored in the app instance data directory.
forget	Remove a saved memory fact by matching a short query.	Core install only.
pollen_robotics_reachy_mini_search_tool__search_web	Search the web and return a short list of results.	Preinstalled MCP Space: pollen-robotics/reachy-mini-search-tool.
pollen_robotics_reachy_mini_weather_tool__get_weather	Report today's weather for a place: current conditions, high and low temperature, and rain chance.	Preinstalled MCP Space: pollen-robotics/reachy-mini-weather-tool.
pollen_robotics_reachy_mini_time_tool__get_time	Report the current time for a timezone or the user's local time, or the difference between two timezones.	Preinstalled MCP Space: pollen-robotics/reachy-mini-time-tool.

Note

remember/forget facts are stored in memory.v1.json inside the app's instance data directory (~/.local/share/reachy_mini_conversation_app/ by default, or the instance path used by the desktop launcher). forget only removes facts matched by query. To reset all remembered facts, delete this file.

Creating and adding tools

Tools can run locally as Python code or remotely in an MCP-compatible Hugging Face Space. Keep robot, camera, and local-data operations in local tools. A Space is a better fit for shareable, stateless services such as search and external API lookups.

Local tools

Create one Python module per tool, with the file name matching the tool's unique name. See idle_do_nothing.py for a minimal implementation.

Each tool subclasses Tool and defines name, a model-facing description, an object-shaped JSON Schema in parameters_schema, and an async __call__ method. Use ToolDependencies for runtime services, and set needs_response = False for actions that should not trigger a spoken follow-up. Catch expected operational failures, log them with the module logger, and return {"error": "..."} so the conversation can continue.

Restart the app after adding the module. Use Tools → Tool access to enable it for a personality, or add its name to that profile's default_tools in profile.md. See External profiles and tools for external directories and autoload behavior.

Hugging Face Space tools

To publish a remote tool, create a Gradio Space, expose its API as MCP with mcp_server=True, and give each function clear type hints and docstrings. Verify that https://<space-subdomain>.hf.space/gradio_api/mcp/schema lists the expected tools before installing the Space.

Use the maintained weather, time, and search Spaces as examples. See Gradio's MCP server guide for additional publishing guidance and Installing Hugging Face Space tools for this app's installation steps.

Advanced features

Built-in motion content is published as open Hugging Face datasets:

• Emotions: pollen-robotics/reachy-mini-emotions-library
• Dances: pollen-robotics/reachy-mini-dances-library

Custom profiles

Create custom profiles with dedicated instructions and per-profile tool access.

Select and save a startup profile in the UI. The choice is stored in startup_settings.json. Before one is saved, REACHY_MINI_CUSTOM_PROFILE=<name> can select profiles/<name>/; otherwise the app uses default.

Every profile directory contains one strict schema-version-1 profile.md. TOML metadata is enclosed by +++; the remaining Markdown body is the realtime assistant prompt:

+++
schema_version = 1
voice = "Aiden"
greeting = "Greet me warmly in one sentence, in character, and vary the wording each time."
hidden = false
default_tools = [
  "dance",
  "camera",
  "sweep_look",
]
+++

## Identity

You are a concise, friendly robot guide.

schema_version, default_tools, and a non-empty Markdown body are required. voice, greeting, and hidden are optional. Set hidden = true to omit a profile from the UI. An empty default_tools list is valid and inherits nothing.

default_tools is the authored baseline. Tools → Tool access stores overrides in instance-local profile_toolsets.json without changing bundled profiles. Restoring defaults removes the override. Active-profile changes reconnect the conversation; other changes apply when selected.

Profile directories are data-only. Python tool implementations belong in src/reachy_mini_conversation_app/tools/, or in REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY for external tools. Each enabled tool ID must resolve to a shared tool, an external tool, or a tool from an installed Hugging Face Space.

See Creating and adding tools for the local tool interface and a maintained example.

To manage personalities in the UI:

With --ui, Home lists the available profiles and the built-in default:

• Tap a card to apply that personality and start talking.
• Tap "Manage tools" on a saved personality to open its tool access directly.
• Tap "Custom" to create a personality with a name, instructions, and optional greeting. It inherits the default tools, which can be changed under "Manage tools". Managed instances store it at user_personalities/<name>/profile.md; standalone runs use external_content/user_personalities/<name>/profile.md.

Switching a personality reloads its prompt and effective tools through a quick backend reconnect. Editing profile.md directly requires re-selecting the profile or restarting the app.

Locked profile mode

To create a locked variant of the app that cannot switch profiles, edit src/reachy_mini_conversation_app/config.py and set the LOCKED_PROFILE constant to the desired profile name:

LOCKED_PROFILE: str | None = "mars_rover"  # Lock to this profile

When set, the app ignores saved startup settings, REACHY_MINI_CUSTOM_PROFILE, and UI selection. The UI marks the profile as locked and disables editing.

External profiles and tools

You can extend the app with profiles/tools stored outside the repository defaults.

• Core profiles are under profiles/.
• Core tools are under src/reachy_mini_conversation_app/tools/.

Recommended layout:

external_content/
├── external_profiles/
│   └── my_profile/
│       └── profile.md
├── external_tools/
│   └── my_custom_tool.py
├── user_personalities/
│   └── my_custom_profile/
│       └── profile.md
├── installed_tool_spaces.json
└── profile_toolsets.json

Environment variables:

Set these values in your .env when you want env-driven external profile/tool selection:

# Optional fallback/manual profile selector:
REACHY_MINI_CUSTOM_PROFILE=my_profile
REACHY_MINI_EXTERNAL_PROFILES_DIRECTORY=./external_content/external_profiles
REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY=./external_content/external_tools
# Optional convenience mode:
# AUTOLOAD_EXTERNAL_TOOLS=1

Loading rules:

• Profiles: each directory requires a schema-version-1 profile.md with explicit default_tools; there is no cross-profile fallback.
• Default mode: enabled IDs must resolve to a shared, external, or installed Tool Space tool.
• Autoload: AUTOLOAD_EXTERNAL_TOOLS=1 adds every valid *.py module from REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY.
• Web UI: Tools → Tool access enables external modules per profile; it does not upload or edit Python.
• Separation: profile directories contain data only; external Python belongs in REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY.
• Tool names: every loaded class needs a unique Tool.name; duplicates fail fast.

Installing Hugging Face Space tools

You can install MCP-compatible Hugging Face Spaces as remote tool sources for this app. Private Spaces work too, as long as HF_TOKEN is set (or you have run hf auth login) for an account that can access them. To publish a new Space, follow Creating and adding tools.

Tools → Tool Spaces installs or refreshes a global source. Its tools then appear under Tools → Tool access for per-profile selection. Removing a Space removes its tools from every profile. Active-profile changes reconnect the conversation; other changes apply when selected.

The app accepts Hugging Face Spaces exposing the standard /gradio_api/mcp/ endpoint, not arbitrary MCP URLs. Installation discovers the Space's tools and assigns namespaced local IDs, so do not guess or hard-code those IDs beforehand.

# install + enable in active profile
reachy-mini-conversation-app tool-spaces add <owner/space-name>

# enable in a specific profile
reachy-mini-conversation-app tool-spaces add <owner/space-name> --profile NAME

# install without enabling
reachy-mini-conversation-app tool-spaces add <owner/space-name> --install-only

# list installed spaces
reachy-mini-conversation-app tool-spaces list

# remove an installed space
reachy-mini-conversation-app tool-spaces remove owner/space-name

Bundled Pollen Spaces use static specs and are enabled by the default profile. Custom Spaces are validated through the Hugging Face Hub; HF tokens are sent only to private Spaces. Tool metadata is cached in:

• installed_tool_spaces.json in the managed app instance directory
• external_content/installed_tool_spaces.json in terminal mode

Startup and profile switching read this cache without discovery or MCP probing. Network access occurs only during install, refresh, or remote tool calls. Per-profile access is stored in profile_toolsets.json beside the manifest, or under external_content/ in terminal mode.

Recommended tags for discoverability on Hugging Face:

• reachy-mini-tool
• mcp

Tags are advisory; installation still requires successful MCP validation.

[!NOTE] Preinstalled Pollen Spaces can be removed like any other (tool-spaces remove pollen-robotics/reachy-mini-weather-tool). To restore access, reinstall the Space and restore or update the relevant profile under "Tool access".

Multiple robots on the same subnet

If you run multiple Reachy Mini daemons on the same network, use:

reachy-mini-conversation-app --robot-name <name>

<name> must match the daemon's --robot-name value so the app connects to the correct robot.

Contributing

See CONTRIBUTING.md for the development workflow and AGENTS.md for coding-agent standards.

License

Apache 2.0