Ming Voice

A fully local, real-time voice assistant for AMD Radeon GPUs (NVIDIA experimental), on Linux or Windows.
Talk to it and it answers out loud, usually starting about a quarter of a second after you stop talking.

Ming Voice turns inclusionAI/Ming-3.0-audio-tiny, a speech-to-speech model, into an assistant you can leave running at home. It hears you, decides when you've finished, thinks, and speaks back, all on your own GPU. It checks the weather, searches the web, sets timers, keeps your shopping list, remembers what you tell it, and controls Home Assistant and Jellyfin.

No audio leaves your machine. With tools on, only search terms and place names go out (to DuckDuckGo or Brave Search, and Open-Meteo); turn tools off in Settings to stay fully offline.

Conversation that feels natural

• ~0.23 s from the end of your sentence to the first sound of the answer
• Smart turn detection (Smart Turn v3) hears from your words and intonation whether you've finished, so it answers sooner when you're done and waits when you're mid-thought
• Interrupt it any time, even on a speakerphone: it recognises its own voice coming back through the mic and only stops for yours
• Pause and carry on: if it starts answering and you keep talking within a second, the answer is withdrawn and everything you said is answered as one turn
• Hands-free, push-to-talk, or wake word ("Hey Ming, …"), with a follow-up window so you don't repeat the name
• Survives restarts: the conversation is saved after every turn and picked up again

Useful out of the box

• Web search, weather, calculator and clock
• Timers and reminders that ring again until you answer
• Shopping and to-do lists, and remembering things you tell it ("I parked on level 3")
• Home Assistant: lights and plugs (on/off, brightness, "dimmer", colours, warm white), music and TVs (pause, skip, volume, "what's playing?"), thermostat, locks, the car (climate, temperature, defrost), the vacuum, scripts and scenes by name. Follow-ups work ("now turn it off", "and the living room too"), and the user's own words are used, not the model's rewording. Clear commands skip straight to Home Assistant. Two at once ("turn off the desk lamp and lock the car"), for later ("turn off the lights in 20 minutes", "start the vacuum at 9 am"), and rooms: pick the room the device is in, and "turn off the lights" means that room's. "Play some jazz on the kitchen speaker" works with Music Assistant installed. Heads-ups: a lock or garage door left open at night, the vacuum stuck or finished. Also Jellyfin, and hand-off to a Hermes Agent for bigger jobs
• Risky actions (unlocking, opening the garage) are confirmed first
• Jarvis mode: a dry-witted British robot butler ("Certainly, sir.") with its own designed voice and wake word
• Say "that's all, bye" and it ends the conversation (in wake-word mode it goes back to sleep)

Made to be lived with

• Runs always on with the PC's own mic and speakers, no browser tab needed
• Use it from your phone over HTTPS on your home network (QR code, installable web app)
• 7 built-in voices plus voice cloning from a 5–15 s sample
• Speaks numbers, money and units properly ("$1,299.99", "−5°C", "24-17", "12 km/h")
• OpenAI-compatible API for text-to-speech, transcription, and chat with audio in and out
• English and Chinese

On your phone: the same app over your home Wi-Fi, installable on the home screen.	Tune everything: turn-taking, voice, timing and persona.

Measured on a Radeon RX 7900 XTX (24 GB):

• Linux (x86_64): Arch, CachyOS, EndeavourOS, Manjaro, Ubuntu, Debian, Pop!_OS, Mint, Fedora, Nobara or openSUSE (others work too if they have git and curl), or Windows 11 with WSL2 (the installer sets it up)
• A GPU with 16 GB+ VRAM:
  • AMD Radeon RX 6800 or newer: RX 7000 / 9000 series and Radeon PRO cards (tested on the RX 7900 XTX). On Linux it only needs the standard amdgpu driver that comes with your kernel: PyTorch brings its own ROCm 7.2, so there's no ROCm to install. On Windows: a recent Adrenalin driver with WSL support.
  • NVIDIA RTX 30 series or newer (e.g. RTX 3090, 4060 Ti 16 GB, 4080, 4090, 5080, 5090), with a current driver (on Linux: your distro's NVIDIA driver, e.g. nvidia-open on Arch/CachyOS). Experimental: the installer sets it up with CUDA PyTorch and the code is vendor-neutral, but it hasn't been tested on a 16 GB NVIDIA card yet. Reports welcome!
• ~30 GB free disk (20 GB model + Python environment)

Linux: open a terminal and paste:

curl -fsSL https://raw.githubusercontent.com/JMS1717/ming-voice/main/install.sh | bash

It installs what's missing (git, curl, openssl, PortAudio) with your package manager, finds your GPU, sets up Python and PyTorch for it (ROCm 7.2 or CUDA 12.8, nothing system-wide), downloads the model (~20 GB) into ~/ming-voice, and adds Ming Voice to your app menu. It may ask for your password (for the packages, and on AMD to add you to the render group that may use the GPU; log out and back in once after that). If anything stops halfway, run it again: finished steps are skipped. Set MING_DIR to install somewhere else.

Windows: open PowerShell and paste:

irm https://raw.githubusercontent.com/JMS1717/ming-voice/main/install.ps1 | iex

That's it. It sets up WSL and Ubuntu 24.04 if you don't have them yet, then installs ROCm (AMD only), the Python environment and the model (~20 GB download), and puts a Ming Voice shortcut on your Desktop. On AMD it asks for your Linux password once (for ROCm). If anything stops halfway, run it again: finished steps are skipped. (Prefer not to paste commands? Download the repo and double-click Install.cmd.)

Already in a WSL Ubuntu shell? Same thing from there:

git clone https://github.com/JMS1717/ming-voice.git ~/ming-voice
bash ~/ming-voice/install.sh

sudo bash scripts/install_rocm_wsl.sh      # AMD only: ROCm 7.2.4 + librocdxg (WSL GPU bridge); then wsl --shutdown
bash setup_env.sh                          # Python env (ROCm or CUDA PyTorch, needs uv) + model download (~20 GB)

• Windows: double-click Ming Voice on your Desktop (or windows\Ming Voice.cmd in the repo, at \\wsl.localhost\Ubuntu-24.04\home\<you>\ming-voice). It starts the server, waits for the model, and opens http://localhost:8000.
• WSL shell: ./run.sh, then open http://localhost:8000.
• Linux: Ming Voice in your app menu, or ming-voice in a terminal. It starts the server in the background, waits for the model, and opens http://localhost:8000. ming-voice --stop stops it (it holds ~20 GB of VRAM), ming-voice --status and ming-voice --logs show what it's doing, ming-voice --help lists the rest. (Or run ./run.sh in the folder, in the foreground.)

Press Start conversation, allow the microphone, and talk. Pause when you're done.

Windows: keep the server window open while you use it: WSL shuts its VM down when nothing is attached. Close it when you're done, since it holds ~20 GB of VRAM.

• Interrupt by talking over it. If it still interrupts itself in a very echoey room, turn barge-in off in Settings.
• Shortcuts: Space hold to talk (push-to-talk mode), M mute, Esc stop the reply, N new conversation.
• Wake word (Settings → Turn-taking) is for a PC, tablet or old phone that listens all the time. It only answers speech that starts with its name: "Hey Ming, turn off the lamp", or "Hey Ming" on its own and wait for the chime. For 8 s after each reply you can follow up without the name. Any name works; "Ming" is recognised best.
• Clone a voice in the sidebar: record or upload 5–15 s of one person speaking. Voices are saved in voices/.
• Type instead of talking in the box at the bottom; the answer is still spoken.
• Jarvis mode: pick Jarvis (robot butler) under Settings → Persona. It switches to the Jarvis voice and the wake word "Jarvis" (your own come back when you pick another preset), and even the tool confirmations are in character ("Very good, sir. I've turned off the lamp."). The voice is an original one made with the Talker's voice design (dev/design_voice.py), not a copy of anyone's.
• End the conversation by saying goodbye ("that's all, thanks", "goodnight", "end the conversation"): it says goodbye and stops listening, or goes back to sleep in wake-word mode.

windows\Ming Voice (always on).cmd starts the server (minimized), phone access, and windows\ming_client.py, a background client that uses the PC's own microphone and speakers. Just say "Hey Ming, …" any time. Timer alarms and announcements play through the speakers too.

On Linux: ming-voice --always-on (or Always on in the app menu entry's right-click menu) does the same, with the same client and settings. ming-voice --autostart --always-on starts it when you log in (add --phone for phone access too; --no-autostart undoes it), and ming-voice --stop stops it. The notes below apply to both, with the paths on Linux being .venv/bin/python windows/ming_client.py and ming_client.log.

• Start at sign-in: run powershell -ExecutionPolicy Bypass -File windows\install_autostart.ps1 once (-Remove undoes it; -Browser autostarts the browser app instead).
• Needs Python for Windows with pip install sounddevice websockets numpy.
• Settings go in .client.json in the project folder (not committed), for example {"wake_word": "Ming", "home": "Paris", "voice": "US_Emma", "input_device": "USB Mic", "barge_in": false}.
• Pick the right microphone: python windows\ming_client.py --test-mics shows each microphone's level while you talk (--list-devices lists them all). The log, windows\ming_client.log, names the mic in use and warns when it has heard nothing for 10 minutes. --stop stops the client.
• On speakers the mic is muted while Ming talks, so it never answers itself. With a headset, set "barge_in": true to talk over it.
• One device at a time: opening the app on your phone takes over, and the PC client takes it back 15 s after the phone app is closed.
• The server restarts by itself after a crash or an unrecoverable GPU error (run.sh with MING_SUPERVISE=1, which the launcher sets).

Start with windows\Ming Voice (phone).cmd (or run python windows\lan_proxy.py on Windows next to a running server), press Phone in the app on the PC, and scan the QR code. The phone opens https://<pc-ip>:8443/?key=…; the key is swapped for a cookie, so the link only has to be opened once.

• Phones only allow the microphone over HTTPS, so the proxy uses a self-signed certificate (created on first run in certs/). The phone warns once that the connection isn't private: tap Advanced → Proceed (Safari: Show Details → visit this website).
• Everything arriving from the network needs the access key (.lan_key, created on first start). Delete the file and restart to change it.
• WSL2 isn't reachable from the network by itself, which is why the proxy runs on Windows (standard library only). If Windows Firewall asks, allow Python on private networks.
• Linux: ming-voice --phone starts the same proxy next to the server. With a firewall on (Fedora and openSUSE have one by default), allow port 8443/tcp: sudo firewall-cmd --add-port=8443/tcp --permanent && sudo firewall-cmd --reload (ufw: sudo ufw allow 8443/tcp).
• Add the page to your home screen to use it like an app; the screen stays on during a conversation.

With Web search & tools on (Settings, default), the model decides when to call a tool; Ming-3.0's LLM was trained for function calling. It says "Let me look that up" while a slow one runs, and the reply shows what it used and its sources.

Guard rails that keep it honest and safe:

• Tools that change something (timers, lists, memory, Home Assistant, Jellyfin, Hermes) never run for a speculative reply. They wait until your turn has really ended, so a pause mid-sentence can't switch a light.
• Risky actions are confirmed first. Unlocking, opening a garage door or gate, and disarming an alarm get a question back ("Just to be sure: do you want me to unlock the front door?") and only happen when you say yes in your next turn, within a minute. MING_CONFIRM_RISKY=0 turns this off.
• Jellyfin only starts playback when you said to (play, watch, put on, listen, …).
• Answers you can listen to: one from search results ends after a few sentences, and any other after about 30 s of speech, unless you asked for something long (a story, a recipe, steps, "in detail", …).
• If a reply starts by claiming it can't look things up ("I don't have access to real-time data…"), it's caught before it's spoken and becomes a web search or weather lookup instead. And if a reply says it changed a list or forgot something without calling the tool, that call is made for it, so what it said is true.

Ming can tell you what happens at home: the doorbell, the washer finishing, the garage left open. With a Home Assistant token configured, the server listens for ming_announce events, so all it takes is an automation with an event action (no integration, no configuration.yaml change, no restart):

alias: Doorbell → Ming
triggers:
  - trigger: state
    entity_id: binary_sensor.front_doorbell
    to: "on"
actions:
  - action: event
    event: ming_announce
    event_data:
      message: Someone is at the front door.
      # chime: false      no chime before it
      # verbatim: false   pass it on in Ming's own words instead of exactly
      # ttl: 300          drop it if it can't be said within this many seconds

Or from a script or another server, POST /api/announce with the same fields as JSON (on the LAN port, with Authorization: Bearer <access key>):

curl -X POST http://localhost:8000/api/announce -H "Content-Type: application/json" \
  -d '{"message": "The washer is done."}'

An announcement waits for a quiet moment, plays a short chime, and is said whether the wake word was heard or not. MING_HA_EVENTS=0 turns the event listener off.

The same server speaks the OpenAI API; interactive docs are at http://localhost:8000/docs.

curl http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" \
  -d '{"input": "Hello from Ming Voice!", "voice": "US_Emma"}' -o hello.wav
curl http://localhost:8000/v1/audio/transcriptions -F file=@hello.wav

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused-on-localhost")
print(client.chat.completions.create(model="ming-3.0-audio-tiny",
      messages=[{"role": "user", "content": "Tell me a fun fact about otters."}]).choices[0].message.content)

From another device, use https://<pc-ip>:8443/v1 with api_key=<contents of .lan_key> and accept the self-signed certificate (curl -k, or pass certs/lan.crt as the CA). API requests share the GPU with the voice session; chat requests run on an isolated copy of the conversation state, so they never disturb a live conversation.

browser (static/index.html) ⇄ WebSocket: 16 kHz PCM in / 44.1 kHz PCM out ⇄ server.py
  server.py   Silero VAD · Smart Turn (turn_detect.py) · speculative turn-taking with rollback · barge-in
              · resume (a reply you talk over within resume_ms is undone and your turn continues)
              · echo guard (echo_guard.py: mic vs. played-audio correlation, so its own voice can't barge in)
              · push-to-talk · voice cloning · background transcription
  api.py      OpenAI-compatible /v1 API (speech, transcriptions, chat with audio in/out)
  tools.py    web search, weather, timers, lists, memory, Home Assistant, Jellyfin, Hermes
  ha.py       Home Assistant device list (registries over the websocket API), command parsing and control
  lan.py      access key + cookie login for the LAN port; windows/lan_proxy.py terminates HTTPS on Windows
  engine.py   Thinker: Ming audio LLM (hybrid Kimi-delta-attention / MLA, 128-expert MoE)
                - static-state cache, one HIP graph per attention-length bucket (8.7 ms/token)
                - prefill graphs too: prompts padded to 32/64/128 tokens, padding made a no-op for the
                  recurrent KDA layers (zero write strength, unit decay, conv state rebuilt from real tokens)
                - fused Triton MoE with the shared expert folded in (fused_moe.py): a GEMV kernel pair
                  for decode, grouped tl.dot kernels for prefill (tokens sorted by expert on the GPU)
                - multi-turn KV reuse, snapshot/restore for speculation, text-history rollover
              Talker: Qwen2 + 10-step flow-matching DiT + AudioVAE
                - fused Triton fast path (fast_talker.py, fastlinear.py): RMSNorm/bias/activation/
                  residual fused into small-M GEMMs, tiny rotary+attention kernel, decode attention
                  over a static KV cache, constants hoisted out of the flow-matching loop
                - HIP graph per step, streaming VAE decode
  direct_load.py  builds the model on the GPU and streams weights in with parallel reads

The main speedups over the reference implementation came from replacing rocBLAS's slow small-M biased GEMMs (~45 µs) with fused Triton kernels (~20 µs), cutting kernel launches (the reference MoE prefill launched ~5 kernels per active expert per layer plus a host sync, ~120 ms per turn, now ~15 ms), graph-capturing every per-token step, and hiding model latency inside the end-of-turn pause.

• No HIP GPUs are available: in WSL, HSA_ENABLE_DXG_DETECTION=1 must be set (run.sh does it). On Linux, your user must be in the render group (groups shows it; the installer adds you, then log out and back in).
• Linux, AMD GPU not listed by PyTorch (e.g. an RX 6700 XT or 7600): the installer writes the closest supported GPU to .gpu_env (HSA_OVERRIDE_GFX_VERSION); edit or delete that file to change it. With an integrated GPU too, .gpu_env also picks the big one (ROCR_VISIBLE_DEVICES).
• The first request after idle takes minutes: under WSL, an AMD GPU that drives no display stops serving WSL work promptly when idle. The server runs a tiny keep-alive kernel a few times per second.
• Slow loading or swapping: the loader streams weights straight to the GPU; Hugging Face's loader peaks at ~15 GB of host RAM, more than WSL allows by default.
• It cuts you off: raise the end-of-turn pause in Settings.
• The always-on client never hears you: run python windows\ming_client.py --test-mics (Linux: .venv/bin/python windows/ming_client.py --test-mics) and set input_device in .client.json.
• Logs are in server.log (Linux: ming-voice --logs); health is at http://localhost:8000/api/health.

• This project: MIT (see LICENSE).
• Model: Ming-3.0-audio-tiny by inclusionAI (MIT). The weights are downloaded from Hugging Face, not included here; the model's own code is loaded from the model folder with trust_remote_code.
• Smart Turn v3 by Pipecat (BSD-2-Clause), downloaded by setup_env.sh.
• Silero VAD (MIT), flash-linear-attention (MIT), PyTorch, ROCm and Triton.

If Ming Voice is useful to you, you can support its development with a donation through PayPal. Thank you!