Skip to content

Speech Studio 0.2.0

Choose a tag to compare

@timoncool timoncool released this 20 Oct 02:52

Added

  • Live: talk to an agent by voice. An agent connected over MCP says each phrase as soon as it has written it; you answer into the microphone and are recognised phrase by phrase, and speaking over the agent stops it. The controls are those of voice modes: mute with M or Ctrl+Shift+Space from any window (hold to talk), text to the agent, the recogniser and the agent's voice, the agent's name on its lines, Ctrl+3 and the command palette. The Microphone and sound window has mixer-style level meters, Hear yourself, a speaker test and the agent's volume; five looks (aura, wave, bars, circle, matrix) and a widget over every other window.
  • Drama 3 from 6 GB cards. When the model does not fit, its experts (18 GB) live in RAM (32 GB needed) and the card keeps a cache of the ones in use; the sound is the same. On an RTX 4090: whole ~8× faster than real time, 12 GB 4.7×, 8 GB 3×, 6 GB 2.2×.
  • Voices of the pack cloned in Drama 3, and cloud voices played phrase by phrase while they voice.
  • Gemini TTS with parallel requests or Batch at half price, and Game Dub's checks of Google's takes.
  • Editing. What to fix on a take, Edit the phrase from the player, trimming a take on its waveform, A/B of takes, a blind comparison of engines, fixing a take by voicing again only the place of the error, approved blocks and notes on words.
  • Text. An SSML subset, prosody hints coloured by word, phone numbers, e-mail and codes, a passage in another language in its own voice, the director placing punctuation and tags without changing a word, and a script check before voicing.
  • Books and plays. Direct speech apart from the narrator's, a character register with voice casting, one voice per project, footnote cleaning, the book's names into the dictionary, a project updated from a corrected book without losing what was voiced, and an HTML book player.
  • Montage. A track editor (shift, trim, fades, volume) with music and sound tracks from files, folders or a song made in YuE2, MiniMax Music3 or ACE-Step Studio.
  • Export. SRT, VTT, ASS and LRC subtitles, sample rate and bit depth, voice effects (radio, phone, loudspeaker), even loudness of pieces, a file per chapter, a track per character in a ZIP, a CSV table of lines, name templates, a SHA-256 manifest and "how it was made".
  • Voices. Recording from the voice picker, trimming a reference on its waveform, character variants, a voice by traits, a voice's licence and several pieces of a long reference.
  • Quality. A traffic light and a project report, and a corpus of hard Russian phrases to check an engine.
  • Nemotron 3.5 for recognition (28 languages with punctuation); in Live it recognises a phrase as it is spoken instead of again from its start.
  • The recording's language under Auto is detected by VoxLingua107, and the recording goes to a recogniser that knows that language; speech is found by Silero VAD v6.2.
  • Listening. Recording from the microphone with recognition on the go, a folder of recordings recognised in the background, quick recognition without the editor, a batch of files in several formats, subtitles by broadcasters' rules (Netflix, BBC and others), who speaks how much, a phrase repeat, numbers as digits in export, and the recogniser picked by the recording's language.
  • The window and agents. Every route and action of the window is an MCP tool and the agent can be the studio's own language model; the Ctrl+K command palette, your own hotkeys, a simple mode, source tabs, Finished files, Restore after deleting a project, "?" hints on every element in six languages, verb-and-object buttons and a phone layout.
  • The project site in six languages with samples of every engine: https://timoncool.github.io/speech-studio/

Changed

  • Recognition on transcribe.cpp. Parakeet-TDT v3 and Parakeet Ultra moved from ONNX Runtime to transcribe.cpp, and the model stays loaded between jobs; on an NVIDIA card the CUDA build runs, without one the processor build, switched in the settings without a restart. Whisper stays on faster-whisper for the time of every word. The engine and the models are the ones Dub Studio uses, so either program takes them from the other. The ONNX recognition models of 0.1.0 are removed at start.
  • First run asks only what you want: voices, recognition (one Whisper model or none) and extras; the engines, libraries and the build for your card come with them by themselves, named in one line with their size.

Fixed

  • Voicing no longer stops on "stress lock" until a restart. When ONNX Runtime was downloaded after start (on the first-run screen), Windows loaded a foreign onnxruntime.dll 1.17 from System32, loading stress failed and every following block refused; the studio now always takes its own ONNX Runtime, and a failed load names its reason without stopping the next block.