Skip to content

Speech Studio 0.3.0

Choose a tag to compare

@timoncool timoncool released this 27 Oct 00:40
· 104 commits to main since this release

Changed

  • Speech Studio for macOS on Apple silicon (macOS 13 and newer, a .dmg): First run downloads the macOS builds of recognition (transcribe.cpp), Qwen3-TTS, separation, the local LLM, ONNX Runtime, ffmpeg and yt-dlp, each with Metal where it has it, and the audio.cpp engine with Metal for Higgs, Supertonic, CrisperWhisper, Parakeet and Nemotron; the script assistant, voices and separation run on Metal too. Drama 3 runs on NVIDIA cards only. A deleted project is taken back from the Bin in Finder (Put Back): macOS lets no program do it.
  • Studio notices come as a window too: a notice of the studio's feed can open as a window over the studio, once ready, after its delay and only while nothing is being voiced, recorded or said aloud; its buttons open a link, the news, Settings → Models or About. A strip or a window shows only in the sections it is meant for (making a work, the voice library, the news, the tools, Settings), and an advertisement carries its label and erid.
  • News from the feed have their picture and buttons, and a button can open a link, Settings or the news.
  • The anonymous report says how fast voicing goes on the card: seconds of speech voiced and seconds the engine worked for them, for local engines (Settings → Privacy says so). The window's own polling (microphone state, recording level, resource monitor, queue) is no longer counted as using the studio.
  • Whisper is CrisperWhisper 2.0 turbo in the audio.cpp engine, as in Dub Studio: 99 languages with word times, the language of Auto heard by VoxLingua107, the Listening prompt passed to it. Faster-Whisper, its six models and its CUDA 11 libraries are gone, and this program's copies of them are removed at start (a model folder shared with another program loses only its link).
  • Higgs Audio v3 runs in the audio.cpp server that CrisperWhisper runs in, as in Dub Studio, instead of its own engine library: the setup screen downloads the server once (131 MB, «Voice and recognition engine (audio.cpp)»), and the old library is removed. It has machine code for RTX 50 cards too.
  • Higgs runs on graphics cards from GTX 900 and GTX 10 to Volta and on drivers 525-579: the audio.cpp engine comes in a CUDA 12.9 build for the cards and drivers CUDA 13 left, with its CUDA 12.9 runtime; before, Higgs said such a card is too old. The engine's CUDA libraries are checked before it starts and a missing one is named.
  • Cloud voicing works without choosing a model or a voice first: OpenRouter starts with Gemini 3.1 Flash TTS (it speaks Russian) and its first adult voice, Google with Gemini 3.8 Flash Lite TTS, and Settings show them; the projects and settings that chose their own keep them.
  • Downloads trust the certificates Windows trusts: behind a proxy or an antivirus that checks HTTPS with its own certificate, components failed with «invalid peer certificate: UnknownIssuer».
  • A graphics card to run on, when a computer has more than one NVIDIA card (Settings → Models → Graphics card; from the next start): the studio and every engine it starts see that card alone. The card the studio names, its memory and the resource monitor are now the card CUDA really uses, found by its PCI bus id, not the first card NVML lists. A chosen card that is no longer in the computer is said in Settings and by every stage that needs it, instead of the work landing on another card (#10).
  • The studio has its own volume (Settings → Language and sounds → Studio volume): everything it plays — voicing, voices, recordings and notification sounds — at that level, so switching to another window no longer brings a sudden loud take (#11).
  • A click outside «Voice from a description» no longer closes it while it holds a description, a sample or made voices, and does not close the voice library behind it either; the window still closes with its cross or Esc (#6).
  • The detached resource monitor follows the light theme: its panel, bars and figures were drawn in dark colours (#7).
  • Voicing names every block that has no voice before any model loads, when the engine of those blocks speaks only with a library voice, instead of failing block by block.
  • An engine that only clones gets a voice to start with: Drama 3 or Qwen3-TTS with Russian stress chosen for a project, speaker or block that has no voice takes a library voice of its language (a favourite first, then one of the pack; for Drama 3 one with its reference text), and Project settings say so with a link to the voice library when the library has none.
  • Recognition runs on Vulkan on an AMD or Intel card and on an NVIDIA card CUDA 13 does not run on (Auto), and Vulkan can be chosen by hand in Settings → Models → Recognition; before, such a computer recognised on the processor.

Fixed

  • Supertonic 3 and speaker diarization run in the audio.cpp engine of this release (server-6d6cb6a5 on Windows and Linux), which reads Supertonic's int8 model and Nemotron 3 Diarization in ONNX.
  • An engine taken from another program counts as installed: the record of the build it replaced stayed, so First run asked for the same engine again after taking it.
  • Recognition engines of transcribe.cpp 0.1.4 on every platform, the Mac with Metal included (it had 0.1.1): the studio takes their pins from the shared recognition crate, as Dub Studio does.
  • A file another program has on another drive no longer removes this program's own copy first: the studio's file is replaced only by a finished link or copy.
  • First run names its button as it is («Download checked» in Russian, Spanish and Portuguese), and a component taken from another install of Speech Studio says so instead of «taken from Speech Studio».
  • A program downloaded on Linux or macOS runs even when its archive carries no execute bit (the audio.cpp server of Higgs, CrisperWhisper and Parakeet answered «Permission denied»): after unpacking, a component's program is made executable.
  • Voicing finishes on Linux with the system's ffmpeg: the AppImage gave ffmpeg, ffprobe, yt-dlp and the file manager its own libraries, and a newer system's ffmpeg stopped with «undefined symbol vaMapBuffer2»; programs of the computer now start without them.
  • The script assistant starts on Linux older than glibc 2.38 (Ubuntu 22.04 and the like): the CUDA builds of llama.cpp need glibc 2.38, so there the assistant runs on the card through the Vulkan build, and Settings → Models says why.
  • A window 768 to 1023 pixels wide no longer scrolls sideways: the top bar shows Speech, Listening and Live by their icons there, with their names from 1024 pixels on.
  • The status line names the voicing model the studio really uses outside Speech too: Higgs with its quantization, or the OpenRouter or Google model; it showed Higgs whatever was chosen.
  • The status line names the assistant the studio really uses: Gemma's file, the OpenRouter or local server model, or the agent (MCP); it showed Gemma whatever was chosen, and now follows a preset or a change by the agent at once.
  • A hardware preset moves every stage and stays shown: «CPU + cloud» left voicing, recognition and separation on the card when they had a device of their own, and Settings → Models showed the recommended preset again once reopened; now a preset sets the device of each stage, and the list shows the preset the settings are on, or Custom.
  • Linux and macOS know the free space of the models folder: First run said the free space is unknown, and a download too big for the disk was not stopped before it began.
  • Nemotron under Auto keeps the first word of what is said («Победа пришла…» was heard as «Пришла…»): Live and Listening tell it the language VoxLingua107 hears among its own, the first two seconds of speech in Live, before it recognises anything.
  • Recognised Chinese, Japanese and Thai text has no spaces between its characters in Live, Listening and the take check (it read «今天 天 气 很好 ,»), and no text has a space before a comma or a full stop.
  • ONNX Runtime sends Microsoft no usage telemetry: the studio sets its environment with telemetry off before any model of it loads (stress marks, noise removal, speaker voices), and does not start if it cannot.
  • Casting a book keeps the book's words whatever the model writes («the director model changed the book's text twice»): the model only says who speaks and where each line begins, and every line's text is cut from the book itself; a chunk fails only when the model's answer is another text.
  • Higgs stops a phrase that does not end a second past its runaway length: the cap counted 75 tokens a second where audio.cpp counts 25 codec frames, so a phrase that would not stop ran three times longer than meant before it was judged and voiced again.
  • Speech detection fetches what it lacks: Live, Listening and the take check download the transcribe.cpp build of the recognition device or the Silero VAD model with the job's progress. Before, Live asked for Silero VAD, which was there, and kept refusing to start, and the take check and Listening stopped with «Silero VAD is not downloaded» when it was the engine build that was missing.
  • Renaming a library voice no longer breaks the projects that speak with it («There is no voice named …»): they follow the new name, and the blocks voiced with it stay voiced.
  • Chinese, Japanese and Korean text in a project of language Auto is no longer failed as «runaway»: its length was judged by the pace of Russian letters, so a long sentence of ideographs counted as speech that would not stop on every attempt; the language is now read from the letters when the project leaves it to Auto.
  • A window opened over the editor pauses what the editor plays (#14): the voice library, the settings or any other window covered the player while a project or a recording went on playing with no way to stop it; it pauses where it was, and Play goes on from there. Pausing right after Play no longer shows «The play() request was interrupted by a call to pause()».
  • Another engine's built-in voice no longer fails voicing block by block («There is no voice named F1»): choosing Higgs for a block or a speaker drops a Supertonic or Qwen voice it cannot speak, changing the project's engine clears such a voice from the voices by language too, and voicing names every block whose voice is neither in the library nor its engine's own before any model loads.
  • A failed download tells the anonymous statistics which component failed and on which site (for example network asr-gguf-nemotron huggingface.co), so a cause behind "network" can be found; nothing about the computer or the person is added.
  • A recording of only silence is not saved as a voice: when the system keeps the microphone closed or it is muted, recording a voice ends with what to check instead of a voice that recognition later finds empty.
  • The director splits a chapter without blank lines into pieces the model can hold: a book whose paragraphs are separated by single line breaks went to the model as one piece, which overran a local model's context or cut its answer short; a long paragraph now splits by lines, a long line by sentences (Chinese and Japanese full stops too), a sentence by its last space.
  • Out of video memory in the Higgs engine says what to do: a lighter quantization, a shorter block or closing the programs on the card, instead of the engine's line alone.
  • Drama 3 says when its video memory limit is the cause: a limit set by hand that leaves its expert cache no room is named with the setting, and a card filled by other programs is said as such, instead of the engine's numbers alone.
  • Recognition no longer closes the program on processors without AVX-512 (Intel Core before the 11th generation and the 12th–14th, most AMD before Zen 4): transcribe.cpp engine-0.1.3 is built for AVX2, FMA and F16C instead of the build machine's own instructions, and a processor without them gets a message instead of a crash. The take check, a voice's reference text, cloning from a long reference and Listening all went through it.
  • Voices of the pack with their text in a .lab file have their reference text (English_Female, Arabic_Female and six more): the studio read only .txt, so Drama 3 refused them for a missing transcript and the studio tried to recognise text it already had.
  • Drama 3 says which voice lacks its reference text and where to add it (Voice library → Reference text, or Recognise the reference text), and does so before loading the model; a long voice with no piece of whole words up to the reference length is named as such.
  • Cleaning a voice works on any sample rate: a voice recorded at 48 or 24 kHz went to the separator as it was, and the separator, which takes 44.1 kHz only, failed with exit code 1.
  • The log is kept on disk and survives a crash (logs/speech-studio.log in the data folder, the previous one beside it as speech-studio.old.log past 8 MB): every step of a job, every failed job with its whole error, a panic of any thread and, on Windows, an exception that ends the process with its code and the module it came from. Report a problem packs both files.
  • Checking the SHA-256 after a download moves the bar (bytes checked of the bytes to check) instead of leaving it at 100 %; a failure inside the download thread ends the download with its cause instead of leaving it spinning.
  • A synthesis that timed out or hung names its engine.
  • The director's verbatim check takes full-width digits and letters (3, A) as the ordinary ones.
  • Drama 3, Qwen3-TTS and the separator check the processor before they load (AVX2, FMA, F16C, BMI2) and name what is missing instead of closing the program.
  • Cleaning a voice works in a folder with Cyrillic or other non-ASCII letters in its path, and a separator failure shows its error line.
  • Higgs voices in a data folder with Cyrillic or other non-ASCII letters in its path: its engine library could not open such a path and said the model folder does not exist.
  • An engine left over from an earlier version is downloaded again: an engine or program counts as installed only when its SHA-256 is the pinned one, not when a file of its name and size is there (a build with other processor flags can be the same size).
  • The separator's graphics card build that found no card stops with that cause instead of cleaning on the processor unnoticed, and the local Gemma build that sees no card says so before loading.
  • CrisperWhisper is not offered Vulkan: Settings → Models → Recognition shows Vulkan only for Parakeet and Nemotron, and CrisperWhisper with Vulkan chosen earlier says it has no Vulkan build instead of running on CUDA or the processor.
  • The status line at the bottom follows the settings: a recogniser, a device or a voice model chosen in Settings shows there at once, not after the next start.
  • The diagnostics check the transcribe.cpp builds' libraries too.
  • Stop ends the engines a job started at once (separator, llama-server, ffmpeg, yt-dlp): on Windows they were ended through taskkill, which waits as long as WMI hangs, so a stopped job could keep running to its end.
  • An edited reference text is used at once: the piece cut from a voice was kept by the audio's size and date alone, so a corrected text or new word timings came in only after the audio changed.
  • Assistant streaming. Text remains intact across network packets, malformed model events report errors, and provider-reported costs are retained when a stream fails.
  • Assistant tool batches. A batch cannot bypass the chat's tool restrictions and start a recursive assistant request.
  • Link import retries. A failed Listening-project import preserves the downloaded media for retry and cleans up the unfinished project directory.
  • Stress model errors. Missing, malformed or non-finite RUAccent outputs now report an error instead of panicking or silently skipping stress marks.
  • Dictionary import errors. Malformed multipart uploads now report an error even when an earlier part contained a valid dictionary.
  • Noise suppression errors. Invalid DPDFNet metadata or output sizes now return an error instead of panicking during processing.
  • DLL diagnostics. Malformed import addresses and unsupported PE headers report an error instead of panicking or passing the check.
  • Credential saves. Failed key and proxy password replacements clean up their temporary files and preserve the previous value.
  • Reliable file saves. Concurrent saves of a voice's card, transcript and image no longer share a temporary file.
    Failed writes and blocked replacements preserve the previous file.