Skip to content

Releases: hehehai/AudioSmith

v0.4.0-beta.8

v0.4.0-beta.8 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 26 Oct 01:57
7bad165

Idle models leave memory

  • Models you have not used for 30 minutes are now unloaded, dictation's speech model included, so AudioSmith gives the memory back to your Mac. They load again by themselves when next used: the first dictation after a long pause may take about a second longer, and the pill says “Loading model…” while it waits. Nothing you record is lost.
  • Settings → Models & network → Memory: “Unload idle models” offers after 5, 15, 30 or 60 minutes, or Never. If you had never changed it, it moves from Never to 30 minutes once; a choice you made is kept.
  • A model is never unloaded while a job uses it, during a dictation or during a live call; the 30 minutes count from its last use.

Fixes

  • Setup: "Download models later" did nothing (issue #1): the wizard put the recommended pack back on the next frame, so the wizard went on with it. The choice now sticks.
  • Dub: a line that is not translated yet opens in the editor on its text. It used to open empty, so the text seemed to vanish when clicked (issue #2). Done without a change no longer saves the source as the translation.

For contributors

  • The app's Go code (package main) moved from the repository root to app/ (mygo.json "main": "./app"); the files it embeds moved to the assets package (assets/, starter voices now in assets/starter-voices). Run it with go run ./app, test it with go test ./app; go tool mygo build, mygo dev and mygo vet . work as before.

Download: actnow.dev/download/audiosmith (a free account) · Changelog · or build it from this tag's source (AGPL-3.0).

Installed copies update themselves: AudioSmith → Check for Updates…

v0.4.0-beta.7

v0.4.0-beta.7 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 24 Oct 06:00

Dub timeline: zoom and pan

  • Zoom the waveform timeline with a trackpad pinch, ⌘ or ⌥ with the scroll wheel, or ⌘= and ⌘−, around the pointer; ⌘0 or a double click on the zoom readout shows it all. It zooms from the whole source to 100 pixels a second.
  • Pan with a sideways two-finger swipe (with the system's inertia), ⇧ with the wheel, a middle-button drag, or the new overview strip under the zoom controls, which shows the part in view over the whole source; drag it, or click beside it to jump there.
  • While playing zoomed in, the view turns the page as the playhead reaches its edge, unless you panned in the last few seconds.
  • Dense lines no longer crowd the timeline with numbers: a number shows only where it fits and leaves room after the last one.

Dub: see the video before it downloads

  • Paste a link and the Dub page shows what it is within seconds, before anything downloads: the cover, the site (YouTube, Bilibili, Vimeo, TikTok and the others yt-dlp knows), the title, channel, length, picture size, estimated download size, date, views, the description (Expand shows all of it) and the caption languages. It is kept in the dub, so it shows again without the network.
  • While the video downloads, a thin bar under the cover gives the size and time left; once it is downloaded, the centre shows the video itself, with play, while transcribing and the steps after it report in the footer.
  • A link that cannot be read (a private or removed video, one that needs a sign-in, a site that is not supported) says so before any download, with the site's reason, and Try again. Cancel stops the reading as it stops a download.
  • A dropped video or audio file shows its poster, length, picture and size the same way.
  • The cover is fetched by the app itself, through the same proxy as the download, from public addresses only, at most 5 MB, and kept only if it is a JPEG, PNG or WebP picture.

Faster with long projects

  • Dub: the editor builds only the lines in view; a 2-hour video's 2,000 lines no longer slow every frame (190 ms → under 1 ms a frame).
  • Audiobook and Stories: a long manuscript (a novel) is checked and its draft saved once you pause typing, away from the window, not at every key; typing in 500,000 characters is as quick as in a short text.
  • Studio and Audiobook editors: wrapping long text remembers what it measured, so opening, resizing and typing in a long manuscript no longer stall (rows still never pass the editor's edge).
  • The invisible watermark is added while the audio streams through, so watermarking an hours-long dub no longer holds the whole recording in memory several times.
  • The engine keeps up to 6 models loaded (small ones don't count) and lets the memory budget decide, so a story render no longer reloads a model at every line.
  • Finished tasks older than 30 days are removed at startup; voice previews no longer copy their audio every frame; the render cache is tidied at most once a minute.

Security

  • Voice names are cleaned wherever they come in (imported .asvoice and .ovsvoice files, the Local API, MCP, the app, Discover voices): line breaks and control characters become spaces, and a name keeps at most 80 characters. Names saved earlier are cleaned the next time they are saved.
  • The agents' skill and setup prompt list voice names as data (quoted, cut short), not as text an agent could take for instructions.
  • MCP voice cloning and design work only from this Mac (or with the API key), like the Local API's; over the LAN, a voice's audio is its sample, never the reference recording.
  • Built with Go 1.27.3 (security fixes in Go's standard library).

Logs

  • An operations log in the app's log (source “ops”, one key=value line each): creating, renaming, deleting, importing and exporting voices, projects, dubs, transcripts, workflows and takes, with who asked (the app, the Local API, LAN, MCP); settings changes by name; each job's start, end, failure (with its error) and cancel (with who cancelled and how long it ran); downloads; Local API and MCP calls that change something, with their status; errors shown as toasts; the app's start and each step of its quit. Never script text, transcripts, prompts or secrets.
  • Logs and diagnostic bundles mask Local API keys, Twilio tokens, keys read from the Keychain and bearer tokens, and show the home folder as “~”; the engines' own log files are masked too.
  • Errors that were dropped silently in background work (cancelling jobs, export records, notifications, files the app writes or removes…) are logged.

Fixes

  • Studio and Audiobook script editors: the last character of a line was cut in half at the right edge on the Mac (a Chinese comma or full stop measured half as wide on its own as within its line). Lines now wrap where they are drawn.
  • Importing a video from a link shows its real progress (percent, size, time left) on the page, in the footer and on the sidebar card, with the video's title; Stop on the card and Cancel in the footer really stop it (yt-dlp's helper process kept downloading before), and activity cards no longer show an empty bar.
  • A copy built on the Mac says, in Settings → About → Software updates, why it cannot update itself, with a link to the download page.
  • My voices: Delete… (and the other items) in a voice card's ⋯ menu did nothing — no dialog, no log line — unless the voice's editor was open: the menu went away with the pointer before the item chosen reached it.
  • Deleting a voice says where it is used (stories, books, dubs, workflows, the Studio, batch dubbing, calls, agents) and can be undone from its toast for about 10 seconds; its files wait in the library's trash until then. Places that named it show “Voice deleted” to choose another one, a story or book that uses it stops with a clear message, and the Studio says its voice went — never another voice in silence.
  • Quitting never hangs: edits not saved yet (drafts, the dub editor's changes) are written first, and jobs that do not stop when cancelled — such as a stuck download — are left after a few seconds (they continue or wait for Retry at the next start). An engine that stops reading its requests is restarted instead of blocking.
  • A job after an interrupted one (the engine stopped twice in a stage) no longer waits forever: it fails, saying why, the dub or book is no longer shown as busy, and Resume continues it. Chains stuck this way in an earlier version recover at the start.
  • The language model's turn on the GPU lasts until its request has really stopped, so the next job is not refused as busy.
  • Models moved by a move that did not finish (cancelled, a model in use by another app, the app quit) no longer show as not downloaded; the move continues after a restart.
  • Deleting a dub waits for its running stage to stop; two runs of the same dub started at once make one.
  • Stories and Audiobook: a rename on the Projects page no longer marks unsaved changes as saved; saving over a project changed elsewhere (by an agent through the Local API) asks whether to save a copy or replace it, and the Local API refuses an edit made on an outdated read.
  • Moving the data folder measures it and cleans up a failed copy in the background, with the dialog saying so, instead of freezing the window.
  • Smaller fixes: a crash in background work is logged instead of closing the app; a draft that could not be written is tried again; unfinished temporary files are cleaned in every folder; downloads left running by a crash are stopped at the next start.

Download: actnow.dev/download/audiosmith (a free account) · Changelog · or build it from this tag's source (AGPL-3.0).

Installed copies update themselves: AudioSmith → Check for Updates…

v0.4.0-beta.6

v0.4.0-beta.6 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 23 Oct 07:27

The window in the ActNow family's layout

  • The sidebar starts with a search field (⌘F): it searches projects, voices, takes, transcripts and exports, and the Projects page shows what it finds ("3 found", Clear search). ⌘K opens the command palette from the button in it.
  • Studio, Projects and History come first; Create (Stories, Audiobook, Dub, Transcribe), Voices and Automate are sections that fold. The ⌘1–⌘9 keys are as before.
  • No status bar: what goes on shows as cards at the foot of the sidebar — a stopped engine (Restart), dictation and calls under way (Stop, Hang up), each story, book, dub and workflow being made, takes, exports, transcriptions and downloads, several of a kind as "2/5", a finished one for 3 seconds. The pointer over them opens the stack; View all opens the new Tasks page, with every job that runs or waits.
  • Under the cards: Settings, the logs panel (⇧⌘L) and Tools, and the notifications' bell (a dot while some are unread).
  • Models and Integrations are Settings sections now (their links open them); Engine and Logs left the sidebar (the palette, the Go menu, Settings → Advanced → Diagnostics). ⌘E's quick switch is a small dialog with the engine's state and the model in use.
  • Settings: 17 sections become 11, each a column of groups with a line under every setting: General (with Appearance), Studio (with Pronunciation), Dubbing, Dictation (with Permissions), Writing help, Integrations (apps and agents, then the Local API), Audio tools, Models & network (the models, then downloads, storage, the Hugging Face token, memory and licences), Privacy, Advanced (storage, usage, diagnostics and the self-check) and About. Old links open the part they were.
  • Page titles are smaller; Projects and History have ‹ › (⌘[ ⌘]) and how many; Projects switches list and cards in its header (⌥1 ⌥2). Toasts show at the foot of the page, except on the Studio, Dub, Stories and Audiobook, whose foot has a bar of its own: there they show in the header.

Updates show in the window

  • MyGo's update window is gone: a new version shows as "Update available" at the foot of the sidebar (a dot on the icon rail's download button), and nothing pops up. Its panel shows the version's notes in AudioSmith's language from actnow.dev (in English where a version has no Chinese notes, marked so), Download and Install with the download's progress, and Restart to Finish the Update. A failed download says AudioSmith is unchanged and offers Try again.
  • Settings → About → Software updates: Check for Updates…, the daily check, installing without asking (a version found by the daily check installs and waits for the restart) and beta updates as before, and Show Changelog…, every version's notes in AudioSmith's language. The choices made in the old update window carry over.
  • Releases send their notes in English and Chinese (CHANGELOG.md and CHANGELOG.zh-Hans.md), so actnow.dev's changelog page and the app show each in its language.

Discover voices: about 35 chosen voices instead of 1,184

  • Discover voices now lists a small set chosen by listening tests' measurements (thread #91's audit): Chinese and English voices first, a few in Japanese, Korean, Spanish, French, German, Italian, Portuguese and Russian. The 1,126 generated combinations and the weaker hand-made voices are gone from the list.
  • Each designed voice has a written character (age, voice, situation) that the model really gets, a seed that was tried, and a reference that ships with the app: Use voice saves it ready to speak, with no reference to wait for. Preview plays a clip made from that reference, as the Studio would say it; nothing is rendered to listen.
  • The gallery also offers the built-in speakers worth starting from (Serena and Dylan for Chinese, Kokoro's best English and European voices): Use voice saves the speaker and opens the Studio.
  • Filters: use (narration and audiobooks, conversation and podcasts, characters and kids, ads and announcing), language, gender and age, and search. Pitch, accent or dialect, style and the nine categories are gone.
  • Voices saved from the earlier gallery keep working (they carry their own recipe), and dub casts or agents that name an earlier gallery voice or design preset still get it: those ids are kept, just not listed.

Design: eight starting points

  • Narrator, Conversational, Newsreader, Presenter, Storyteller, Elder, Villain and the new Soft (which replaces Whisper and Calm Guide). Upbeat, Kid and Northeast Buddy are gone (a Northeastern voice is in Discover voices). None asks for slow speech any more.

Fewer weak built-in speakers

  • Kokoro's af_nicole (a whisper) and bm_lewis (rough) are no longer offered, and Qwen3-TTS's Ryan is not offered in Chinese (he gave up on Chinese numbers in tests). A project or a setting that uses one keeps it, and the API still takes them by name.

Settings breathe, and say it shorter

  • Every Settings page follows one spacing: the groups' padding, room above and below each row with a line between rows, a group's note as a block of its own, and room between titles, lines and controls. Writing help's model picker no longer cuts its value, and the status line that repeated the rows is gone.
  • 28 texts are shorter, in English and Chinese.

Designed voices no longer say leftover reference text

  • Slow designed voices (Storyteller, Elder, Meditation guide, and a few gallery voices) were cut off half-way through their reference: Qwen3-TTS stopped at a length limit that slow speech reached (engine patch 0008 raises it from 0.48 s to 0.8 s per text token, and AudioSmith's own limit can now raise it as well as lower it). The reference was still saved with its whole text, so clones said the missing half before every line.
  • A reference made from a voice must now say its text: when the recognizer hears none of the three takes say it (or they were cut off), they are made once more; a kept take that says something else gets what it says as its text, or none is kept and the voice says why. Its text is never one the audio doesn't say.
  • References made before are heard once, the first time a take, story or book clones them: one that doesn't say its text is made again (a note says so) or gets the text it says.
  • Design prompts no longer ask for slow speech ("speaks slowly", "语速偏慢") from a preset or a picked pace: it made references slow and clones sound read. A description you write keeps it.
  • Voices designed for Japanese, Korean, German, French, Spanish, Italian, Portuguese or Russian say their reference in that language, not English.
  • Gallery previews use the tuned sampling the Studio uses (temperature 0.7, top-p 0.8 for Qwen3-TTS) instead of the engine's defaults; previews are made again once.
  • The Hindi gallery voice is hidden: no model here designs or clones Hindi.

Dictation's sounds to choose

  • Settings → Dictation → Sounds: the sound of listening starting, of it stopping and of an error are each chosen from the ActNow family's sound library (Tally, Wrap and Shrug by default), macOS's system sounds or none. A compact picker searches them (names and tags), filters by type, plays each with ▶ (again to stop) and sets the volume in steps; Try plays the choice.
  • The sounds are React Bits Sound Lab's and come with the builds from actnow.dev and GitHub releases only: they are not in this repository or under its licence (NOTICE). A build from source plays the system sounds AudioSmith used before (Tink, Pop, Basso).

Models move to the ActNow folder, out of Time Machine

  • Models are kept in /Library/Application Support/ActNow/models, shared with the other ActNow apps, instead of the Hugging Face cache (/.cache/huggingface), which cleaning tools treat as a cache. Models already there are brought over at the first start without downloading them again (hard links: the cache stays as it is, for the tools that use it).
  • Time Machine leaves models out (they can be downloaded again); Settings → Advanced → "Back up models with Time Machine" turns it back on for all ActNow apps.
  • A model file cut short (a disk error, a copy that stopped) is no longer taken for a downloaded model; downloading again repairs it. A download of another version left half-way no longer makes a downloaded model look incomplete. Checking a model's files waits for other ActNow apps working on it and tells them when a file was bad.
  • If the ActNow folder's model claims are deleted (a cleaning tool), AudioSmith claims its models again when it comes to the front, and keeps models another app may still use.

Audiobooks keep their rendered parts

  • The parts of an audiobook already rendered are kept in AudioSmith's data folder instead of ~/Library/Caches, which cleaning tools empty: Resume after a stop or a failure no longer renders hours of a book again. Those of earlier renders move over at the first start.
  • Temporary files of recordings and imports cut short are cleaned at start.

Keys stay out of logs

  • Google Translate (Basic)'s key is sent in a header, not in the address, so a network error no longer writes it into the log. Logs and the diagnostic bundle leave out keys, tokens and proxy passwords.

Dub editor: ⌘Z and ⇧⌘Z undo and redo again

  • In the Dub page's line editor ⌘Z did nothing on the Mac (the Edit menu's Undo took the key before the editor saw it), and with menu keys that have ⇧ fixed (MyGo patch 0009) ⇧⌘Z would have done nothing either. Edit → Undo and Redo, and their keys, now undo and redo the editor's changes whether a line, the toolbar or the timeline was clicked last; while a line's text is being edited they undo its typing, as before.
  • Menu items with ⇧ in their keys show and take exactly those keys (⇧⌘Z is Redo, not also ⌘Z).

The local writing model answers in full

  • The built-in writing model (Qwen3 4B) could cut its answers short or garble them when they had to follow a fixed form. The AI director left the last line of a story undirected; the emotion director could l...
Read more

v0.4.0-beta.5

v0.4.0-beta.5 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 19 Oct 14:12

AudioSmith joins the ActNow family: it works together with the other ActNow apps on this Mac (Dicta, Auralink, Pik, VoxtNew). The apps share one copy of each model, some settings, and, if you agree, API keys, and they don't get in each other's way. AudioSmith names another app only when a released version of it is installed; with no other ActNow app, nothing on screen changes.

Downloads and updates move to actnow.dev

  • AudioSmith now has its home on our website: actnow.dev/audiosmith. Download it from actnow.dev/download/audiosmith (free; the download needs a free account), and read every version's notes at actnow.dev/audiosmith/changelog. The source code stays on GitHub under the AGPL-3.0, and you can still build AudioSmith yourself.
  • Updates come from actnow.dev: the daily check reads the version file there, and with Receive beta updates on also the betas' one, and offers the newer version. Copies of 0.4.0-beta.4 and earlier update to this version from GitHub, one last time.
  • Settings → Diagnostics and about links to the website, the download page and the release notes.

Models shared with other ActNow apps

  • On first run, if another app (say Dicta) has already downloaded some of the models AudioSmith needs, the setup says so ("3 models are already on this Mac · Installed by Dicta · no download needed") and uses them. Only the rest is downloaded. Choose models… lets you pick them yourself on the Models page.
  • The Models page shows which models AudioSmith shares with other apps ("Shared with Dicta"), which ones another app has loaded right now ("Also loaded in Dicta"), and which ones another app installed ("Installed · from Dicta", with Use).
  • Deleting a model that another app also uses asks what to do. Remove from AudioSmith keeps the files on this Mac for that app. Delete for all apps deletes them; the other app then has to download the model again. Clean up and "Erase everything" keep what other apps use, and say which app uses it. A model another app deleted for all apps shows "Deleted in Dicta · Download again".
  • Settings → Storage shows how much space AudioSmith shares with other apps. It also offers to clean up models that only uninstalled apps used.
  • If you accepted a model's licence in another app, AudioSmith still shows the licence, says when you accepted it, and asks you to continue.
  • Kokoro's, Chatterbox's and Whisper's helper files now always load from the copy AudioSmith downloaded, at that version. Before, the engine could download them again on first use, or pick up another app's version.
  • Downloads coordinate with the other apps: a file another app is downloading is not fetched twice ("Downloading in Dicta…"), and a model is not deleted while another app downloads or uses it.
  • The engine no longer guesses which version of a model to load when several are on disk.

Settings shared with other ActNow apps

  • The interface language, proxy, download source and model folder are shared with the other ActNow apps. With another app installed, each of these settings says whether it applies to all ActNow apps, and you can keep a different one for AudioSmith only.
  • When another app keeps its models in a different folder, the Models page offers to move AudioSmith's models there. The move can be cancelled, and models the folder already has are not copied again.

API keys shared with other ActNow apps

  • API keys and the Hugging Face token can be shared with the other ActNow apps. With another app installed, a checkbox under the key ("Let other ActNow apps on this Mac use this key") chooses whether to share it.
  • AudioSmith offers a key saved in another app ("Use the DeepSeek key from Auralink"). It reads that key only after you click Use this key.
  • Removing a key that another app also uses asks whether to stop using it here or to delete it for all apps. If another app deletes a shared key, AudioSmith asks you for a key again. "Erase everything" never deletes another app's key.
  • The Hugging Face token is looked up in this order: $HF_TOKEN, the shared token, AudioSmith's own, the huggingface-cli token file.

Shortcuts and memory

  • If another ActNow app uses the same keys as the dictation shortcut, Settings → Dictation says so. The app that took the keys last shows the notice once, with Change shortcut, Open Auralink settings and Ignore. While you record a new shortcut, "Also used by Auralink (Transcribe)" warns you, but you can still save it.
  • The ActNow apps share one memory budget. Before AudioSmith loads a model, it unloads its own idle models first. If that is not enough, it asks the other apps to unload models that have been idle for a minute. It does the same when they ask, and never unloads a model a job is using.
  • Dictation's paste no longer overwrites the clipboard while another ActNow app is pasting.

Dub: conversations recorded together (multi-speaker dialogue)

  • A new switch in the Dub page's Cast step, 「多人对话配音」 (Multi-speaker dialogue): the people of a conversation are recorded together by the dialogue model "Dialogue · Light" (SoulX-Podcast), each line hearing the one before it, so answers sound like answers. Each line is then fitted to its own time as before (Smart fit and the other timings) and placed where it was said, so the video's pauses and overlaps stay. Monologues, one person speaking alone and lines over 4 s apart stay line by line.
  • On by default for new dubs into Chinese or English with two speakers or more; dubs made before keep their audio (off). Each speaker is cloned from about 12 s of their own lines, with what they say there, even from another language (an English or Japanese video dubbed into Chinese); a saved or designed voice from its own reference.
  • A line the dialogue model does not say clearly is made again (after the lines before it, another seed), then line by line; such a line is checked too, and marked "Listen" when it is not clear either. Editing a line records it alone again after the lines before it; changing a voice records the conversation again. The line list shows each conversation's lines with a coloured bar and a 「对话 N」 chip; each line's ⋯ menu can record the whole conversation again or keep a line out of it.
  • Greyed with the reason on Macs under 16 GB, with the 「逐句」 voice match, and for other languages (a Japanese track stays line by line; a dub into Chinese and Japanese uses it for Chinese only). Not downloaded yet: the switch offers the download (2.4 GB), and dubs are made line by line until it is there. The Generate step's model picker lists it too. Slower than line by line.
  • The translation of such a track aims a little shorter: the dialogue model speaks about 1.01–1.12× the estimate (learnt from the lines it makes).
  • Local API and MCP: create_dub, edit_dub and batch_dub take dialogue (auto or off) and dialogue_model (soulx), edit_dub a line's no_dialogue; get_dub returns dialogue, dialogue_blocked, dialogue_stretches and each line's dialogue, made_by (dialogue or single), fallback_reason and flag; run_dub with segments naming a conversation line records it again after the lines before it.

Audiobook: dialogue passages (多人对话段)

  • A scene in a book can now be spoken by a dialogue model, each line hearing the lines before it, so the characters answer each other. Mark it in the manuscript as a dialogue passage: [对话] on a line of its own ([dialogue] in English; 【对话】 works too), then one 名字:台词 (Name: line) line per turn, then [/对话]. A line without a name is narration, read by the narrator; a line in parentheses is a stage direction, not read; 名字(低声): gives a direction that is not read either; [pause 1s] on its own line lengthens the gap; [voice:NAME] lines work as well. The rest of the book stays on its model.
  • Insert → "Dialogue passage" writes an empty passage with the latest names of the manuscript (the first one selected, to type over), or makes the selected lines one, keeping every word: quotes become the turns of whoever the speech verb names (or, between two speakers, the other one), the words around them narration lines, and lines it cannot attribute are marked "?" for you to fix. Undo in the toast.
  • In the editor a passage has a faint tint and a dot per line, each speaker's name its own colour, stage directions grey, and a summary after [对话] ("3 人 · 5 句 · 多人对话 · 轻量"); names of the cast are offered as you start a line; the context menu has "Preview this dialogue" and "Turn back into paragraphs".
  • The inspector has a "Dialogue passages" section while the book has some: each passage with a preview button, the dialogue model, and what happens without it. Passage speakers are part of the cast (marked "dialogue"); a speaker nobody is cast as gets a voice of their own, unlike the others', when the book is generated (or at "Assign voices").
  • Without the dialogue model (not downloaded, or a Mac with less than 16 GB) the passages are read line by line in their voices, with the same tight conversational gaps; so are passages of one speaker and passages in languages other than Chinese and English. The dialogue model's lines are brought to the level of the chapter's narration. Chapter markers, the read-along and M4B export work as before; books without passages render exactly as before.
  • The Local API and MCP: update_longform takes the manuscript as script and dialogue/dialogue_model for audiobooks; get_longform returns script (up to 100 KB) and dialogue_blocks (chapter, index, speakers, lines, uncast speakers, mode dialogue or per_line and why); render_longform takes dialogue_block: {chapter, index} to preview one passage and lists the voices it gave as voices_given.

Dub: foreign-language videos dubbed into Chinese speak Chinese

  • Dubbing a Japanese or Korean video into Chinese with the default clone ...
Read more

v0.4.0-beta.4

v0.4.0-beta.4 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 16 Oct 15:12

Updates and downloads behind a proxy

  • Updating failed behind a proxy ("unexpected EOF" part way through the download): the updater did not use the system proxy. It now goes through the proxy of Settings → Models (automatic: the environment, then the macOS system proxy), like model downloads, and a download whose connection drops resumes where it stopped.
  • Importing a video from a link also uses that proxy: yt-dlp, started from the app, did not get it.

v0.4.0-beta.3

v0.4.0-beta.3 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 16 Oct 14:30

Dubbing

  • Importing a video from a link works again for YouTube videos offered only as separate video and audio streams (most of them now): AudioSmith gives yt-dlp the ffmpeg it uses (or one found on this Mac) to join them. Downloads stop at 1080p. Without any ffmpeg the error says to set it up in Settings → Audio tools, instead of "Requested format is not available".
  • When a step fails, the dub shows why that step failed, not "A job it depends on failed".

Diagnostics

  • Failed jobs (a dub step, a render, a transcription…) are written to the log, so Settings → Logs shows why.

Licence

  • The app ships THIRD-PARTY-NOTICES.txt with the full licence texts of the 45 components it includes; Settings → Diagnostics and about → Third-party licences opens it.

v0.4.0-beta.2

v0.4.0-beta.2 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 16 Oct 13:40

Voices

  • Hear a voice before you pick it: every voice list (Studio, Stories, Audiobook, Dub) has a ▶ on each voice, ■ to stop. Built-in speakers play a short sample that ships with the app, so they play before their model is downloaded; your own voices play the first seconds of their sample. One preview at a time; closing the list stops it. Keyboard: P plays the highlighted voice.
  • Stories, Audiobook and Dub use the same voice picker as the Studio.
  • Kokoro's Japanese voices are hidden, like its Chinese ones: our engine does not speak them intelligibly yet.

Licence

  • AudioSmith is now licensed under the GNU Affero General Public License, version 3 (AGPL-3.0); earlier releases, up to and including 0.4.0-beta.1, remain under the Apache License 2.0. Settings → Diagnostics and about shows the licence and links to the source code; the Local API's /, /health and capabilities document carry source_url and license. The app ships LICENSE and NOTICE.
  • Contributions need the Contributor License Agreement (CLA.md): a bot asks you to sign it on your first pull request, by a comment. See CONTRIBUTING.md.

v0.4.0-beta.1

v0.4.0-beta.1 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 16 Oct 13:22

The first signed release is a beta (a GitHub prerelease): copies that receive beta updates get the next beta, and 0.4.0 once it is out.

Updates

  • AudioSmith updates itself: once a day it reads the version file of its latest release on GitHub and, when a new version is out, shows its notes (this file's section of the version) and offers to install it, skip it or remind you later. Updates are signed; AudioSmith checks the signature before it installs one, and usually downloads only what changed. AudioSmith → Check for Updates…, the command palette and Settings → Diagnostics and about check at once; that page also turns the daily check and installing without asking on or off. Copies built on your own Mac don't update themselves.
  • Releases are signed with a Developer ID and notarized by Apple, so they open without Gatekeeper's warning (docs/release.md).
  • Beta updates: Settings → Diagnostics and about → Receive beta updates. Betas get new features sooner but may be less stable; turned off, AudioSmith only updates to releases. It is on in a beta copy (such as this one) and off in a release until you choose. Turning it off never goes back to an older version: a beta stays until a release newer than it is out. About shows a Beta badge next to a beta's version.
  • MyGo 0.5.5: MyGo's own texts (text fields' menus, date pickers, dialogs) follow the app's language, and Chinese text is drawn with the Chinese forms of its characters.

Transcription with speakers

  • New model "Transcription with speakers" (区分说话人转写; MOSS-Transcribe-Diarize 0.9B, 8-bit, 1.3 GB, Apache-2.0): it transcribes and tells who says what in one pass. Download it from the Models page, Transcribe or the Dub page; nothing downloads by itself.
  • Transcribe: a Speakers option in the toolbar. Transcripts list their speakers (S1, S2…) with their lines; click one to name it. Copy and TXT export put one paragraph per turn after the speaker's name, SRT cues start with it, VTT marks voices, JSON has each segment's speaker. Dictation and plain transcription stay on Qwen3-ASR (MOSS is about 4× slower: a 10-minute recording takes about 4 minutes on a MacBook Air M4).
  • Dubbing: with the model downloaded, a source with more than one speaker is transcribed by it and every line gets its speaker from it (in our tests 1.00 of the lines right on three of four 2–4 speaker dialogues and 0.93 on a 13-minute one with five people, where Qwen3-ASR + Sortformer managed 0.50–0.69 and never found more than 2 speakers; no limit of 4 speakers); the aligner still times the words, and the glossary's terms are its vocabulary. Without it, the dub works as before (Qwen3-ASR + Sortformer) and suggests the download, then Transcribe again.
  • Long recordings go in 10-minute windows that overlap by a minute, linked by who speaks in the overlap; stretches without speech are skipped, and the model's chat answers to silence ("I'm sorry, I can't assist…") are dropped.
  • Local API and MCP: speakers=true, model=gpt-4o-transcribe-diarize or response_format=diarized_json; segments carry speaker. MCP transcribe takes speakers: true.

Fewer transcription and writing models; Qwen3-ASR transcribes everything

  • Speech recognition is Qwen3-ASR 0.6B ("Chinese & English transcription") everywhere: dictation, Transcribe's Fast and Accurate modes, dubbing and its read-back check, workflows, Convert, clone references and calls. It made 0.5–4 % errors on Chinese where Whisper made 8–33 %, punctuates, and is 2–4× faster (thread #44's measurements). Whisper large-v3 turbo ("Multilingual transcription", tagged Fallback) is used on its own for the languages Qwen3-ASR does not know (it knows 30) and for translating speech into English. Fast and Accurate now differ only in the word timings.
  • Long files: the vocabulary is only sent with recordings up to 30 s (Qwen3-ASR copied it into the end of longer transcripts); the " The." (or "嗯。") Qwen3-ASR sometimes invents after the last sentence of a long file is dropped; files longer than about 4½ minutes are timed word by word in windows of at most 4 minutes, cut between sentences (the aligner's times broke down after about 4½ minutes).
  • Removed: Parakeet TDT 0.6B v3 (no Chinese) → Qwen3-ASR, or Whisper for a language Qwen3-ASR does not know; Mel-RoFormer (its output ended in 2–3.5 s of silence, and it was 5–9× slower) → HTDemucs; Gemma 4 E4B, Gemma 4 E2B and Qwen3 1.7B (slower or broken JSON, weaker translations and clean-up) → Qwen3 4B Instruct 2507, now the only language model, named in Settings → Writing help instead of a choice. Settings that named a removed model go back to the default once, with a notice; a model you chose that is still there (Whisper) stays. API and MCP: a transcription that names Parakeet is answered by its replacement with replaced_by and a warning (X-AudioSmith-Replaced-By / X-AudioSmith-Warning); GET /api/models/{id} describes the removed models, and they are not listed. Their files stay until Models → Clean up, which now also lists them (and shows on their tab).
  • Onboarding: Balanced no longer downloads Whisper; Max adds it instead of Mel-RoFormer.

Models named for what they do

  • Every model has a plain name that says what it is for — Standard voices, High-fidelity clone, Fast clone, Voice design, Fast English voices, English clone, Multilingual all-rounder; Chinese & English transcription, Writing assistant · Standard… (标准音色、高保真克隆、快速克隆…) — a one-line description and a few tags (Chinese, Cloning, Built-in voices, Low memory, Non-commercial…). Pickers, the Models page, ⌘E, Settings, the bar beside Generate, History, onboarding, notifications and messages use them.
  • The family, version, size, quantisation, repository and licence are still there: in a model card's Details on the Models page, in the pickers' tooltip and ⓘ, and in History's details (Technical name). Ids, repos and saved projects don't change.
  • The model picker and the Models page have a search field that finds a model by its name, its technical name, id, repo or a tag.
  • Local API and MCP: model lists add display_name (English), ui_name (as the app shows it), description and tags; a friendly name picks a model too.

Portable voices, and simpler Design, Clone and Export

  • Make a portable copy of a model's built-in speaker or of a designed voice (My voices → ⋯, the Studio's voice list, or Make portable where a voice can't be used): its model says a ten-second line, evened out, and the take is saved as a cloned voice " (approximate)" that every model that clones can speak with. It sounds close to the original, not the same, and says so.
  • When a voice can't be used with a model, the message says why and what to do (make a portable copy, or pick a model that designs).
  • Design shows only what the model can't express; what it will hear is behind ⓘ, and the seed moved into Details. Clone: style and language are under Advanced. Export audio: format, loudness, file name and extras in view; bit depth or bitrate, sample rate, channels and trimming under Advanced.

Simpler Stories, Audiobook and Dub

  • Stories and Audiobook: the inspector keeps Setup, Cast, Performance (Stories) or Chapters and Cover & details (Audiobook), and one Advanced section for the rest — language, merging and evening out voices, refinement and guidance, gaps, trimming, seed, a stem per character, a story's M4B with its cover and details, an audiobook's own pronunciations — with N changes and Restore recommended (your pacing stays).
  • The Project section moved to a Project ▾ menu in the header: Save (under the title — the title is the project's name), Save a copy, New, recent projects, All projects…
  • A story shows only what its model hears: personas, the scene, the AI director and line directions disappear on models that say lines as written, and a line's speed on models without one.
  • Dub: each step keeps its main choices; the rest folds into Advanced at the foot of the step (speakers and review, more languages and the translator's options, voice match, the voice model's parameters, a WAV's bit depth). The buttons the editor's ⋯ menu already had left the panels; Transcribe again… and Translate everything again joined that menu.

A simpler Studio

  • The inspector shows what most takes need: the model, one Voice list (the model's speakers, your saved voices, a clip to clone, Manage voices…) and the speed. Everything else — style, reference transcript, length, refinement, guidance, sampling, seed, reading numbers, evening out the volume — is under Advanced, filled with the model's recommended settings; its header says "Recommended settings" or how many you changed, with Restore recommended inside. Advanced stays open or closed as you left it.
  • The Voice controls and Audio quality buttons are gone, and so are the WAV precision (takes are kept at 24-bit; choose the file's bit depth in Export audio), Preload / Unload (in the ⌘E engine switcher, now with Unload), the autoplay switch (Settings → Studio) and the steps slider (three tiers remain).
  • A clip chosen to clone is transcribed for you when it is a WAV.
  • The bar beside Generate says which model and voice will speak, and how many settings you changed; a click shows the inspector.

Recommended models and settings

  • Every speech model was measured on a MacBook Air (intelligibility with two recognisers, naturalness, voice stability, speed, memory). Each page now recommends two or three models for its job and your language, with a one-line "best for" and a speed and quality hint; the others are one click away under "Show all models", and the ones that can't do the job (no cloning for a dub) are greyed out with the reason.
  • Tuned defaults: Qwen3-TTS uses temperature 0.7 and top-p 0.8 unless you (or an API request) set them — the models' own 0.9 / 1.0 sounded worse and less stable; OmniVoice uses guidance 1.5 with 32 steps (16 for dubbing). The Local API and MCP tools follow the same defaults. Render...
Read more