Auris Studio v0.5.0

Downloads

File	What it holds
…-macos-universal.tar.gz	Auris Studio.app, auris, auris-mcp, and auris-agent — Apple Silicon and Intel universal binaries
…-windows-x86_64.zip	auris-studio.exe, auris.exe, auris-mcp.exe, and auris-agent.exe
…-linux-x86_64.tar.gz	the auris command line tool only

Every archive carries MuseScore General, a General MIDI SoundFont under the MIT
licence, in a SoundFonts directory — which is most of the download size and all of the
instruments beyond the four built in. Beside it, a Dictionary directory holds the
NAIST Japanese dictionary (BSD-3-Clause, jpreprocess's build): what reads kanji
lyrics, and what gives a melody composed from lyrics the words' pitch accent to follow.
On macOS both live inside the bundle, so dragging Auris Studio.app to /Applications
takes them along.

None of it is code-signed. macOS will refuse the first launch and say the developer
cannot be verified: open it from the right-click menu once, or run
xattr -dr com.apple.quarantine "Auris Studio.app". Windows SmartScreen wants
More info → Run anyway.

Auris Studio is at 0.x, where nothing is stable: the project format, the configuration
files and every public API may change in any release, with no migration.

---

The voice trainer moved in

• training/ is now part of this repository. The Python project that trains the singing
  voices — PyTorch and Lightning, its own uv environment, its own documentation — was a
  repository of its own (uthree/auris-singer) and is now a directory here, history and all. It
  is not part of the Rust workspace, cargo never sees it, and no release archive carries it.
• The voice file format is checked across both languages. An exported .onnx is a contract
  between the trainer that writes it and auris-singer that reads it, and several halves of that
  contract were written down twice — the metadata key, the format version, the reserved symbols,
  the phoneme table down to which symbols are voiceless. Nothing could compare them while the two
  lived apart, to the point where a comment in auris-vocal asserted that its voiceless list
  matched the trainer's symbol for symbol: true when written and unverified ever after.
  training/tests/test_host_contract.py is that comment executable, and CI runs it on every
  change to either side.
• PyTorch is no longer pinned to a CUDA index. The trainer named the CUDA 12.8 wheels
  outright, which is right on a machine with a card and wrong on every other;
  uv pip install --torch-backend=auto reads the driver and chooses.
• A voice is measured through the host, not only through PyTorch. Training's validation
  sings in PyTorch and export's verification compares one runtime against another on random
  inputs; neither sees what auris-singer does with the file — the chunking of a long
  timeline, the arrangement of frames into tokens, the energy scale, its own noise, its own
  copy of the runtime — or what auris-vocal decided before it. training/scripts/evaluate_host.py
  sings the validation set's own curves through the application and reports the trainer's
  own metrics beside a PyTorch reference and beside the same utterances sung as one song, so
  a seam or an arrangement bug shows as a number; --score composes from lyrics and sings
  through auris sing, the path a person walks. Two CLI commands are the doors: auris frames writes what a singer track will be sung as, and auris sing-frames sings such a
  file through a voice into a WAV, with --report writing the session's account of the render
  — chunks, load and render time, which processor sang.
• The words are measured, not only the tune. The corpus run splits its spectral distance
  by the manner class of the phoneme on each frame — vowels, consonants, the sibilants on
  their own — and measures the sibilant tilt, the energy above 4 kHz against below on /s/
  frames, render against recording. It is the consonant-width study's measurement made a
  permanent column: a voice that tracks its pitch to the cent and hums through every /s/ now
  shows it as a number.
• A rest is a seam. The singer cut a timeline into inferences only where twenty seconds
  of frames forced it, so a four-line verse with two seconds of rest between the lines was
  one inference — and a voice is trained on phrases cut at their pauses, so a rest longer
  than a breath inside one inference is a place it has never been. Measured, the lines after
  the rests came out with phoneme error rates of 0.47 and 0.67 sung whole, 0.10 and 0.23 sung
  each in its own inference. Chunks now break at every rest longer than half a second as
  well, which costs nothing audible since the cut was in silence anyway.
• A voice says how loud its consonants are, and the frames listen. The energy a singer
  track's frames carried was the note's velocity on every phoneme, a plateau — and measured
  on JSUT-song a voiceless plosive or fricative sings twenty-odd decibels under the vowel
  after it, a voiced consonant six to nine, an approximant three. A /k/ at the vowel's level
  is a /k/ the model has never heard, and on the labelled corpus that plateau alone cost the
  phoneme error rate 0.25 → 0.56, more than the consonant widths and the note-flat pitch
  together. An export now carries phoneme_levels, measured by
  training/scripts/measure_phoneme_levels.py from a labelled dataset the way the widths are
  measured from labels; choosing the voice copies the table into the document beside the
  widths, and auris-vocal turns each consonant down by its measured decibels.
• The pitch travels between notes that touch. The frames a voice was fed held each
  note's pitch to its last frame and jumped on the first frame of the next, and no singer
  does that: measured on JSUT-song, across 1,568 note changes of a semitone or more, the pitch
  takes a median 60 ms to cross from one note to the next, starting some 20 ms before the
  next note's first phoneme and ending 50 ms after it. A model trained on that has never
  seen a step. auris-vocal now draws an 80 ms straight line centred on every boundary where
  one note ends exactly as the next begins, each half capped at a quarter of its note. It is
  a note change, not a portamento: a rest between the notes leaves each on its own pitch, and
  a fall or a scoop written on either note replaces it, since those are the slides somebody
  meant. The roll's sung pitch curve draws the same line. Through the host over ten takes the
  words came out at a phoneme error rate of 0.29 against the step's 0.30, inside the spread
  between takes; the line is there because every voice was trained on one, not for that
  number.
• A voice can start from another run's weights. training/scripts/train.py --init-from
  takes the generator and discriminator out of a checkpoint and nothing else — no optimizer
  state, no step count — so a voice can be pre-trained on a speaker's read speech and finished
  on their songs. configs/preprocess/jsut.yml prepares JSUT, the JSUT-song singer's 6.6 hours
  of prose, through the song's own preparation script; the consonants a 21-minute singing
  corpus holds a few hundred of, the speech holds thousands. Measured on the base voice,
  speech first and then the song took the corpus phoneme error rate from 0.20 to 0.12 against
  the recording's own 0.09, and a composed verse's from 0.31 to 0.23 — the stops and the
  flap, which the tally had named, most of all.
• The Namine Ritsu preparation writes its labels too. The database's hand-corrected mono
  labels were read for the phrase boundaries alone; prepare_namine_ritsu.py now writes them
  out as seconds per phoneme beside each phrase, and the preprocessing config names the
  folder, so the voice trains on the labels the way JSUT-song does.
• A batch is all-labelled or all-searched. Training takes the labelled durations only
  when every utterance in a batch has them, so a corpus mixing a labelled source with an
  unlabelled one — JSUT-song beside VocalSet — quietly sent most of the song's batches back
  to alignment search. The bucket sampler now keeps the two apart. VocalSet's clips carry
  labels of their own besides: a clip is silence, one vowel, silence, and the silence
  detection that cut it is its alignment.
• The consonant tables are the speaker's. A voice's consonant widths and levels were one
  table for the whole model; a model trained on several corpora now carries one of each per
  speaker, and choosing a speaker copies that speaker's into the document. The metadata format
  is 2 for it — a version-1 table has no speakers and would read as no table at all, so
  the host refuses a version-1 voice with a line saying to re-export it, and the exporter
  refuses a version-1 table. measure_phoneme_durations.py --data measures every speaker of
  a labelled dataset at once, as the level script always did.
• A voice's speakers can be chosen. A voice trained on several corpora carries one
  speaker per source, and the host always sang the first. A singer track now names the
  speaker who sings, saved with the document and part of what pins a take; Track → Next
  Speaker moves round the voice's list, auris sing --speaker and auris sing-frames --speaker take the name, the sing tool has a speaker argument, and
  evaluate_host.py --speaker measures one. A name the voice does not have is refused with
  the ones it does, at the command and never mid-render. train.py --init-from lets the
  speaker tensors start afresh, so a multi-speaker base can finish as a single voice.
• A consonant releases into its vowel. The level table turns each consonant down by
  its measured decibels, and the frames held that level to the consonant's last frame — a
  stop at its closure's level to the end, which is a stop that never bursts. Measured on the
  labels, a voiceless plosive sits 25 dB under its vowel through the closure and 8 dB under
  it over its last 20 ms, and every consonant class rises the same way. The last twenty
  milliseconds of a consonant are now sung at the vowel's level. Through the host over ten
  takes of the composed verse, /k/ was heard as /k/ 37 times in 40 instead of 25 and the
  phoneme error rate went 0.23 → 0.13.
• The listener says which phonemes it lost. The phoneme error rate was one number per
  column; the host evaluation now keeps the edit path behind it and tallies every phoneme
  asked for against what was heard, over every take of every utterance. The table ends with
  the worst phonemes — asked, heard right, dropped, heard as what — and the phonemes the
  listener inserted, and the report carries the counts per column.
• The host spells ざ and にゃ the way the voices were trained. Both of auris-vocal's
  front-ends — the kana table and the OpenJTalk map — wrote ざ行 as z and にゃ行 as nʲ,
  while the trainer's own OpenJTalk map writes dz and ɲ, so every voice was trained on
  those and the host sang ざ, ず, ぜ, ぞ, にゃ, にゅ, にょ through embeddings nothing had ever
  trained. Both symbols were in the phoneme table, which is all the contract checked; it now
  also checks that every symbol the host can emit is one the trainer's front-end writes.
• A corpus's labels align the training, where it has them. Training recovered every
  phoneme-to-frame alignment by monotonic alignment search, labels or no labels, and
  measured against JSUT-song's labels the search gives ɕ two thirds of its frames and ts
  under three fifths, one ɕ in three no more than two frames, and puts a boundary
  100–170 ms off on average — a consonant's worth. A preprocessing source can now name
  a duration_dir of labelled seconds per phoneme; the preprocessor stores them as frames,
  training expands the phonemes by them and skips the search (data.use_durations), the
  host evaluation measures every voice on the same labelled alignment, and
  scripts/compare_alignment.py prints the search against the labels by class.
  scripts/prepare_jsut_song.py writes the durations beside the transcripts.
• And whether the words can be heard. --asr transcribes every render with a
  recogniser in the voice's language — ReazonSpeech for Japanese, the asr extra — turns
  the transcript back into IPA through the trainer's own front-end and reports the phoneme
  error rate against what was asked for, with the recording's own rate beside it as the
  ceiling a speech recogniser reaches on song. Another language is one class registered
  under its code.

The words write the tune

• Singing synthesis can take the GPU. Settings → General chooses where a voice model
  runs its inference: Auto (the default) sings on the platform's own GPU provider —
  DirectML on Windows, Core ML on macOS — wherever the runtime offers one, and falls back
  to the CPU where it does not, including mid-render: a provider that accepts the session
  and then refuses its shapes (DirectML does, to this model family, today) demotes the
  voice to the CPU and the render finishes. Forcing GPU makes those refusals visible
  errors instead; CPU opts out entirely. The choice applies from the next render — no
  relaunch.

• Consonants take the width their voice measured. A newer auris-singer export carries
  per-phoneme consonant durations measured from its training data (phoneme_durations in
  the model's metadata — consonant length in sung Japanese spans a factor of three by
  phoneme class, so one fixed width mistimed half the inventory). Choosing the voice copies
  the table into the document beside its name, and everything that lays phonemes out reads
  it: the sung frames, the drawn segmentation, the divider grab, the dragged-note preview.
  Old exports, and tracks with no voice, keep the fixed sixty milliseconds. A table in a
  unit this build cannot read is refused outright rather than misread by two orders of
  magnitude.

• Compose from lyrics, the Orpheus way. compose_lyrics at both model doors — and
  Session::compose_from_lyrics beneath it, for every frontend to come — takes Japanese
  lyrics and writes a song: phrases cut where a singer breathes, one note per mora, and a
  melody found by dynamic programming under the constraint Orpheus made famous: the tune
  must not contradict the lyric's spoken pitch accent. Chords are stamped visibly into the
  harmony lane first (王道進行 by default; a harmony already written is left alone), the
  standard band comes along behind on its usual recipes, and every note lands carrying its
  mora and phonemes, ready for sing. Kana lyrics compose without any dictionary — the
  accent constraint simply has nothing to say, and the answer says so — while a configured
  Japanese dictionary reads each phrase's accent nucleus and makes the melody follow the
  words. The same lyrics and seed write the same song.

• Under the hood, three seams left open on purpose: Contour (rise / fall / no-fall /
  free) lives in auris-core and names no language, so another prosody — or a learned
  model — can produce it; the accent analysis in auris-vocal reads jpreprocess's own
  accent phrases, from the same dictionary run that already yields phonemes; and the pitch
  search in auris-compose is one function with rhythm assignment as its own stage, so a
  richer rhythm scheme or a trained melody engine can stand beside either without touching
  the rest.

• In the window: File → Compose from Lyrics… — also findable in the command palette,
  with an empty keystroke slot for whoever wants one. Type the words — Return breaks a
  phrase, secondary-Return composes — and the piano roll opens on the song; every run draws
  a fresh seed and the status bar names it,
  so a take can be asked for again at any of the model doors. The refusals speak through
  the same field: empty words, or kanji with no dictionary anywhere, say exactly that.

• Lyrics join the song sheet, right on the sheet. The sheet's third column is now the
  words themselves: one multi-line box per section, in the order the form plays them, no
  button and no popup between the writer and the verse. Click a box and it is a real editor
  in place — Return breaks a phrase, Tab walks to the next section, a click lands the caret,
  the IME composes where the text is — and every keystroke is already on the sheet's dials;
  Escape just puts the keyboard down. The margin measures the words as they are typed: a
  note count per line (one note per mora), and in each box's heading the bars the sung
  rhythm needs against the bars the section has — red once the words would outrun it, and
  computed by the very reading and rhythm Write uses (Session::measure_lyrics), so the
  display cannot drift from what happens. The parts moved beneath to make the room: a strip
  of cards, two abreast, scrolling past its height. The .asong format grows the matching
  lyrics = "..." field, round-tripped like every other, shown in the annotated reference
  example. Writing the piece adds a Vocal track beside the band, one clip per playing of
  each lyrical section, the melody searched over that section's own harmony; every playing
  of a chorus sings the same chorus. Lyrics nobody can read cost their sections and are
  named in the report, never the piece. File → Compose from Lyrics… grew the same multi-line
  field: Return breaks a line there too, and secondary-Return composes.

• A composed vocal arrives ornamented, by rule. The first note of each phrase scoops
  in, notes held past half a second sway (the vibrato waits out the front of the note and
  fades in), and the line's last note falls away — the phrase-final syllable is now held a
  half note, which is both what a sung phrase does and the room the sway needs. The
  ornaments are the ordinary scoop / fall / vibrato fields, visible on the pitch curve and
  editable one by one; the rules are a starting point, never a verdict.

• Singer voices join the library, on the instruments' own terms. The browser grows a
  Voices section: every .onnx in a Voices folder — beside the binaries, in the
  configuration directory, or under AURIS_VOICES — plus any folder registered from the
  section's Add Voice Folder… row (remembered, never copied, exactly like plugin
  folders). One click puts a voice on the selected singer track the way one click puts a
  sound on an instrument track; the search box finds voices by name beside everything
  else; Track → Choose Voice… keeps the file dialog for the one-off file somewhere
  unusual.

• The Japanese dictionary now ships with a release. naist-jdic (jpreprocess's build,
  BSD-3-Clause) travels in a Dictionary directory beside the SoundFonts — inside the
  bundle on macOS — fetched at packaging time by tools/fetch-dictionary.sh against a
  digest recorded once, in auris_session::library, where the fonts' manifest already
  lives (auris dictionary --manifest prints it, auris dictionary says what is
  installed). Kanji lyrics read out of the box, and a melody composed from lyrics follows
  their accent out of the box. The settings entry becomes an override: choose a folder
  to swap in your own build, Clear to return to the shipped one, AURIS_DICTIONARY to
  point the search elsewhere entirely. The shipped archive is v0.14.0's — jpreprocess's
  v0.15.0 release ships no standalone dictionary — and the accent test proves it loads
  under the 0.15 crate rather than assuming so.