You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
YuE2 Studio 4.1.0 — the Ruach Studio fine-tune and decoder, songs one by one, Portuguese, export
Ruach Studio's fine-tune of YuE2 and their YuE2-VAE-RS x3 decoder, in one "Ruach Studio" preset (Settings - Models): a fine-tune of the whole model - by its authors' account the first large public one of YuE2 - sings in place of the set's model and their decoder turns the song into sound; one button sets both and downloads what is missing. The fine-tune, RS-TW 2.0×0.75 ("Tsunami Wave"), is not a style LoRA over the base: both halves of the model were taught again, the one that writes the music and the one that makes the sound, on 109,250 tracks (about 6,650 hours, two thirds of them instrumental, the sung part in English, Russian, Hebrew, Greek, Ukrainian and eighteen more languages), on top of a first wave of 32,633 tracks. Its authors chose this strength in blind listening as the one that holds a song together from the first bar to the last, and in their measurements it leaves half the stray tones of the stock model (2.15 a take against 4.33). Their style vocabulary, 2,920 words the fine-tune was taught, is on its page. Two things to know before trying it: a LoRA trained on the stock model does not carry over to the fine-tune, and its authors measured a few more misheard words than with the stock model (23.0% against 20.0%). It comes in Q8_0 and Q6_K under CC BY-NC-SA 4.0. The decoder stands beside the VAE, which keeps its other half: a recording given to the VAE is still read by the VAE's own encoder, which the decoder's file does not have. "Decoder" in the model picker offers the v1 and v2 releases and the VAE's own decoder on any set, without the fine-tune. Against the VAE's own decoder it keeps the level of the 10-16 kHz band of the source (the VAE's own loses 1.6-1.9 dB there) and adds nothing above 20 kHz. It is heavier, so the sets keep the VAE's own decoder unless you choose otherwise: it takes 4 GB of the card while it decodes, where the VAE's own takes 2.5 GB, a minute of sound decodes in about a second on an RTX 4090, twice the time, and the file is 0.8 GB. It was checked against its authors' reference on CUDA and on the processor. When the card is short of room the engine decodes in smaller tiles, and it now leaves room for what other programs hold on the card. models_preset does the preset for an agent. The catalog entries are from pull request #250, the engine's reader from yue2.cpp pull request #2, both by Ruach Studio.
Songs of one request come one by one: each goes into the library, and into the playlist the request names, as soon as the engine has finished it, and can be played while the rest are still made; the request keeps a card for each song it has yet to make, and a finished song takes the row of its card. A request stopped on the way keeps the songs it finished. song_job_get shows the songs so far.
The studio in Portuguese (translation by DJ José Silva): Português is the sixth language of the window, chosen in Settings and taken by itself on a computer set to Portuguese. The whole window, the training guide, the reasons a song failed and the forms for reporting a problem on GitHub are in Portuguese of Portugal; the writing assistant answers in it.
Export, in a song's menu (by jaredlaham): a song is written out as a new file - WAV in 16, 24 bits or 32-bit float, FLAC in 16 or 24, MP3 from 128 to 320 kbps, at its own rate or at 44.1, 48, 88.2 or 96 kHz - and can be brought to a loudness target on the way: -14 LUFS for streaming, -16 for Apple Music and podcasts, -23 for broadcast, or your own. Loudness is measured to ITU-R BS.1770 and set by one gain held under a true-peak ceiling; there is no limiter, and the report says when the ceiling stopped the gain short. Dither is added only when bits are taken away. The report shows the loudness and true peak before and after and everything done to the sound; the song in the library is not touched. An agent does the same with song_export. The same window lists the song's MIDI, each file named for where its notes come from: the score the song was generated with - whole, or its lead, voice line or chords alone, at the score's tempo, meter and key with its section markers - and the track's own MIDI, transcribed or edited. And it makes the song an Ableton Live 12 project: a set with the song on a track that plays, its stems on tracks that are switched off and its MIDI on MIDI tracks, at the score's tempo and meter, zipped with the files beside it and a read-me (song_export_live for an agent).
The models can live on another disk (after DjagbleyEmmanuel's idea): the setup screen and Settings - Models show where the downloads are kept and how much room that disk has, and "Move to another folder" takes the model sets, LoRAs, the stem separator, karaoke, the assistant's models, the trainer and Audio to MIDI there. A link stays where each folder was, so the studio and the other programs that share its models go on working; on the same disk the move is instant, to another disk the files are copied and checked before the old copy goes. Songs, datasets and training runs never move. "Back into the studio's folder" undoes it; storage_get and storage_set do the same for an agent.
Six more community models: Ruach Studio's IRA and IRAE merges: IRA carries Mothersuperior's Instrumental and Real Audio adapters in its weights; IRAE, IRAE v2 and v3 add monsterovich's Epic Trailer at three strengths; IRAE-IR and IRAE-IR v2 add his Industrial Rock and are experimental, for rock instrumentals. Settings - Models - Your own YuE2 model lists them in Q8_0 and Q6_K, pinned and checked by SHA-256 like the others.
A stem mixer in Tools (by jaredlaham): a song's stems play together on one transport over the song's waveform, each with mute, solo, volume and pan, and a loop over a section set in seconds or from the playhead - to hear the drums with the bass alone, or the band without the voice. What is set there is for listening; the stem files stay as separated.
Two more visualizers, "Black hole" and "Tunnel" (by telesoho): a black hole with its disk of light, bent as gravity bends it, brightening and tilting with the song; and a flight through a tunnel that jumps on the loud notes, with a threshold of its own. Both sit beside MilkDrop and the spectrum, in the panel and in the visualizer's window.
The studio starts without a window for an agent, and can stay in the tray: --headless starts it with no window on screen, only a tray icon, and it stops by itself after ten minutes with no call from an agent and nothing at work; --stop ends a running studio whose window is hidden and that is not working; "Keep running in the system tray when the window is closed" in Settings hides the window in place of quitting; a second launch brings up the studio that is running and no longer opens a second window on the same service. From pull request #251 by x3vyx.
Fixed
The engine starts on computers whose processor has AVX-512 that the system keeps switched off (#237, #215): the engine chose its processor library by what the processor reports alone, and on such a computer the first AVX-512 instruction closed it at start with 0xc000001d. It now asks the system as well and takes the AVX2 library there.
A dataset whose lyrics are in Cyrillic, Chinese or any script but Latin is prepared for training with its words: HOT-Step's preparation took such lyrics for an instrumental. The training tools, the captioner and the VST host are built from HOT-Step 75923e70 now; a studio that has the earlier tools is offered the new ones once, on the update screen.
A failed "Describe by ear" leaves its whole run in the studio's log (#204): how the captioner was started, every line it wrote and how it ended go to logs/yue2-studio.log, which a problem report carries; before, a captioner that fell without a word and without a Windows record left nothing to go by.
A recording longer than about four and a half minutes is transcribed into a score (#259, found and explained by anatomix59-bit): the engine refused a request body over 100 MiB, which the 32-bit WAV of such a track exceeds, and the studio said "413 Payload Too Large". The engine now takes a body of 2 GiB.
An engine that cannot start is no longer started again every two seconds: the starts are 5, 15, 30 and 60 seconds apart, then five minutes apart, and at once when the engine settings or the chosen models change; the log says it once and no longer fills with the same failure. From pull request #256 by AIwork4me.
"Keep models loaded: Auto" leaves the card its headroom (#238, #235): Auto kept the models on a card they only just fitted, and whatever reached the card after the start - the desktop, a browser - pushed the next song into the system's memory, where it runs many times slower and may run out. Auto now keeps the models only where 2.5 GB stay free beside them: a 12 GB card loads them for each song, as before 4.0.0, a 16 GB card keeps the Q8 set and not the BF16 one, a 24 GB card keeps either. The switch saved by a version before 4.0.0 reads as Auto, not as Yes.
On Linux the engine is no longer killed in the middle of a song (#246, found and explained by BitOBSessiOn): the engine was tied to the life of the thread that started it, not of the studio, and died with SIGKILL when that thread was retired - about one song in three or four. The engine and the writing assistant's llama-server now start from a thread that lives as long as the studio. From pull request #248 by BitOBSessiOn.
A song carried on past what the KV cache holds is carried on as far as it holds: "+N seconds" on a song near the end of the cache was refused with "the prompt and the score take ... so a song can last ... s". The studio now makes it again with the seconds that fit, and the engine log says how many.
A model file counts as one download on Hugging Face, not as hundreds (#249, reported by igrbible): a download was cut into 16 MB pieces, about 230 requests for a 3.8 GB model, and Hugging Face counts each request for a file as a download of it. The studio now asks Hugging Face for a file once and takes its pieces from the address that answer leads to, asking again only when that address runs out (about an hour). A file is fetched in one piece a connection, four at most; a dropped connection asks only for the rest of its piece, and a download closed halfway goes on from the last 16 MB it noted. A download left by an earlier version starts over. The single request is from pull request #250 by igrbible.
A name that came with an example, a reused song or the assistant's draft goes with that one song (#252): it stayed in the Create form across songs and restarts, so every next song got the same title whatever its words. A name typed by hand stays until it is changed.
A download started again after its unfinished file was removed arrives whole: the notes of how far a download had come could stay beside a removed file (an assistant model's did), the next download of the same file trusted them, skipped those bytes and ended with "its SHA-256 differs from the pinned". A download trusts the notes only beside the file they were written for, and removing an assistant model takes them away.
A hub popup is closed for good by any answer: a button or a link inside it closes it as the cross does, and it does not come back.
With LoRA, a score that cannot be sung is written again more times: five instead of two before the song fails with "the score the model wrote cannot be sung".
An engine that exits at its start says why: the error, and the anonymous report of it, carry the engine's own FATAL line.
"The assistant is unreachable" says where its address and model are set (#239): Settings - Models - Writing assistant.
The studio says when the engine ended up on the processor: with a graphics card in the computer whose CUDA or Vulkan did not start, Auto went on to the processor and the window still read "engine reachable" while songs took tens of minutes. The status line now says the engine computes on the processor, and Settings - Local engine says where the reason is.
A card smaller than the smallest set is told so before the first song: the setup screen and Settings - Local engine say how much video memory the card has and how much the smallest set needs, instead of leaving it to an out-of-memory error.
Restarting the engine under a song stops that song, and is not counted as a failure: the song says the engine was restarted from the studio, not that it "stopped and restarted".
Songs a request: a count the form kept from a time the engine took more is cut to what it takes now instead of refused (#244).
"Describe by ear", training, the writing assistant and VST effects no longer depend on the Visual C++ runtime Windows has (#204): the programs the studio downloads for them came without the runtime they were built with, and with an older one in Windows they stopped at their start with 0xc0000005 and not a word - "the captioner wrote nothing for this song". The studio now puts its own runtime beside every program it starts.
The "Load MIDI" window opens over the whole studio (#240): it was drawn inside the create form, cut by the form's edge with its button under the player.
The buttons over a score are readable in the dark theme (#236): the sheet is white in both themes, and the dark theme drew them pale grey on it.
The count beside a search and on the "All songs" tab counts songs: "4 of 224" and the tab counted each stem of a separated song as a song of its own, while the heading above the list said 190; all three now count songs.
The create form can be made wider (#243): up to 70% of the window instead of 40%, with the song list keeping its 280 pixels.
The equalizer, visualizer and Winamp mode keep their sound on Linux (by DjagbleyEmmanuel): a Web Audio context that the system interrupted - WebKitGTK calls the state "interrupted", where the studio only knew "suspended" - stayed silent while the song played on, and only a restart brought the sound back. The player's context, Winamp's and the visualizer window's are now resumed whenever they stop running.
A video export no longer hangs (by Sheng Gao): the export waited without end for a video encoder that had stopped taking frames, and for a background clip asked for the frame it already showed. A stalled encoder is now given 15 seconds, after which the export starts over with the software encoder, and the progress moves with every few frames.
"Edit audio" opens inside the studio (by DjagbleyEmmanuel): the AudioMass editor was handed to the system browser; it now opens in a window of the studio, like the visualizer, on the file asked for, and a second edit reuses that window. A plain browser still opens it in a tab.