Ruach Studio 2.0.3

A point release with new rooms of work: sound to MIDI, the Videographer from its plan to a finished clip, a MIDI orchestra and an Ambient step in the Refiner, the Refiner's canvas closer to ComfyUI, adapters weighed against their model, and a check that a song is sung to its end (HERESY 1815–1963).

What changed

• Sound to MIDI. MuScriptor (Mirelo and Kyutai; its weights CC BY-NC 4.0, non-commercial use only) turns a take, a stem or any file of its tree into MIDI with a track for each instrument it hears. Its three sizes come from Engine → Models; the datasheet and the Refiner's MIDI step run it, and Open in the roll puts it into the Creator's score.
• The Videographer (beta). A take cut into shots at its sung lines; the clip's idea, look and each shot's scene written with the Writer; 🎨 Draw the shots has the Artist paint every shot without a picture, around one hero kept from shot to shot. 🎬 Shoot all sets each shot in motion with MiniMax H3 (a draft or the final, long shots made in pieces without a seam), and SeedVR2 makes them larger. An edit page in the manner of DaVinci Resolve, the cuts moved onto the beat, an animatic, and a clip with the sung lines burnt in, a title and credits, a cut, a dip to black, a flash or a dissolve between sections, in 16:9, 9:16 or 1:1; the plan as an editor's project (OTIO, EDL) for DaVinci Resolve or Kdenlive. Its four rooms (Editing, Shots, The idea and the look, Animatic and clip) fit one 1080p screen.
• The Refiner's canvas, closer to ComfyUI. Each node's window as large as what it holds; Ctrl+wheel from 50 to 200 %; Mute and Bypass with ComfyUI's keys (Ctrl+M, Ctrl+B); a bucket of one to ten takes run as a batch; the way kept as a profile in the studio's files, each flow with its own layout, Ctrl+S to keep it; the running node breathes, wears a bar with its time left and a ■ that stops the way where it stands.
• Adapters against their model. An adapter learns on the weights it is trained on: one of YuE2 tears on the RS-FW fine-tune, and one of RS-FW on YuE2. The Creator marks such an adapter ⊘, asks before it goes in and mutes it when the model changes, bringing it back when the model is its own again; the studio's trainer writes the base into every adapter it saves.
• The Refiner's MIDI orchestra. A stem played again by sampled instruments (sfizz for SFZ, FluidSynth for SF2, built from their sources), each note as strong as in the stem; the new version comes back as a strip in the DAW.
• Ambient. A Refiner step lays a fireplace, rain, thunder, a forest, a waterfall, a mountain river or birds under the song as a strip of its own: field recordings from a bank, looped without a seam, or synthesis for light rain and thunder; it breathes with the song and comes closer before the voice. The bank of recordings is not in this release: without it, only the synthesis plays.
• Sung to the end? Whisper times a take's lyrics line by line and says whether its last lines were sung. The datasheet names the lines that were not and offers Regenerate from the second the singing stopped; the Librarian checks many takes at once and marks them «end not sung».
• The Library. Reactions beside 🔥 and 💎, blind listening for a workspace, the player's stream in FLAC or Opus, a take's place kept where you left it, and a section never shown without its workspace.
• The Trainer's meter and guard: each card's VRAM peak at every step, a cap on the music half's input, the shortest tracks to the first card; its curves zoom.
• Smaller things: three more Colours in the Remaster (cassette, radio, hall), Rise and Fall for an adapter's wave, a take's earlier covers to put back from its datasheet, End at… finding the coda after the song, Engine → Studio performance, About naming the studio's own repository and its AGPL-3.0, the update's choice always in hand, and the player's, the console's and the Library's fixes.

Install / update

• New: INSTALL.md, or Pinokio on Linux.
• From 2.0.2 or earlier: use Engine → Updates in the page, or Pinokio's Update. The models you have stay. MuScriptor's sizes come from Engine → Models; MiniMax H3 and SeedVR2 from heresy/fetch-heresy.sh --h3 int8 and --upscaler 3b|7b|all, when you want them.

Known limits

MuScriptor runs on the processor (a song of four minutes: about a minute and a half small, eleven minutes large). MiniMax H3 int8 takes 69 GB on disk; a 24 GB card holds 4.4 s of a shot at 768p or 11.5 s at 480p at once, a longer shot is made in pieces. The Ambient step's bank of recordings comes later; the MIDI orchestra's instrument sets are fetched apart.