v1.10.0

• Where a plan sings (ASK Q16): the prompt carries no duration, so the planner alone decides the instrumental lead-in (0 to 57 bars on one 128 BPM request, median 18) and the song's length (26 to 220 bars); a % intro never carries the voice and [Intro] lyrics are sung in the next section. ABCScore.Layout (ABCScore.layout, ABCScore.layout(of:), also tolerant of a plan cut by the ABC token cap) gives the sections with their bars and sung notes, firstSungBar / firstSungSeconds, lastSungBar / lastSungEndSeconds, bars and nominal seconds; every plan.json now carries it as layout (SymbolicPlan.layout, recomputed for plans saved before; pipeline.py ignores the key).
• ABCScore.trimming(_:leadIn:tail:): keeps at most N bars before the voice's first group and after its last one by dropping whole V: Vocal/V: Ins groups as written (no bar re-spelled, groups with inline fields or uneven voices never dropped); render the result as an imposed score.
• CLI: yue2 score --abc <plan dir | file.abc> [--lead-in N] [--tail N] [--out short.abc] (no weights).
• One short phrase under several texts (ASK Q17 b): documented as outside the model's contract — an identical melody calls the same text, and re-spelling its notation changes nothing. Measured levers, safest first: one sung block, a different melody per text, plus lyrics-only guidance (--cfg-scale 1.5 --negative-style set to the request's own style; doubles the semantic phase). No code change.

v1.9.1

• Timeline order guaranteed (ASK Q14): a word the score would place at or before the previous one (lyrics sung past the end of a short score, no word-head anchor) now follows it off the score instead of jumping back to its note; a syllable never ends after the next one starts (a shared melisma note made them overlap on 6 of 8 songs). lines, words and syllables are in lyric order with increasing starts. Word-start accuracy unchanged on the eight measured songs (indie 87 → 89 %).
• LyricTimeline.wordsOnScore (share of words sung on the score's notes; under 0.9 the render did not follow its score) and LyricTimeline.fit(abc:lyrics:language:) (sung notes per lyric syllable of an imposed score, checked before generating: the model's own plans give 0.9–1.2).

v1.9.0

Song timeline (bars, notes and lyrics on the audio) and instrumental songs.

Timeline: timeline.json

A player can now follow a generated song from times the engine computes once: the start of every score bar and beat, every sung note, every line, word and syllable, with their musical position (bar, fractional beat). The times come from the LM itself: while it writes each 40 ms frame, four AR heads attend to the score bar being played (0.33 s ahead) and one to the next lyric word. A teacher-forced pass over 19 layers reads them back after the semantic phase, reusing the generation's KV cache (1–2 s on an M3 Max, identical in int4). No audio analysis, no extra model.

Measured on eight songs (four generated on an iPhone in int4; EN and FR, 87–132 BPM; reference: MMS forced alignment of the Demucs-separated voice): word starts 71–97 % within 300 ms, median error 59–80 ms; bar starts 60–160 ms (median) from a SheetSage2 transcription of the render. It holds where placing the score on a SheetSage2 beat grid fails (the voice leaving the planned melody: 10–12 % there).

let timeline = try AttentionTimeline.timeline(model: s.model, tokenizer: s.tokenizer, semantic: semantic)    // after generateSemantic
let timeline = try AttentionTimeline.reanalyze(song: directory, model: s.model, tokenizer: s.tokenizer)      // a song saved earlier
let now = timeline.position(at: player.currentTime)                                                         // bar, beat, note, line, word, syllable

yue2 generate --request song.json --reference 4bit-lean --timeline --out run/
yue2 karaoke --song a/ --song b/ --skip-existing      # migration, one model load

Songs cut to length keep only what is sung; words sung away from the score's notes are marked onScore: false. The pass waits on YuE2GPUGate per layer and honours cancellation. ABCScore reads the score dialect, including inline meter changes.

Instrumental: yue2 generate --instrumental

The upstream recipe (plan, move the Vocal melody to Ins, render with section tags and a "no vocals" style) plus a negative CFG branch on voice tags (SongRequest.negativeStyle, experimental) and a SheetSage2 voice check that re-renders on the next seed. 6 renders in 8 had no voice on the first seed.

Also: yue2 transcribe writes events.json as its help said; the SheetSage2 beat-grid clock no longer shifts bar 1 when no onset is heard.

Docs: API.md, CLI.md.

Validation: 210/210 tests green with weights, BUILD IOS OK.

🤖 Generated with Claude Code

v1.8.1

Fix: a prequantized pack downloaded into an empty models directory now loads on its own.

ModelDownloader.download(pack:) / yue2 download --model <pack> used to fetch only the weights, while ModelSession also reads YuE2-3B/config.json, yue2_generation_config.json and the tokenizer. On a device whose models directory held only the downloaded pack, loading failed on config.json (reported by the app team, ASK Q13).

These four small files are now published under support/ in VincentGOURBIN/yue2-mlx-packs: config.json and yue2_generation_config.json copied from m-a-p/YuE2-3B (CC BY-NC 4.0), plus tokenizer.json and tokenizer_config.json, the Qwen2.5 tokenizer converted for YuE2 (Apache 2.0). They are verified against support/SHA256SUMS. The downloader fetches any that are missing into YuE2-3B/ (ModelDownloader.downloadPackSupport()) and leaves existing ones alone.

yue2 download --model int4-mixed-head,vae   # pack + support files + VAE: a complete, loadable models directory

Checked end to end: an empty directory, yue2 download --model int4-mixed-head without a token, then yue2 plan loads and generates. PackSupportTests keeps it so (a directory with only the pack and the support files loads).

Validation: 188/188 tests green with weights, BUILD IOS OK.

🤖 Generated with Claude Code

v1.8.0

Three fixes asked by the app team.

Resume an interrupted "redo from here" (Q8)

Synthesizer.synthesize now accepts edit: together with resume:. For .keep ("redo from here"), the kept-frame mask is rebuilt from the seed's noise and the kept latents and re-imposed from the resumed step on; for .variation, the checkpoint's state supersedes the variation's start. The resumed solve is bit-identical to an uninterrupted one (tested for both edits), so leaving the app during a "redo from here" costs one ODE step instead of the whole stage (up to ≈ 5 min on a 60 s song on the iPhone).

Quiet hums (Q9)

MelodyTranscriber sizes its silence gate from the take: min(0.01, 0.1 × the 90th percentile of the frame RMS). A loud take keeps the former behaviour, and a quiet phone-microphone take (≈ −38 dBFS) is no longer half cut. Options.minimumRMS forces an absolute gate. Octave errors of the pitch tracker on low voices are folded back by melodic continuity (Options.octaveContinuity).

Follow the style's key and tempo (Q12)

In YuE2 the style line is text conditioning: the planner treats "key of D major" or "72 BPM" as hints and often departs from them (in the app's sample, 1 song in 7 kept the style's key). --follow-style-key (SongRequest.followingStyleKey()) reads the key and tempo from the style and forces the score header, through the existing forced-prefix mechanism. The planner then writes the song in that key and tempo. It is opt-in: checked for coherence (key of D major, 120 BPM → K:D, Q:1/4=120, D / Bm harmony), not yet by ear. When a score is imposed, put the score's key and tempo in the style, as the cover recipe does.

Validation: 187 weight-free tests green (new: bit-exact edited resume for .keep and .variation, quiet take, octave folding, style key parsing and header), BUILD IOS OK. Weight-backed suite run afterwards (2026-10-04): 187/187 green, covering 1.7.0, 1.7.1 and 1.8.0 (YuE2 and SheetSage2 parity, fp16/q8/q4 packs, multi-window songs, resumes).

🤖 Generated with Claude Code

v1.7.1

Fix: a transcription no longer crashes an iOS app that leaves the foreground.

iOS refuses GPU work from a backgrounded app, and MLX turns the refusal into an abort that cannot be caught. YuE2 already waits on YuE2GPUGate before each unit of GPU work; the SheetSage2 transcription did not, so leaving the app during a transcription crashed it (reported by the app team, ASK Q11).

SheetSage2Transcriber.checkpoint

Called before every unit of GPU work — the STFT or each of its chunks, each ConvNeXt block or chunk, each of the 24 Conformer blocks, the projection, each decoding step. Nothing is submitted to the GPU between two calls, so an app waits on its gate there; throwing from it cancels, and a cancelled Task stops at the same points.

var transcriber = SheetSage2Transcriber(model: model, profile: .named("16bit-lean")!)
transcriber.checkpoint = { progress in
    guard YuE2GPUGate.shared.wait(cancel: { userCancelled }) else { throw CancellationError() }
    Task { @MainActor in bar.progress = progress.overallFraction }
}

SheetSage2Progress carries the stage (encoding / decoding), the window, the encoder fraction and the token count, which gives the app a progress bar for the encoder too. The same hook exists on SheetSage2Model.encode(_:checkpoint:) and SheetSage2Generator.generate(…checkpoint:). SheetSage2Core still does not depend on YuE2Core: the app injects its gate.

Pausing changes no result: tokens identical with and without pauses (tested), and the real-weight gate is unchanged (tokens and ABC identical to upstream). As with YuE2, an eval already in flight at the very moment iOS withdraws the GPU can still abort; the lean profiles keep those units shortest.

Validation: 181 weight-free tests green (new: checkpoint before every encoder stage and decoding step, pause invariance, stop on error, Task cancellation), real-weight generation and notation gates green, BUILD IOS OK.

🤖 Generated with Claude Code

v1.7.0

Prequantized SheetSage2 packs: the 8-bit and 4-bit transcription profiles no longer quantize at load.

Packs on Hugging Face

Pack	Size	Profiles	Load (M3 Max)	Transcription peak, 30 s
sheetsage2-mlx-fp16	1.4 GB	16bit-fast, 16bit-lean	0.2 s, 1.4 GB	3.3 / 2.2 GB
sheetsage2-mlx-q8	0.9 GB	8bit-fast, 8bit-lean	0.19 s, 0.9 GB	2.8 / 1.7 GB
sheetsage2-mlx-q4	0.6 GB	4bit-fast, 4bit-lean	0.16 s, 0.6 GB	— / 1.4 GB

Before, a quantized profile loaded the fp16 model and quantized it at every load (0.33 s, 1.6-1.9 GB load footprint), and the lean profiles, which release the encoder after each transcription, paid it every time.

yue2 download --model sheetsage2-q8,sheetsage2-q4
yue2 transcribe --audio song.m4a --profile 8bit-lean --out run/score    # picks SheetSage2-q8 when present

The packs are bit-identical to the profiles' on-the-fly GPU quantization (MLX's CPU kernel rounds differently, so they are written with the GPU one): same scores, same transcription times. The quantized matmul does not speed the encoder up on 7 500 frames; the gain is memory and load time. Quantization still changes some scores against fp16 on rubato orchestral material: 16bit-lean stays the recommended iPhone profile, the quantized ones are there for tighter memory. Profiles table: docs/References.md.

API and CLI

• SheetSage2Model.load(directory:profile:) recognizes a pack (weights_format: mlx-quantized) and loads it as stored; a pack of another bit width than the profile's throws.
• SheetSage2Pack.save(_:bits:to:) and yue2 sheetsage2-pack --model <fp16 pack> --bits 8|4 --out <dir>.
• YuE2Model.sheetsage2Q8 / .sheetsage2Q4 for ModelDownloader (SHA-256 sidecar verified).
• yue2 transcribe --load-only prints load time and footprint without running the model.

All packs: CC BY-NC 4.0 derivatives of SheetSage2 and MERT-v2-FullSong (m-a-p), non-commercial use, attribution in each model card.

Validation: 176 weight-free tests green (new: pack round trip, packs identical to the on-the-fly quantization), anonymous download of both packs verified against the local files; BUILD IOS OK for YuE2Core and SheetSage2Core. The weight-backed suite was green on 1.6.0; this release adds code on the load path only and was checked by the pack tests, not by a full GPU run (the GPU was busy with other measurements).

🤖 Generated with Claude Code

v1.6.0

Transcription profiles, as for YuE2, and a ready-to-use SheetSage2 pack.

Profiles: yue2 transcribe --profile <bits>bit-fast|lean

Lean computes the STFT and the ConvNeXt by chunks (the global GRN normalization in two passes), encodes every window first and releases the encoder before decoding, and caps the MLX cache at 128 MB; in fp32 it gives exactly the same tokens and score.

M3 Max, release build:

Profile	30 s	500 s	Peak (30 s / 500 s)	Scores
16bit-fast (default)	3.0 s	20.9 s	3.3 / 3.6 GB	identical to fp32 on the parity excerpts
16bit-lean	3.2 s	22.1 s	2.2 / 2.4 GB	identical to 16bit-fast
8bit-lean	3.1 s	21.2 s	1.9 GB	pop identical, orchestral score changed
4bit-lean	3.1 s	21.3 s	1.6 GB	pop: one note added, orchestral changed

Recommended: 16bit-fast on a Mac, 16bit-lean on an iPhone. The quantized profiles (encoder only; the decoder is never quantized) are measured, not adopted. Full table: docs/References.md.

Merged fp16 pack on Hugging Face

VincentGOURBIN/sheetsage2-mlx-fp16: SheetSage2 and its MERT-v2-FullSong parent (m-a-p, CC BY-NC 4.0) merged and cast to fp16, ConvNeXt front kept fp32, 1.4 GB, SHA-256 verified. It gives the same tokens and scores as the upstream release merged at load, loads in 0.2 s and 1.4 GB, and is the default transcribe model when present.

yue2 download --model sheetsage2-fp16
yue2 transcribe --audio song.m4a --profile 16bit-lean --out run/score

Non-commercial use only, with attribution to MERT2 and SheetSage2 (m-a-p).

Correction

m-a-p/SheetSage2 and m-a-p/MERT-v2-FullSong are public, not gated: no Hugging Face token is needed (1.5.0 said otherwise).

Validation: 173 tests green with weights (new: fp32 parity of the lean mechanics, fp16 pack against the upstream release); BUILD IOS OK for YuE2Core and SheetSage2Core.

🤖 Generated with Claude Code

v1.5.0

SheetSage2 runs natively: a recording becomes an editable ABC score in Swift, on the GPU, without Python.

yue2 transcribe and SheetSage2Core

SheetSage2 (MERT-v2-FullSong encoder with LoRA adapters, BART decoder, grammar-constrained decoding, ABC notation) is ported to MLX Swift as a new library product, independent of YuE2Core. The score feeds a cover with cot: melody (--abc-file, or --abc-prefix-file so the planner writes the rest around it).

yue2 download --model sheetsage2                 # public repositories (CC BY-NC 4.0), no token needed — corrected in 1.6.0
yue2 transcribe --audio song.m4a --out run/score # score.abc (melody only; --chords for the full lead sheet)
yue2 generate --style "…" --lyrics "…" --cot melody --abc-file run/score/score.abc --out run/cover

The upstream release and its MERT parent are fetched at pinned revisions (MERT's main has moved past the revision SheetSage2 was trained against) and merged at load; no merged weights are redistributed.

Fidelity

Tokens and ABC are byte-identical to upstream in fp32 on two excerpts (orchestral, pop) and on a 500 s movement transcribed in four overlapping windows. fp16 (default) is identical on the excerpts, with the small ConvNeXt front kept in fp32 (its normalization overflows fp16); bf16 changes the score and is kept for comparison only. Greedy decoding is sensitive on long rubato pieces: there fp16 drifts from fp32, less than a mere change of input resampler does.

Cost (M3 Max, release build, fp16)

	Time	Footprint
13 s excerpt	2.4 s	3.3 GB peak
30 s excerpt	3.0 s	3.3 GB peak
500 s song (4 windows)	21 s	3.5 GB peak

Model loaded in 0.6 s, 1.6 GB resident. Upstream PyTorch on the same Mac (MPS, fp32): 7.9 s and 17 GB for the 30 s excerpt, and it collapses past ≈ 3 000 decoding steps. The iPhone integration is next (docs/iOS.md, Transcription).

Attached

beethoven5_opening_transcribed_swift.abc: the opening of Beethoven's Fifth (Musopen recording, public domain) transcribed by yue2 transcribe.

Validation: 170 tests green, weight-free and with weights (new: tiny parity, real encoder/decoder parity, generation, notation, adapter merge, whole-song windows); BUILD IOS OK for YuE2Core and SheetSage2Core, iOS Simulator build OK.

🤖 Generated with Claude Code

v1.4.0

Covers from a recording, the way the model is built for: through the score.

SheetSage2 → ABC → cot: melody

SheetSage2 (upstream, PyTorch, gated) transcribes a mixed recording into an ABC score in the model's dialect; it runs on Apple silicon through MPS in its own Python environment (recipe and two Mac workarounds in docs/CLI.md). yue2 generate --abc-file score.abc --cot melody then sings a song on that melody, accompaniment and style from the prompt. Measured on the M3 Max: a 13 s excerpt transcribed in 43 s including the model load, 16.5 GB peak footprint.

Forced beginning of score

generate --abc-prefix-file / SongRequest.abcPrefix (abc_prefix): the planner continues a given beginning (header with key and tempo, first sections) — a transcribed theme becomes the opening of a longer song, verses and chorus written around it in the same key and tempo.

Whistled melodies

yue2 melody --min-hz/--max-hz/--octave-shift: a whistle (two octaves above a voice, up to 2 kHz) is brought into the singing range automatically; short glides are absorbed into the note they lead to.

Tried and dropped

Keeping a real recording's VAE latents inside a generated song (inpainting, or anchoring it on the first ODE steps). The semantic audio tokenizer the model was trained with is not released, so the model never hears the excerpt: a paste-in at best. Not shipped.

Attached

• Beethoven, Symphony No. 5, 1st movement opening (Musopen recording, public domain): the 30 s source, its SheetSage2 transcription (.abc, the motif lands in the instrumental voice) and the same melody moved to the sung voice; three songs on it with cot: melody in bf16: orchestral (score as is, 30.8 s), symphonic metal (sung, 30.0 s), French synthwave (sung score as --abc-prefix-file, the planner adds a verse and a chorus built on the motif, 68.4 s).
• A real whistled melody, its transcription and the song generated on it, with a variation and a regeneration.

Validation: BUILD OK, BUILD IOS OK, 156 weight-free tests.

🤖 Generated with Claude Code