A day of A/B testing the parameter surface (XL-SFT + 5Hz-LM 4B, MLX/Apple Silicon) — genre-conditioned harmonic bias, a keyscale fix, a docs contradiction, and one unresolved limitation

[gurubusvoyage]

Note: I ran the tests and made the calls on what sounded right/wrong; Claude (Anthropic's Claude Code) helped organize the session notes into this writeup and ran some of the code-level checks (e.g., confirming the use_adg gating and the shift doc contradiction directly in the model source).

Setup

• Hardware: Mac Studio, Apple Silicon, 36GB unified memory, MLX-native backend
• Config: acestep-v15-xl-sft (4B DiT) + acestep-5Hz-lm-4B, "best quality" preset
• Test material: a personal ~115 BPM Korean-language pop/rock-leaning track (original composition), used as reference_audio/src_audio across most tests, plus isolation tests with no reference audio at all
• Method: single-variable A/B changes against a fixed baseline, evaluated by ear by the composer/producer of the reference track across most runs; several findings cross-checked with 2-3 seeds to separate "this setting fixed it" from "this seed got lucky"

This was an iterative diagnostic session, not a controlled study — sample sizes per condition are small (often 1-3 runs). Treat these as practitioner observations to check against your own use case, not statistically validated claims.

Finding 1 — Genre-word choice in the caption strongly biases harmonic movement (EDM/house captions → static/looping chords)

Across several generations with an EDM/"dance house" style caption, we consistently got near-zero chord movement over a 3-4 minute duration (a static or barely-moving harmonic bed), regardless of audio_cover_strength (0.1-1.0), guidance_scale (5-9), seed (3 different seeds), or duration (80-240s).

Swapping the caption's genre language from "dance house" to "pop rock ballad" — with everything else identical (same lyrics, same seed, same bpm) — immediately produced normal chord movement.

We also confirmed a lighter-touch fix: keeping the EDM/dance genre language but adding explicit variation-descriptor phrases ("dynamic arrangement with evolving sections, varied and moving chord harmony throughout, building intensity from verse to chorus") also resolved the static-harmony symptom, without abandoning the genre.

This matches reports from other AI music tools (e.g., community writeups about Suno "getting stuck" looping 4-8 bars specifically for beat-driven/ambient genres) — likely a genre-conditioned bias from training data where EDM/house tracks legitimately do sit on static harmony more often, and the model over-applies that prior even when the user wants a fuller arrangement.

Takeaway: if you ask for EDM/house/dance-pop and get static harmony, try adding explicit "evolving/dynamic/varied chord" language to the caption before concluding the model can't do it.

Finding 2 — Cross-instrument chord-quality clashes (major chord in the backing, minor in the melody, same instant) — keyscale helps

Separately from Finding 1, we observed a different defect: at specific moments (notably instrumental-only intro/outro sections with no vocal), the backing accompaniment and the melody/other instrument voice would disagree on chord quality at the same instant (e.g., accompaniment plays a major triad while another voice plays the minor third against it).

This is not a relative-major/minor key ambiguity (i.e., "is this song in C major or A minor") — it's disagreement between simultaneously-generated instrument layers within the same short passage.

Setting keyscale explicitly (e.g., "G Minor") resolved this in our re-test: the same caption/lyrics/seed that previously produced the clash instead produced a fully diatonic, internally consistent chord progression (intro/verse cadences on the submediant bVI, final ending resolving cleanly on the tonic i — all correct for the specified key). The clash did not reappear.

Hypothesis: without an explicit key anchor, and without a vocal melody in that section to provide contextual grounding, the different generation pathways for each instrument voice have nothing forcing them toward the same target key at that instant — leaving it to (usually-correct-but-not-always) learned convention. keyscale gives an explicit, unambiguous target the whole texture can condition on.

This tracks with more general literature on diffusion-based polyphonic music generation, which typically has no principled mechanism enforcing harmonic consistency across simultaneously-generated voices — it's a learned tendency, not an enforced constraint.

Takeaway: if you see instruments disagreeing on chord quality (especially in sparse/instrumental sections), try setting keyscale explicitly rather than leaving it on auto-detect.

Residual issue: even with keyscale set, the vocal melody line itself still hit 2-3 out-of-scale notes in one 95-second clip. Smaller defect than the cross-instrument clash, not fully eliminated.

Finding 3 — Duration needs ~10-15s of buffer beyond your target length, or the ending gets cut mid-fade

Twice today (once with task_type="lego" adding a strings layer, once with a plain text2music generation), setting duration to exactly the length we wanted produced an ending that cut off abruptly — sounding like it was mid-fade-out when the audio just stopped, with no [Outro] resolution.

Both times, giving ~15s more than the target duration (and, for the lego case, adding an explicit "ending with a clear resolved outro" phrase to the caption) produced a natural, complete fade-out instead.

Takeaway: don't set duration to your exact target length if you want a real ending — pad it by 10-15s.

Finding 4 — use_cot_caption=True / use_cot_metas=True improved quality in our test, contrary to our prior assumption

We had been running with both forced to False for months, based on an earlier-diagnosed bug where the LM's CoT step would ignore the caption and generate an unrelated song when bpm/timesignature were left unset. Our pipeline always sets bpm/timesignature explicitly, so that specific trigger condition doesn't apply to us.

Turning use_cot_caption/use_cot_metas back on (LM refines the caption / fills metadata via CoT) produced noticeably better audio quality and harmonic flow in our test, with everything else held constant.

Takeaway: if you disabled CoT caption/metadata refinement to work around the "ignores caption" bug, check whether you're actually still hitting that trigger condition (missing bpm/timesignature) — if not, it may be safe (and beneficial) to re-enable it.

Finding 5 — use_adg=True works with xl-sft despite docs saying "base model only"

The parameter docstring (and INFERENCE.md) describe use_adg as "only works for base model." Checking the actual model source, use_adg is implemented identically in models/xl_base/ and models/xl_sft/ (both gated the same way, if not use_adg: ...), and only the turbo/xl_turbo variants explicitly warn-and-ignore it. We ran xl-sft with use_adg=True successfully (no errors, sensible output).

Takeaway: "base model only" in the docs appears to mean "not turbo," not literally "only the plain base checkpoint" — SFT/XL-SFT users may be skipping a usable quality knob based on the docstring wording.

Finding 6 — A real contradiction between two parts of INFERENCE.md regarding shift

The "For Best Quality" tips section recommends shift=3.0. The more detailed parameter reference table in the same document states the opposite: the documented defaults (guidance_scale=7.0, shift=1.0) target base/SFT models specifically, and shift=3.0 is the turbo-specific recommendation. These two sections disagree for base/SFT-family checkpoints.

We tested both on the same xl-sft config (ADG on, 64 steps, gs=7.0, 3 seeds each): neither value produced consistently better harmony. One seed with shift=3.0 sounded natural (with one wrong note in an intro guitar line); the other two seeds at shift=3.0 sounded worse than the shift=1.0 baseline. shift=1.0 on its own seed produced technically-correct-but-"forced-sounding" chord changes.

Takeaway: neither shift value reliably fixed harmony issues in our tests — the seed-to-seed variance dominated over the shift setting itself. Worth flagging the doc contradiction regardless; happy to open a docs PR if useful.

Finding 7 — retake_seed + low retake_variance is a genuinely useful "nearby exploration" tool, distinct from re-rolling to an unrelated seed

retake_variance does spherical interpolation between the original seed's noise and an independent retake_seed's noise (v=0 → unchanged, v=1 → equivalent to just using retake_seed directly), per the source comments.

Jumping to a completely different, unrelated seed (in our case, 2 different seeds at the same settings) reliably produced worse harmony than our best seed. Using retake_seed + retake_variance=0.3 on top of the good seed instead preserved the good seed's harmonic character while introducing a controlled variation.

Takeaway: if you've found a seed you like and want a variant rather than a gamble, retake_seed+low retake_variance (0.2-0.4) seems like a better tool than re-rolling the main seed.

Finding 8 — cover and extract task types underperformed text2music + reference_audio for our use case

We tried task_type="cover" (which skips the 5Hz LM entirely, per skip_lm_tasks) hoping the LM's independent chain-of-thought reasoning — which has no knowledge of the source audio's actual harmony — might be part of why text2music+reference_audio sometimes fought against the reference's own harmonic content. Instead, cover produced less clear vocals and lower overall audio quality than the equivalent text2music run, without fixing the harmonic-stagnation issue either.

task_type="extract" (stem isolation, also LM-skipped) similarly just degraded audio quality without cleanly isolating the target stem, in our one test.

Takeaway: for this use case (harmonically-aware remix/reinterpretation of an existing track), text2music + reference_audio + explicit bpm/keyscale outperformed the LM-skip task types (cover, extract) in our tests. If you need actual stem separation, a dedicated tool (Demucs, etc.) is likely more reliable than ACE-Step's extract.

Finding 9 — flow_edit_morph needs ~51GB and that ceiling doesn't move by freeing up other RAM

flow_edit_morph=True on text2music (the "preserve source structure, morph toward new caption" overlay) failed with RuntimeError: MPS backend out of memory (MPS allocated: 22.55 GiB, other allocations: 28.47 GiB, max allowed: 51.00 GiB).

The traceback shows this path runs through PyTorch's MPS backend (torch.nn / sdpa_attention_forward), not the MLX-native path used by regular generation — so it's paying full PyTorch-MPS memory cost on top of the already-loaded MLX models.

We retried after closing every other application to maximize free RAM (29.7GB available beforehand) — the failure reproduced with essentially identical numbers (22.68GB / 28.46GB). This rules out contention with other processes; the ~51GB ceiling appears to be a fixed watermark tied to total system RAM (36GB), not currently-available memory.

Takeaway: flow_edit_morph is likely infeasible on 36GB unified-memory Macs regardless of what else is running. Systems with more total RAM (64GB+) should be checked — we couldn't test that ourselves.

Finding 10 (unresolved) — Individual chords are diatonically correct, but cadential preparation is often missing, making resolutions sound abrupt/unearned

This is the one we couldn't fix with any parameter. Even once Finding 2 was resolved (all chords correct and in-key), the composer/producer testing this pointed out a more subtle problem: sections repeatedly start on the submediant (vi/bVI) rather than the tonic, so no tonal "home" is ever established before a tonic arrival happens — and when the tonic chord does arrive, there's no ritardando, rest, or other signal preparing the listener for it. The result is technically-correct harmony that still reads as "crammed in" rather than earned.

This matches (and gives a concrete musical mechanism for) issue #1114's report of "lack of gradual build-up toward key sections" and "energy transitions feel abrupt rather than intentional" — closed as not planned/stale. Our read: this isn't a parameter you can tune around; it's the model generating locally-plausible chords without phrase-level arrangement judgment (when to slow down, when to leave space, how to prepare an arrival). None of keyscale, shift, use_adg, lm_temperature, lm_cfg_scale, or duration padding addressed it in our tests.

We don't have a fix. Flagging it in case anyone working on the model/training side has thoughts, or in case this description helps someone else recognize the same symptom rather than assuming it's a caption-wording problem (it isn't, in our tests).

Related open issues this touches

• Option to specify a chord progression? #841 / More musical control #926 — chord progression control isn't supported via caption (confirmed, consistent with our results — no caption wording let us specify or reliably stabilize a progression)
• [Important] Technical Feedback on ACE-Step 1.5 XL (Music Quality & Expressiveness) #1114 — see Finding 10 above

Happy to share more detail/raw parameter dumps for any of these if useful, and can open docs PRs for Finding 5/6 if maintainers want them.

[gohan2091]

Interesting stuff. I can't say I understand a lot of it but that's because I'm not as experienced or knowledgeable as you. A couple of sections I'll experiment with. I use the xl srt model as I have a 4090 gpu. I tend to enjoy making remix / covers of existing tracks. I use a source audio and reference audio and get chatgpt to write the caption. I get the lyrics from a Google search. I'm experimenting with remix strength and cover strength. Do you have any tips or valuable information to share about that? I've recently trying to change the singers voice to that of the reference audio using a low below 0.15 Co ef strength.