Repository navigation
ACE-Step 1.5 — Latent-Init Reference Audio with Duration Control (ComfyUI) #1366
Replies: 2 comments 1 reply
|
Keeping the reference timbre while changing lyrics sounds like a balancing act when the latent still carries the old timing. Glad that trade-off is called out here. |
|
2026-11-05 18:31:02 Warning! Sometimes the workflow produces blank audio files. I'm not sure why this is it may be an issue with VAE, GPU power, VRAM, etc. Changelog: *Changed the name of the workflow with version number to 'ACE-Step 1.5 — Latent-Init Reference Audio (ComfyUI) v0.01' *Tested each sampler. *Slot 2 Diff SFT *Band or Song Reference or description To Ollama updated for Pop music *Prompt to Ollama updated for Pop music *TextEncodeAceStepAudio1.5 cfg_scale set to 6.0 from 2.0 for lyrical adherence *Loop Reference (A + A) so duration can exceed the file length set to 'before' *Ollama Tags Ollama Connectivity keep_alive set to 5 from 0 *Lyrics Connectivity keep_alive set to 5 from 0 *Lyrics Connectivity, Ollama Tags Ollama Connectivity & Filename / Song Title Ollama Connectivity set to *Vocals using MDX (isolate singing) & EmptyAceStep1.5LatentAudio (UNUSED - bypassed by Latent-Init) disabled because they have no affect on latent reference ACE-Step 1.5 — Latent-Init Reference Audio (ComfyUI) v0.01.zip ACE-Step 1.5 — Latent-Init Reference Audio (ComfyUI) v0.01.json |
Uh oh!
There was an error while loading. Please reload this page.
ACE-Step 1.5 — Latent-Init Reference Audio with Duration Control (ComfyUI)
A ComfyUI workflow for ACE-Step 1.5 that generates a new song starting from a reference track's audio latent instead of from pure noise. The result keeps the reference's vocal timbre and instrumentation far more strongly than the standard "reference audio" conditioning does, while lyrics and tags still come from an LLM (Ollama) or from your own text.
This build also adds duration control: output length follows a "Length in seconds" value instead of being locked to the length of the reference file.
Workflow file:
Manny_ACE_DURATION_CONTROL.jsonHow it works
Standard approach (what this replaces)
The model starts from noise and is only nudged toward the reference. Timbre carries over weakly.
Latent-Init ("hard" conditioning)
The reference is encoded into the same latent space the sampler works in, and the sampler starts from that latent.
denoisesets how much of the reference survives:Duration control
VAEEncodeAudioencodes whatever audio it receives, so the latent length equals the file length. To decouple them:TrimAudioDurationcuts it to the requested length.durationand the latent length now come from the same value, so they always agree.Limit: maximum output length is 2× the reference file. Beyond that, chain another
AudioConcat. Past the original file length, the loop point is a hard splice; the sampler usually smooths it, but you may hear it.Workflow layout
VAEDecodeAudioTiled):LoadAudio→AudioConcat→TrimAudioDuration→VAEEncodeAudio→latent_imageon all four samplers.ACEStep15XLReferenceAudiofor its positive conditioning.SaveAudioMP3,SaveAudioWithTags,PreviewAudio, plus optional lyrics/data.txtexport.EmptyAceStep1.5LatentAudio(old noise-start latent) andAudioSeparateVocalsare left in the graph but not connected to any output. Unconnected nodes are not executed.Requirements
ComfyUI: a recent version with ACE-Step 1.5 support and the built-in audio nodes (
TrimAudioDuration,AudioConcat,VAEEncodeAudio).Custom node packs (install via ComfyUI-Manager):
showAnything,saveText, cache/GPU cleanupSaveAudioWithTagsACEStep15XLReferenceAudioandAudioSeparateVocalscarry no pack metadata in the file. If ComfyUI reports them missing, use Manager → Install Missing Custom Nodes.Models (place in the usual ComfyUI folders; the filenames below are what the workflow currently selects, so change them in the loader nodes to match yours):
ace_1.5_vae.safetensorsqwen_0.6b_ace15.safetensors,qwen_4b_ace15.safetensorsOllama: a running server (default
http://127.0.0.1:11434) with the model named in the connectivity node pulled. Not needed if you supply tags and lyrics manually.Usage
Manny_ACE_DURATION_CONTROL.jsonin ComfyUI.denoiseper sampler.Tuning notes
These come from the author's testing, not a guarantee. Results depend heavily on the track.
denoise.denoise.Known issues
ACEStep15XLReferenceAudio, lyrics can be skipped or sung out of order, sometimes worse than standard ACE-Step. The reference latent carries its own timing, which competes with the new lyrics. Things to try, one at a time:reference_strengthonACEStep15XLReferenceAudio.TextEncodeAceStepAudio1.5output to see whether the reference conditioning is the cause.ffmpeg -i input.mp3 -ar 44100 -ac 2 -c:a pcm_s16le output.wav.Credits
License
Add a license for the workflow file (e.g. MIT). Model weights and custom node packs have their own licenses; check them before commercial use.
Disclaimer
Only use reference audio you have the right to use. Generated output that imitates a real artist's voice may raise legal and ethical issues depending on where and how it is used.
ACE-Step 1.5 — Latent-Init Reference Audio with Duration Control (ComfyUI).zip
All reactions