model: enable HIP backend for Vibe Voice TTS - #168
Merged
Conversation
Allow the VibeVoice session on the HIP backend and align its voice prompt cap with CUDA (30 s), and enable the conv-transpose1d col2im fast path for HIP. Validated on gfx1151 (ROCm 7.14, Linux): - prompt-side fingerprints (text encoding, acoustic encoder, prompt embeddings, prefill logits/top-5) match CPU/CUDA within ~1e-3 relative float noise - ggml test-backend-ops passes 11953/11953 on ROCm0 - residual end-to-end divergence vs CPU is the same chaotic argmax amplification that CUDA shows vs CPU, not a HIP-specific defect
IIIIIllllIIIIIlllll
changed the title
Comment on lines
47
to
53
| if (options.backend.type != engine::core::BackendType::Cpu && | ||
| options.backend.type != engine::core::BackendType::Cuda && | ||
| options.backend.type != engine::core::BackendType::Hip && | ||
| options.backend.type != engine::core::BackendType::Vulkan && | ||
| options.backend.type != engine::core::BackendType::Metal) { | ||
| throw std::runtime_error("VibeVoice session supports only CPU, CUDA, Vulkan, and Metal backends"); | ||
| throw std::runtime_error("VibeVoice session supports only CPU, CUDA, HIP, Vulkan, and Metal backends"); | ||
| } |
Contributor
There was a problem hiding this comment.
Given these are all the backends, can we just remove this check?
Contributor
Author
There was a problem hiding this comment.
Of course, that's how the original author wrote it, and I didn't change it.
Owner
|
@IIIIIllllIIIIIlllll Merged, thanks! |
mpulukkinen
pushed a commit
to mpulukkinen/audio.cpp
that referenced
this pull request
Aug 2, 2026
Allow the VibeVoice session on the HIP backend and align its voice prompt cap with CUDA (30 s), and enable the conv-transpose1d col2im fast path for HIP. Validated on gfx1151 (ROCm 7.14, Linux): - prompt-side fingerprints (text encoding, acoustic encoder, prompt embeddings, prefill logits/top-5) match CPU/CUDA within ~1e-3 relative float noise - ggml test-backend-ops passes 11953/11953 on ROCm0 - residual end-to-end divergence vs CPU is the same chaotic argmax amplification that CUDA shows vs CPU, not a HIP-specific defect
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Removes the ROCm/HIP restriction for VibeVoice TTS after validation on AMD hardware:
src/models/vibevoice/session.cpp: addBackendType::Hipto the sessionbackend whitelist; align the voice prompt duration cap with CUDA (30 s).
src/framework/modules/conv_modules.cpp: enable the conv-transpose1dcol2im fast path for HIP (used by the VibeVoice acoustic decoder).
docs/HIP.md: move VibeVoice TTS out of the "not yet enabled" list andrecord the validation results.
Validation (gfx1151 / Strix Halo iGPU, ROCm 7.14, Linux)
embeddings, prefill logits + top-5) match CPU and CUDA within ~1e-3
relative float noise.
test-backend-opson ROCm0: 11953 OK / 0 FAIL.Note on numerical parity: with identical inputs (same seed + injected
diffusion noise), HIP, CUDA, and CPU all agree for the first ~34
autoregressive steps, then diverge at a near-tie argmax step. Crucially,
CUDA diverges from CPU the same way (one step earlier than HIP), so
this is inherent bf16 summation-order noise amplified by autoregressive
generation — not a HIP-specific defect.
AI usage : I reviewed the VibeVoice logic, designed a workflow, and had Kimi K3 run tests separately on my servers (equipped with 2080TI, 3090, and GFX1151). I confirmed that there are no significant differences between the HIP and CUDA backends.
Output:
out_hip.wav