Skip to content

feat: add module backend assignment support#1500

Merged
leejet merged 1 commit into
masterfrom
split-backend
May 16, 2026
Merged

feat: add module backend assignment support#1500
leejet merged 1 commit into
masterfrom
split-backend

Conversation

@leejet
Copy link
Copy Markdown
Owner

@leejet leejet commented May 16, 2026

Summary

  • Add --backend and --params-backend module assignment support.
    • Supports module assignments such as te=cpu,vae=cuda0,diffusion=vulkan0.
  • Add SDBackendManager to own and share backend instances by resolved backend name.
  • Pass runtime and parameter backends into GGMLRunner externally.
  • Split ggml_extend_backend.hpp into ggml_extend_backend.h/.cpp.
  • Document backend selection behavior in docs/backend.md.

Related Issue / Discussion

Since #1184, there have been multiple PRs containing backend selection related changes. However, none of them achieved the behavior I expected, and the implementations were intertwined with other unrelated changes. I decided to write a standalone implementation myself that better matches my design goals, while also adding support for parameter-based backend selection.

This PR was inspired by the work in stduphf’s PR. When merging this PR, I will add @stduhpf as a co-author.

Additional Information

  • Ran smoke tests with several model pipelines

Checklist

@leejet leejet merged commit 3633072 into master May 16, 2026
12 checks passed
@leejet leejet deleted the split-backend branch May 16, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant