examples: refactor diffusion generation#22590
Conversation
There was a problem hiding this comment.
I'm not sure if stuff in examples is supposed to be create libs, but this refactor looks good to me from static code analysis. cc @CISC @ggerganov for approval
I guess the question is if this is mature enough to be moved out of |
I’ve actually built an API based on llama.cpp for diffusion models and this refactoring is just a small part of the work I’ve done If you support the idea of creating an experimental API for diffusion LLMs, I will finish the implementation and submit a PR for review |
ggerganov
left a comment
ggerganov
left a comment
There was a problem hiding this comment.
It's OK to have common functionality extracted into libraries in examples and reused. These libs can't be used directly by 3rd party projects. When the lib matures enough, it can potentially become part of the public API and be used by 3rd party projects.
|
|
||
| enum diffusion_algorithm { ORIGIN = 0, ENTROPY_BASED = 1, MARGIN_BASED = 2, RANDOM = 3, CONFIDENCE_BASED = 4 }; | ||
|
|
||
| // Unified transfer scheduling methods | ||
| enum transfer_schedule { | ||
| TIMESTEP_BASED = 0, // Dream-style: (1.0 - s/t) * remaining | ||
| BLOCK_BASED = 1, // LLaDA-style: process in blocks with get_num_transfer_tokens | ||
| }; | ||
|
|
There was a problem hiding this comment.
The enums should be named appropriately - see the CONTRIBUTING.md for guidelines.
There was a problem hiding this comment.
Done. Renamed following the guideline
4 participants
Overview
This PR refactors the diffusion example code by moving shared diffusion types and functions into diffusion.h and diffusion.cpp. The llama-diffusion-cli behavior is initially intended to stay unchanged. The CLI keeps the command-line handling, model/context setup, prompt handling, progress display, and output printing, while the reusable diffusion generation logic now lives in a separate module. This refactor separates the core diffusion generation logic into a reusable format, making it easier to integrate into new projects while maintaining full support for CLI usage
Additional information
No Python code or diffusion algorithm changes are included in this PR. This is intended to be a simple refactor preserving the initial behavior
Requirements