Feature Request: Support for the LLaDA2.0 model series of diffusion llms #16973
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
Inclusionai a few weeks ago has released two new open source text diffusion moe models:
16a1b
https://huggingface.co/inclusionAI/LLaDA2.0-mini-preview
100ba6b
https://huggingface.co/inclusionAI/LLaDA2.0-flash-preview
Motivation
The flash model is the biggest open source text diffusion model and I saw some diffusion models are already implemented #14644
I think diffusion models are cool and should in general benefit this project (;
Possible Implementation
It was implemented in chatllm.cpp:
Also as far as I understand, this model is based on the BailingMoeV2 arch which has support in llama.cpp already.