DiffusionGemma by danielhanchen · Pull Request #24423 · ggml-org/llama.cpp

danielhanchen · 2026-06-10T15:56:37Z

Worked on prelim Diffusion Gemma support!

Has normal chat similar to llama-cli via llama-diffusion-cli -cnv -n 2048
Has a visualization method to show diffusion live via llama-diffusion-cli -cnv -n 2048 --diffusion-visual

To try this PR:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
gh pr checkout 24423
cmake -B build -DGGML_CUDA=ON
cmake --build build -j --config Release --target llama-diffusion-cli

then use a GGUF (any can work but for eg)

pip install -U "huggingface_hub[cli]"
hf download unsloth/diffusiongemma-26B-A4B-it-GGUF \
    --local-dir unsloth/diffusiongemma-26B-A4B-it-GGUF \
    --include "*Q8_0*" # Use "*Q4_K_M*" for a smaller 16 GB download

then use chat or visualization:

./build/bin/llama-diffusion-cli \
  -m unsloth/diffusiongemma-26B-A4B-it-GGUF/diffusiongemma-26B-A4B-it-Q8_0.gguf \
  -ngl 99 -cnv -n 2048

or

./build/bin/llama-diffusion-cli \
  -m unsloth/diffusiongemma-26B-A4B-it-GGUF/diffusiongemma-26B-A4B-it-Q8_0.gguf \
  -ngl 99 -cnv -n 2048 --diffusion-visual

Example below (a bit blurry to limit to 10MB on Github :()

Disclaimer Heavy usage of AI, but verified logits matching with transformers, checked FP16 vs FP32 KV cache, long context checks and much more

Some diffusion cli and visual updates

ggml-gh-bot · 2026-06-10T16:01:39Z

Hi @danielhanchen, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

Large PR: Large changes require prior discussion (e.g. an issue or RFC) and maintainers may not be able to review this PR as-is. Consider splitting it into smaller, focused PRs.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

pwilkin · 2026-06-10T16:38:47Z

Oof, that's a big one.

There's a ton of debugging stuff left in there that needs throwing out, for one. I'm also not convinced about the idea to make a server just for one model - I think if we're intending to support diffusion models in a server mechanism, it should be a general diffusion-server (but that's just my opinion, probably have to wait for what @ggerganov thinks about this one).

danielhanchen · 2026-06-10T16:42:31Z

Haha sorry - this PR was more of a direct translation / proof of concept that it works!

danielhanchen · 2026-06-10T16:42:44Z

I'll edit the PR - sorry we're juggling multiple things haha

gaugarg-nv · 2026-06-10T16:43:20Z

Another PR for DiffusionGemma: #24427

CISC · 2026-06-10T16:50:14Z

You have some failing tests to fix. :)

coder543 · 2026-06-10T16:50:37Z

if we're intending to support diffusion models in a server mechanism, it should be a general diffusion-server

With a block diffusion model, couldn't the regular server just return each block when it is finished diffusing? It would be nice to just have one server, and the API could remain fully compatible so clients don't need to be aware that they're dealing with a diffusion model. (We don't show the distribution of sampled logits for AR models, and I don't see why people would need to see the intermediate diffusion steps either, since those won't be useful.)

quasar-of-mikus · 2026-06-10T16:56:48Z

Doesn't build on Windows:

[421/433] Building CXX object examples\diffusion\CMakeFiles\llama-diffusion-cli.dir\diffusion-cli.cpp.obj
FAILED: examples/diffusion/CMakeFiles/llama-diffusion-cli.dir/diffusion-cli.cpp.obj
ccache C:\PROGRA~1\MICROS~3\2022\COMMUN~1\VC\Tools\Llvm\x64\bin\clang-cl.exe  /nologo -TP -DGGML_BACKEND_SHARED -DGGML_SHARED -DGGML_USE_CPU -DGGML_USE_CUDA -DLLAMA_SHARED -D_CRT_SECURE_NO_WARNINGS -IC:\Textgen\llama.cpp\src\..\include -IC:\Textgen\llama.cpp\ggml\src\..\include -IC:\Textgen\llama.cpp\common\. -IC:\Textgen\llama.cpp\common\..\vendor /DWIN32 /D_WINDOWS /EHsc /O2 /Ob2 /DNDEBUG -std:c++17 -MD /utf-8 /bigobj /showIncludes /Foexamples\diffusion\CMakeFiles\llama-diffusion-cli.dir\diffusion-cli.cpp.obj /Fdexamples\diffusion\CMakeFiles\llama-diffusion-cli.dir\ -c -- C:\Textgen\llama.cpp\examples\diffusion\diffusion-cli.cpp
C:\Textgen\llama.cpp\examples\diffusion\diffusion-cli.cpp(10,10): fatal error: 'sys/ioctl.h' file not found
   10 | #include <sys/ioctl.h>
      |          ^~~~~~~~~~~~~
1 error generated.

danielhanchen · 2026-06-10T16:57:01Z

Yep will fix haha - I also added a short GIF of it working edited in description!

…s, drop debug hooks - guard sys/ioctl.h behind _WIN32 and add a GetConsoleScreenBufferInfo fallback for the visual viewport size, so diffusion-cli builds on Windows - skip diffusion-gemma in test-llama-archs like gemma4 (shared ISWA backbone, no synthetic fixture params yet) - remove the DG_DUMP_KV_LAYER / DG_NSWA debug scaffolding and its llama.h API - fix flake8 E306 in conversion/diffusion_gemma.py

stepfunction83 · 2026-06-10T19:53:44Z

I was able to compile it successfully on Linux for my 4090, but when running it, I get the following error after sending a user message:

0.13.483.785 E ggml_cuda_compute_forward: SOFT_MAX failed
0.13.483.796 E CUDA error: invalid argument
0.13.483.798 E   current device: 0, in function ggml_cuda_compute_forward at /home/LLM/DiffusionGemma/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:3163
0.13.483.798 /home/LLM/DiffusionGemma/llama.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:103: CUDA error
E   err
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-base.so.0(+0x1c1ab)[0x7bb8567401ab]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-base.so.0(ggml_print_backtrace+0x21c)[0x7bb85674062c]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-base.so.0(ggml_abort+0x15b)[0x7bb85674080b]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-cuda.so.0(_Z15ggml_cuda_errorPKcS0_S0_iS0_+0xb7)[0x7bb853260997]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-cuda.so.0(+0x27a810)[0x7bb85327a810]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libggml-base.so.0(ggml_backend_sched_graph_compute_async+0x817)[0x7bb85675def7]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libllama.so.0(_ZN13llama_context13graph_computeEP11ggml_cgraphb+0xa1)[0x7bb855ee04c1]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libllama.so.0(_ZN13llama_context14process_ubatchERK12llama_ubatch14llm_graph_typeP22llama_memory_context_iR11ggml_status+0x114)[0x7bb855ee2c94]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libllama.so.0(_ZN13llama_context6encodeERK11llama_batch+0x240)[0x7bb855ee6c80]
/home/LLM/DiffusionGemma/llama.cpp/build/bin/libllama.so.0(llama_decode+0xf)[0x7bb855eec1ef]
./build/bin/llama-diffusion-cli(+0x1941c)[0x5a66e697341c]
./build/bin/llama-diffusion-cli(+0x8143)[0x5a66e6962143]
./build/bin/llama-diffusion-cli(+0x60e4)[0x5a66e69600e4]
/lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7bb85562a1ca]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7bb85562a28b]
./build/bin/llama-diffusion-cli(+0x6a45)[0x5a66e6960a45]
Aborted (core dumped)

The command I'm running is:

 CUDA_VISIBLE_DEVICES="" ./build/bin/llama-diffusion-cli \
  -m unsloth/diffusiongemma-26B-A4B-it-GGUF/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \
  -ngl 99 -cnv -n 2048 --system-prompt-file sysprompt.txt \
  --diffusion-eb auto \
  --diffusion-eb-max-steps 48 \
  --diffusion-eb-t-max 1.0 \
  --diffusion-eb-t-min 0.6 \
  --diffusion-visual

To compile it, I used:

rm -rf build
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_COMPILER=$(which nvcc)
cmake --build build -j --config Release --target llama-diffusion-cli

quasar-of-mikus · 2026-06-10T20:06:24Z

Yep will fix haha - I also added a short GIF of it working edited in description!

Builds and runs on Windows now.
1x 3090 Q4KM: time per step: 326.13ms
2x 3090 Q8_0: time per step: 878.83ms

arkham000 · 2026-06-10T20:10:32Z

is it only cli at this point? no llama-server ?

kroaton · 2026-06-10T20:16:25Z

Thank you for putting this together! An Issue I found is that --fit doesn't work with this PR.

icedream · 2026-06-10T21:54:09Z

Test run on my system with AMD hardware (7900 XTX) in it, Q4_K_M - time per step: 364.45ms

Screencast_20260610_234032_c.webm

lucasbinder · 2026-06-10T21:57:15Z

@icedream What (equivalent) tokens per second are you getting? I also tried running it with an AMD GPU (R9700 w/ vulkan) and only got ~27t/s.

icedream · 2026-06-10T22:24:59Z

@lucasbinder Not 100% sure if that's the right way to calculate it but based on two more runs with the same prompt, calculating with 256 tokens per full canvas diffused (I left out the last canvas as tail end of response), taking the start/end timings per canvas from the --verbose output:

Run 1

2.327213 - 9.803998 = 7.476785 = 34.24 t/s
10.119182 - 17.995429 = 7.876247 = 32.50 t/s
18.412472 - 23.738067 = 5.325595 = 48.07 t/s
24.261660 - 32.754977 = 8.493317 = 30.14 t/s
33.345178 - 51.575057 = 18.229879 = 14.04 t/s

Run 2

2.328457 - 9.777839 = 7.449382 = 34.37 t/s
10.092776 - 17.935576 = 7.8428 = 32.64 t/s
18.349154 - 23.654470 = 5.305316 = 48.25 t/s
24.176041 - 32.655410 = 8.479369 = 30.19 t/s
33.245244 - 51.415716 = 18.170472 = 14.09 t/s

(Also I should clarify I used ROCm, not Vulkan in my case so that may be influencing the performance as well.)

gbgh1 · 2026-06-11T01:49:10Z

Unofficial prebuilt binaries for anyone who wants to test this PR without setting up a CUDA toolchain:

https://github.com/gbuznote-beep/llama-diffusion-cli-prebuilt

Linux x86_64 / WSL2 — CUDA 12.8, sm_86 (RTX 30-series / A4000–A6000), glibc ≥ 2.39, self-contained (cudart/cublas/cublasLt/nccl bundled)
Windows x64 — CPU-only
Pinned to c84e85a; SHA256SUMS + reproducible build scripts included (other GPU archs rebuild in ~10 min)

Data points from testing (256 tokens, EB sampler): A5000 full-GPU 0.98 s/step; RTX 3070 Ti Laptop 8 GB via WSL2 (-ngl 99 --n-cpu-moe 22) 5.9 s/step; i7-12700H CPU-only 17.1 s/step. Output quality looks coherent (thinking-style drafts + self-critique). Thanks @danielhanchen for the implementation!

joey-zmw · 2026-06-17T08:21:07Z

./llama-server -m /data3/diffusiongemma-26B-A4B-it-Q4_K_M.gguf -t 96 --port 2604 --host 0.0.0.0
model server is OK in the clinet web
but it reports error when inferring the input content, while other models do not

Server Error
The server responded with an error message. Review the details below.
the current context does not logits computation. skipping

…anvas The visual server forced n_ubatch == n_ctx, so the whole prompt went through one non-causal encode. The O(prompt^2) attention overflows the 32-bit CUDA softcap index past ~12k tokens (n_head * N^2 > 2^31, a crash), and the encode also built an [n_tokens, n_vocab] fp32 logits buffer it then discarded. - llama-context: encode() honors cparams.n_outputs_max and reserves/copies only the flagged rows (no-op when n_outputs_max >= n_tokens). - diffusion-gemma: prefill the prompt in n_ubatch-sized causal chunks into a grow-only K/V store at an offset; off=0 is the single-shot prefill. - visual server: cap n_outputs_max to the canvas and size the prefill chunk so n_head * chunk * n_ctx stays under 2^31 (2048 up to ~32k, smaller past that). The per-turn compute buffer is now flat ~566 MiB regardless of context, output is byte-identical when the prompt fits one ubatch, and prompts to 60k+ tokens work where the single-shot encode crashed. Chunked-prefill approach from potto007.

aaronsb · 2026-06-18T22:39:54Z

I wasn't able to get this to work on ROCm until I made some changes to the ggml backend. in src/models/diffusion-gemma.cpp the function llama_diffusion_device_sample() the lines

ggml_backend_reg_t reg = ggml_backend_reg_by_name("CUDA");
if (!reg) { return false; }

does not register as CUDA because it fails to host fallback.

I added a basic fallthrough to catch ROCm:

  ggml_backend_reg_t reg = ggml_backend_reg_by_name("CUDA");
  if (!reg) { reg = ggml_backend_reg_by_name("ROCm"); }
  if (!reg) { reg = ggml_backend_reg_by_name("MUSA"); }
  if (!reg) { return false; }

Then the diffusion sampler worked.

potto007 · 2026-06-19T21:26:46Z

I tested it with the KV prompt store set to Q4 and I didn't see any degradation of output or reasoning - was able to run it with Cline with a context window size of 131k tokens. Fixed a bug where the CoT was being fed back to the model, which is explicitly stated in the Gemma family model cards as needing to be omitted. That was causing it to stop mid-turn -- after fixing that, I was able to have it successfully read the contents of one of my projects and reason about the code. Rough stopwatch timings so far, but it tends to run about 2x faster than my local llamacpp-run gemma-4-12b.

potto007@1b25994

Integriert DiffusionGemma (block text-diffusion MoE auf Gemma-4 Backbone) als monolithischen Port, da unser Fork die upstream llama_model_base- Hierarchie (Commit 994118a, 2026-05-04) noch nicht hat. Geaendert: - llama-arch.h/.cpp: LLM_ARCH_DIFFUSION_GEMMA, 5 neue Tensor-Typen, 7 Diffusion-KV-Keys (canvas_length, eb_*) - llama-model.h: Diffusion-spezifische Felder (canvas_length, sc_*, pkv_*) - llama-model.cpp: 4 Cases fuer load_hparams, load_tensors, build_graph, type_name Mapping - llama-model-loader.cpp: Template-Instanziierung get_key<int64_t> - include/llama.h: C-API Erweiterungen (diffusion_set_sc, set_phase) - models.h: Forward declaration llm_build_diffusion_gemma - gemma4-common.h: llm_graph_qkv Helper-Struct - diffusion-gemma.cpp: Monolithischer Graph-Builder (547 Zeilen) - tools/diffusion-cli: Minimales CLI-Tool fuer Tests - common/arg.cpp + common.h: CLI-Flags fuer Diffusion-Parameter - gguf-py: DiffusionGemma Arch/Tensor/KV-Mappings - tests/test-llama-archs.cpp: Arch in Skip-Listen eingetragen Neu (aus PR ggml-org#24423, unveraendert uebernommen): - conversion/diffusion_gemma.py: HF->GGUF Konvertierung - ggml-cuda/diffusion-sampling.cu: CUDA-Kernels fuer Diffusion-Sampling - pocs/: Dokumentation (Entscheidung, Masterplan, Status) Getestet: - Build erfolgreich (libllama.so, llama-server, llama-cli, llama-diffusion-cli) - Modell-Loading: Q4_K_M laeuft (25.25B Parameter) - Inferenz: Einzelner Forward-Pass mit greedy decode - GPU-Offloading: 8 Layers auf RTX 3070 Laptop (~2.5 t/s) Offen: - Echter Diffusion-Decoding-Loop (Prefill -> bidirektionaler Decode x N) - Entropy-bound Decoder - Visual Diffusion Mode - Upstream-Tools (diffusion-gemma-eval, -server) bauen nicht

Portiert aus PR ggml-org#24423 (Unsloth). Der Entropy-Bound Decoder ist der Kern-Algorithmus von DiffusionGemma: - Canvas mit zufaelligen Tokens initialisiert - Pro Step: Argmax, Entropie, Multinomial-Sample pro Position - Acceptance nach Entropie-Bound (niedrigste zuerst) - Renoising: nicht-akzeptierte Positionen -> frisches Rauschen - Output = Argmax-Canvas (nicht gesampelte Tokens) - Lineare Temperatur-Schedule (t_max -> t_min) - Konvergenz: Stabilitaet + Konfidenz Technisch funktionsfaehig. Output-Qualitaet verbesserbar durch: - Self-Conditioning (SC-Tensoren fehlen noch) - Mehr Steps (20 getestet, 48 empfohlen) - Chat-Template Prompt-Formatierung

Dark-Athena · 2026-06-22T08:27:15Z

    }

    diff_params.mask_token_id    = mask_token_id;
    diff_params.seed             = params.sampling.seed;


When the CLI doesn't pass a seed, the output will be exactly the same. Please refer to 'get_rng_seed'.

Dark-Athena · 2026-06-23T00:58:30Z

On my strix halo AI MAX 395(windows vulkan), it can only run at 10 tok/s because it frequently outputs "W ggml_vulkan: Failed to allocate pinned memory (Requested buffer size exceeds device buffer size limit: ErrorOutOfDeviceMemory)", even though the RAM and VRAM aren't even half used. I tried making some changes with GPT, forcing llama_set_embeddings(ctx, false);, and only extracting embeddings and logits when if (embd.data && t_embd && n_outputs_enc > 0), which boosted the speed to 18 tok/s, but I'm not sure if this change will affect the generation results.

…dates # Conflicts: # include/llama.h

The Stage-1 device sampler looked up the ggml-cuda backend by the literal name "CUDA", so on HIP and MUSA builds (registered as "ROCm"/"MUSA") the lookup failed and every step fell back to the host logits path. The backend exports the same ggml_backend_cuda_diffusion_sample proc address from shared source regardless of build, so probe ROCm and MUSA as well. Reported by aaronsb.

CapCeph · 2026-06-25T06:20:36Z

Hey everybody, I missed the rule of no generated output with my previous comment.

My bad, anyways I run a 9070xt. The gpu sampling switches back to the host when using Rocm cus line 665 only checks for cuda.

If you add lookups for rocm and Musa you can get it to work and It gave me around 11% of a performance improvement.

rhjdvsgsgks · 2026-07-02T11:57:16Z

https://diffrun.dev/status/

looks like self promoting a paid service https://diffrun.dev/production-guide/

pwilkin · 2026-07-03T09:05:56Z

@rhjdvsgsgks thanks, hidden.

Integriert DiffusionGemma (block text-diffusion MoE auf Gemma-4 Backbone) als monolithischen Port, da unser Fork die upstream llama_model_base- Hierarchie (Commit 994118a, 2026-05-04) noch nicht hat. Geaendert: - llama-arch.h/.cpp: LLM_ARCH_DIFFUSION_GEMMA, 5 neue Tensor-Typen, 7 Diffusion-KV-Keys (canvas_length, eb_*) - llama-model.h: Diffusion-spezifische Felder (canvas_length, sc_*, pkv_*) - llama-model.cpp: 4 Cases fuer load_hparams, load_tensors, build_graph, type_name Mapping - llama-model-loader.cpp: Template-Instanziierung get_key<int64_t> - include/llama.h: C-API Erweiterungen (diffusion_set_sc, set_phase) - models.h: Forward declaration llm_build_diffusion_gemma - gemma4-common.h: llm_graph_qkv Helper-Struct - diffusion-gemma.cpp: Monolithischer Graph-Builder (547 Zeilen) - tools/diffusion-cli: Minimales CLI-Tool fuer Tests - common/arg.cpp + common.h: CLI-Flags fuer Diffusion-Parameter - gguf-py: DiffusionGemma Arch/Tensor/KV-Mappings - tests/test-llama-archs.cpp: Arch in Skip-Listen eingetragen Neu (aus PR ggml-org#24423, unveraendert uebernommen): - conversion/diffusion_gemma.py: HF->GGUF Konvertierung - ggml-cuda/diffusion-sampling.cu: CUDA-Kernels fuer Diffusion-Sampling - pocs/: Dokumentation (Entscheidung, Masterplan, Status) Getestet: - Build erfolgreich (libllama.so, llama-server, llama-cli, llama-diffusion-cli) - Modell-Loading: Q4_K_M laeuft (25.25B Parameter) - Inferenz: Einzelner Forward-Pass mit greedy decode - GPU-Offloading: 8 Layers auf RTX 3070 Laptop (~2.5 t/s) Offen: - Echter Diffusion-Decoding-Loop (Prefill -> bidirektionaler Decode x N) - Entropy-bound Decoder - Visual Diffusion Mode - Upstream-Tools (diffusion-gemma-eval, -server) bauen nicht

Portiert aus PR ggml-org#24423 (Unsloth). Der Entropy-Bound Decoder ist der Kern-Algorithmus von DiffusionGemma: - Canvas mit zufaelligen Tokens initialisiert - Pro Step: Argmax, Entropie, Multinomial-Sample pro Position - Acceptance nach Entropie-Bound (niedrigste zuerst) - Renoising: nicht-akzeptierte Positionen -> frisches Rauschen - Output = Argmax-Canvas (nicht gesampelte Tokens) - Lineare Temperatur-Schedule (t_max -> t_min) - Konvergenz: Stabilitaet + Konfidenz Technisch funktionsfaehig. Output-Qualitaet verbesserbar durch: - Self-Conditioning (SC-Tensoren fehlen noch) - Mehr Steps (20 getestet, 48 empfohlen) - Chat-Template Prompt-Formatierung

Ninka-Rex · 2026-07-10T18:48:02Z

It's so over for the GPU poor

bhuv-z · 2026-07-11T18:13:44Z

Looks like all checks passed, when is this getting merged?

…— coherence Reconciled DiffusionGemma-26B against llama.cpp's from-scratch GGML `diffusion-gemma` graph (ggml-org/llama.cpp#24423), used as a version-independent fp oracle to sidestep the transformers/diffusers dev-version bug that makes the released Python reference emit garbage logits (256 '.' then NaN). llama.cpp generates COHERENT text from the same Unsloth Q8_0 checkpoint ("The capital of France is Paris.", 7 entropy-bound steps), proving the checkpoint is fine and the released libs were the whole problem. Root cause of fni8's incoherence: the forward flattened the model's REGION-AWARE encoder-decoder structure into a uniform decoder-only pass, wrong in 3 load-bearing ways (confirmed against the reference graph + the real HF weight names): 1. input embedding — prompt rows must be the RAW scaled embedding (encoder pass); only canvas rows get self-conditioning + weightless post-norm (decoder pass). Old code post-normed all rows. 2. per-layer scalar — prompt rows use the ENCODER scalar (`model.encoder.language_model.layers.{i}.layer_scalar`), canvas rows the decoder scalar. Old code applied the decoder scalar everywhere and never loaded the encoder scalar (the converter dropped it with `model.encoder.*`). 3. attention mask — prompt queries are causal-over-prompt-only (never attend the canvas); canvas queries are bidirectional over prompt+canvas (SWA-clipped on sliding layers). Old code was fully bidirectional, so the prompt attended the canvas — corrupting the very context the canvas denoises against. Fix: thread the canvas boundary (P = S - canvas_len) through the forward; apply region-aware embedding, per-layer scalar, and attention mask; retain the encoder per-layer scalar in the converter (promoted to `.enc_layer_scalar`). All changes are backward-compatible: with no split, or a pre-fix `.fni8` lacking the encoder scalar, behaviour falls back to the previous uniform path. Tests (real V100, int8 dp4a): 9 passed. New `test_diffusion_region_aware_prompt_ independent_of_canvas` asserts the encoder region is causal-independent of the canvas (0.0 delta) while the canvas depends on both — the old path leaked canvas into the prompt region (max delta 2.08e-2). Adds `tools/reconcile_diffusion_gemma.py` (the llama.cpp final-logit oracle harness) for the real-weight numeric gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019VoDhxVLE7T7sjiwHfv34M

diffusion-visual updates

c5fe75b

Some diffusion cli and visual updates

danielhanchen requested review from a team, CISC and am17an as code owners June 10, 2026 15:56

github-actions Bot added model Model specific examples python python script changes labels Jun 10, 2026

danielhanchen marked this pull request as draft June 10, 2026 15:58

danielhanchen changed the title ~~diffusion-visual updates~~ DiffusionGemma Jun 10, 2026

github-actions Bot added the testing Everything test related label Jun 10, 2026

danielhanchen mentioned this pull request Jun 10, 2026

diffusion_studio: serve DiffusionGemma in Unsloth Studio with the optimized visual decoder unslothai/unsloth-zoo#748

Merged

pearsonkyle mentioned this pull request Jun 10, 2026

Add DiffusionGemma (Gemma-4 MoE block-diffusion) support: MoE imatrix variants, AWQ MoE groups, llama.cpp bump, diffusion eval backend pearsonkyle/Quant-Tuner#4

Draft

github-actions Bot added the CUDA Related to the CUDA backend label Jun 17, 2026

danielhanchen force-pushed the diffusion-visual-updates branch from 3efe1fb to ef5e2dc Compare June 17, 2026 23:14

Dark-Athena reviewed Jun 22, 2026

View reviewed changes

malibio mentioned this pull request Jun 23, 2026

Integrate DiffusionGemma via llama.cpp when block-diffusion PRs land NodeSpaceAI/nodespace-core#1451

Open

6 tasks

danielhanchen added 2 commits June 23, 2026 12:56

Merge remote-tracking branch 'origin/master' into diffusion-visual-up…

1be9bcc

…dates # Conflicts: # include/llama.h

This comment was marked as spam.

Sign in to view

black-snow mentioned this pull request Jul 1, 2026

[Bug] diffusiongemma not OpenAI-v1 compatible unslothai/unsloth#6732

Closed

Fix merge conflicts

1d2faac

github-actions Bot added the conversion label Jul 3, 2026

Add tool calling

c3fb972

danielhanchen mentioned this pull request Jul 8, 2026

Bump DiffusionGemma #24423 pin to c3fb9724 (Add tool calling) unslothai/llama.cpp#32

Merged

This was referenced Jul 13, 2026

DiffusionGemma-26B-A4B: real uniform-state block-diffusion decode + arch reconcile jajmangold/fni8-serve#261

Open

[epic] New arches: DiffusionGemma + Gemma4/gemma3n jajmangold/fni8-serve#58

Closed

Uh oh!

Conversation

danielhanchen commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ggml-gh-bot Bot commented Jun 10, 2026

Uh oh!

pwilkin commented Jun 10, 2026

Uh oh!

danielhanchen commented Jun 10, 2026

Uh oh!

danielhanchen commented Jun 10, 2026

Uh oh!

gaugarg-nv commented Jun 10, 2026

Uh oh!

CISC commented Jun 10, 2026

Uh oh!

coder543 commented Jun 10, 2026

Uh oh!

quasar-of-mikus commented Jun 10, 2026

Uh oh!

danielhanchen commented Jun 10, 2026

Uh oh!

stepfunction83 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

quasar-of-mikus commented Jun 10, 2026

Uh oh!

arkham000 commented Jun 10, 2026

Uh oh!

kroaton commented Jun 10, 2026

Uh oh!

icedream commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lucasbinder commented Jun 10, 2026

Uh oh!

icedream commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gbgh1 commented Jun 11, 2026

Uh oh!

joey-zmw commented Jun 17, 2026

Uh oh!

aaronsb commented Jun 18, 2026

Uh oh!

potto007 commented Jun 19, 2026

Uh oh!

Dark-Athena Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Dark-Athena commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CapCeph commented Jun 25, 2026

Uh oh!

This comment was marked as spam.

This comment was marked as spam.

rhjdvsgsgks commented Jul 2, 2026

Uh oh!

pwilkin commented Jul 3, 2026

Uh oh!

Ninka-Rex commented Jul 10, 2026

Uh oh!

bhuv-z commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

danielhanchen commented Jun 10, 2026 •

edited

Loading

stepfunction83 commented Jun 10, 2026 •

edited

Loading

icedream commented Jun 10, 2026 •

edited

Loading

icedream commented Jun 10, 2026 •

edited

Loading

Dark-Athena commented Jun 23, 2026 •

edited

Loading