Skip to content

Kokon engine (1814814)

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 22 Oct 22:40

Kokon's stable-diffusion.cpp engine (1814814): upstream fd21ecf plus these commits:

  • 1814814 ci: publish the engines when a push to kokon passes every job
  • 33da65a ci: upstream base fd21ecf
  • aa613d6 Merge upstream master (fd21ecf) into kokon
  • dfaab76 ci: check the Windows CUDA engine's DLLs instead of starting it
  • cc182cf fix: give reused graphs ids without ggml's internal counter
  • c151a67 ci: build every Kokon engine from branch kokon
  • d3176a8 build: take ggml from Kokon's fork
  • 811a3d4 feat(server): add POST /sdcpp/v1/text_gen
  • 205a0a9 feat: add generate_text() to the C API
  • 1359009 feat: add a seeded sampler to LLM text generation
  • efdfefa feat: compute the LLM layers that do not fit the VRAM budget on the CPU
  • b807f0e build: use CUDA graphs only for reused graphs (ggml)
  • d985968 perf: reuse the LLM decode graph between steps
  • 806390a feat: release the vision tower's staged weights before text generation
  • e6576ff feat: move other models to RAM when text generation needs the VRAM
  • ece2540 feat: add a KV cache and greedy generation to LLMRunner
  • 126a195 feat: add an on-demand LM head and next-token logits to LLMRunner
  • 9bf8d9d feat: add byte-level BPE decoding and a streaming token decoder

Windows: CPU, CUDA 12.8 (CUDA graphs; runtime in cudart-sd-bin-win-cu12-x64.zip), Vulkan, ROCm 10.1.0.
Linux (Ubuntu 22.04 / glibc 2.35+): CPU, CUDA 12.8 (CUDA graphs; runtime in cudart-sd-bin-linux-cu12-x64.zip), Vulkan, ROCm 10.1.0.
macOS 15.0+: Metal, universal binary.