Skip to content

b12101

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 04 Nov 08:32
960b1d5

ggml : write host-buffer tensor data directly in gguf writer (#31144)

The gguf writer staged each tensor into a temporary vector before
writing it. For host-buffer tensors the data pointer is directly
writable, so skip the staging copy and write it straight to the file.
This avoids the per-tensor allocation, zero-init and memcpy.

Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: