Skip to content

b10699

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 30 Aug 18:07
73f56d1

ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960)

some backends (Metal, SYCL, WebGPU) require additional memory for
fleeting data for certain ops, which is reflected in their
get_alloc_size implementations.

add ggml_backend_op_alloc_size_may_expand() to the backend utils,
listing these ops, and assert in ggml_backend_buft_get_alloc_size
that a backend expanding the alloc size of a compute op only does so
for ops listed in the helper.

use the helper in the RPC backend to decide whether to query the
remote server for the actual alloc size, instead of a hardcoded list.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: