Skip to content

Conversation

@ixgbe
Copy link
Contributor

@ixgbe ixgbe commented Nov 24, 2025

This PR introduces the CPU features detection for the RISC-V platform and allows for dynamic backend loading when compiled with -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON.

1、Build this PR using:

cmake -B build -DLLAMA_CURL=OFF -DCMAKE_BUILD_TYPE=Release -DGGML_OPENMP=OFF -DLLAMA_BUILD_EXAMPLES=ON -DLLAMA_BUILD_TOOLS=ON -DLLAMA_BUILD_TESTS=ON -DGGML_RV_ZICBOP=OFF -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON
cmake --build build --config Release -j $(nproc)

2、Check that there are 2 libggml-cpu*.so files built:

 ls -la build/bin | grep libggml-cpu-
 -rwxr-xr-x  1 root root  499184 11月21日 16:59 libggml-cpu-riscv64_0.so
-rwxr-xr-x  1 root root  552544 11月21日 17:00 libggml-cpu-riscv64_v.so

3、Run a test prompt and let me know which library is loaded via:
build/bin/llama-cli -m Qwen3-0.6B-Q4_K_M.gguf -no-cnv --seed 42 -n 50 -p "Write me a dog walking business idea 1. " 2>&1 | less

Help me paste the first few outputs from the top. It should print something like this at the top and it should run the prompt completely without problems.

load_backend: loaded CPU backend from /home/yangwang/llama.cpp/build/bin/libggml-cpu-riscv64_v.so
build: 7083 (2376b7758) with cc (Bianbu 14.2.0-4ubuntu2~24.04bb1) 14.2.0 for riscv64-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_loader: loaded meta data with 32 key-value pairs and 310 tensors from Qwen3-0.6B-Q4_K_M.gguf (version GGUF V3 (latest))

Signed-off-by: Wang Yang <yangwang@iscas.ac.cn>
@github-actions github-actions bot added the ggml changes relating to the ggml tensor library for machine learning label Nov 24, 2025
@ixgbe ixgbe requested a review from slaren November 24, 2025 09:19
@ggerganov ggerganov merged commit 5f55c38 into ggml-org:master Nov 24, 2025
74 checks passed
bool has_rvv = false;

riscv64_features() {
uint32_t hwcap = getauxval(AT_HWCAP);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be using hwcap as it's known to be broken on some kernels for some boards that falsely report RVV v0.7 as RVV v1.0. It should be using riscv_hwprobe syscall instead. See an example at https://riscv-optimization-guide.riseproject.dev/#_detecting_risc_v_extensions_on_linux

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new PR for fixing this issue : #17567

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants