RVV1.0 Supported tests for RISC-V #16682

alitariq4589 · 2025-10-20T13:22:03Z

This PR adds tests supported for RISC-V

Tests which are added for execution with RISC-V

debian-cpu-cmake-rv64-native
debian-trixie-cmake-sanitizer-riscv64-native * 3
debian-trixie-llguidance-riscv64-native
debian-trixie-cmake-rpc-riscv64-native
ggml-ci-riscv64-native-cpu-low-perf

Dependencies which are not yet supported for RISC-V:

rocblas-dev
hipblas-dev
vulkan-sdk
mthreads/musa:rc4.3.0-devel-ubuntu22.04-amd64
intel-oneapi-compiler-dpcpp-cpp
intel-oneapi-mkl-devel
cuda
macOS*
windows*
torch

Tests which are not added for RISC-V due to above unmet dependencies:

macOS-latest-cmake-arm64
macOS-latest-cmake-x64
macOS-latest-cmake-arm64-webgpu
ubuntu-24-cmake-vulkan
ubuntu-22-cmake-webgpu
ubuntu-22-cmake-hip
ubuntu-22-cmake-musa
ubuntu-22-cmake-sycl
ubuntu-22-cmake-sycl-fp16
macOS-latest-cmake-ios
macOS-latest-cmake-tvos
macOS-latest-cmake-visionos
macOS-latest-swift
windows-msys2
windows-latest-cmake
ubuntu-latest-cmake-cuda
windows-2022-cmake-cuda
windows-latest-cmake-sycl
windows-latest-cmake-hip
ios-xcode-build
android-build
openEuler-latest-cmake-cann
ggml-ci-x64-nvidia-cuda
ggml-ci-x64-nvidia-vulkan-cm
ggml-ci-x64-nvidia-vulkan-cm2
ggml-ci-x64-cpu-amx
ggml-ci-mac-metal
ggml-ci-mac-vulkan
ggml-ci-arm64-cpu-high-perf-sve
ggml-ci-riscv64-native-cpu-high-perf

Additional Notes

Note 1

Due to a warning (treated as error) related to RISC-V simd mappings

In file included from ../../../ggml/src/ggml-cpu/arch/riscv/quants.c:6:
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp16_to_fp32':
../../../ggml/src/ggml-cpu/simd-mappings.h:101:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  101 |         _Float16 hf;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp32_to_fp16':
../../../ggml/src/ggml-cpu/simd-mappings.h:108:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h:108:24: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |                        ^~~~~~~~
cc1: all warnings being treated as errors
make[2]: *** [ggml/src/CMakeFiles/ggml-cpu.dir/build.make:261: ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/arch/riscv/quants.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from ../../../ggml/src/ggml-cpu/quants.c:5:
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp16_to_fp32':
../../../ggml/src/ggml-cpu/simd-mappings.h:101:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  101 |         _Float16 hf;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp32_to_fp16':
../../../ggml/src/ggml-cpu/simd-mappings.h:108:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h:108:24: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |                        ^~~~~~~~
cc1: all warnings being treated as errors
make[2]: *** [ggml/src/CMakeFiles/ggml-cpu.dir/build.make:135: ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/quants.c.o] Error 1
In file included from ../../../ggml/src/ggml-cpu/common.h:7,
                 from ../../../ggml/src/ggml-cpu/unary-ops.h:3,
                 from ../../../ggml/src/ggml-cpu/ggml-cpu.c:12:
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp16_to_fp32':
../../../ggml/src/ggml-cpu/simd-mappings.h:101:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  101 |         _Float16 hf;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp32_to_fp16':
../../../ggml/src/ggml-cpu/simd-mappings.h:108:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |         ^~~~~~~~
../../../ggml/src/ggml-cpu/simd-mappings.h:108:24: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
  108 |         _Float16 hf = (_Float16)f;
      |                        ^~~~~~~~
../../../ggml/src/ggml-cpu/ggml-cpu.c: In function 'ggml_cpu_fp32_to_fp16':
../../../ggml/src/ggml-cpu/ggml-cpu.c:3239:32: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic]
 3239 |         __riscv_vse16_v_f16m1((_Float16 *)&y[i], vy, vl);
      |                                ^~~~~~~~
cc1: all warnings being treated as errors
make[2]: *** [ggml/src/CMakeFiles/ggml-cpu.dir/build.make:79: ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o] Error 1
../../../ggml/src/ggml-cpu/vec.cpp: In function 'void ggml_vec_dot_f32(int, float*, size_t, const float*, size_t, const float*, size_t, int)':
../../../ggml/src/ggml-cpu/vec.cpp:93:41: error: 'vsum' may be used uninitialized [-Werror=maybe-uninitialized]
   93 |         vsum = __riscv_vfmv_v_f_f32m8_tu(vsum, 0.0f, vl);
      |                ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
../../../ggml/src/ggml-cpu/vec.cpp:90:22: note: 'vsum' was declared here
   90 |         vfloat32m8_t vsum;
      |                      ^~~~
cc1plus: all warnings being treated as errors
make[2]: *** [ggml/src/CMakeFiles/ggml-cpu.dir/build.make:219: ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/vec.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2199: ggml/src/CMakeFiles/ggml-cpu.dir/all] Error 2

-DLLAMA_FATAL_WARNINGS=ON has to be turned off for all the tests otherwise CI fails

(This can be created a separated issue. I can track the contributor and ping him if you want)

Note 2

One RISC-V board may not be optimal for running all these tests so by the end of RISC-V summit North America, we are expecting more boards to arrive (around mid of November). So this PR can be treated as draft for reviews till then.

… user

CISC · 2025-11-27T08:13:37Z

@alitariq4589 Any progress on this?

Also, it looks like the runner has gone offline...

alitariq4589 · 2025-11-27T09:14:39Z

@CISC There was a minor internet outage. It is fixed now. Can you check and let me know if the board still shows offline? If that is the case, I will restart the container.

Any progress on this?

I had ordered and integrated 9 RISC-V boards with RVV1.0 so all tests can run smoothly, I am currently testing the ccache by running multiple tests if it works. Here is the build. As soon as I am done testing, I will open the PR for review. Additionally I will also need a single runner token for adding all those boards in llama.cpp repository once I open the PR for review.

It would be great if we have some faster means of communication other than issues and emails. Do you use some other messaging platform (discord, mastodon etc.?). If it is okay, you can also join this discord server.

CISC · 2025-11-27T09:25:27Z

@CISC There was a minor internet outage. It is fixed now. Can you check and let me know if the board still shows offline? If that is the case, I will restart the container.

I cancelled all the old jobs, but there are currently 2 new ones queued and not picked up yet.

Any progress on this?

I had ordered and integrated 9 RISC-V boards with RVV1.0 so all tests can run smoothly, I am currently testing the ccache by running multiple tests if it works. Here is the build. As soon as I am done testing, I will open the PR for review. Additionally I will also need a single runner token for adding all those boards in llama.cpp repository once I open the PR for review.

Great, ping Georgi when you do.

It would be great if we have some faster means of communication other than issues and emails. Do you use some other messaging platform (discord, mastodon etc.?). If it is okay, you can also join this discord server.

Sorry, email only.

alitariq4589 · 2025-11-27T10:10:22Z

@CISC I have restarted the runner and it is picking up the jobs again

Corrections included: 1. Changed the test names from debian to ubuntu as it is more stable than Debian Trixie 2. Added explicit compiler in cmake command as GCC compiler below version 14 have been recorded to throw errors with rvv1.0 and some other extensions 3. Added dependencies which are not installed by default in the RISC-V Ubuntu 24.04 4. Separate ccache directory for all jobs as all the ccache results are not the same and may cause ccache to not work

alitariq4589 · 2025-12-02T07:07:53Z

@CISC This PR is ready for review. I have excluded ggml-ci-riscv64-native-cpu-low-perf as it was causing boards to crash due to high resource utilization. There are 10x Banana Pi F3 boards arriving. When they do, I will use them for testing.

The result of the added builds can be seen here in my fork. There are multiple attempts, so you can check each of them. As the number of boards integrated is greater than the number of tests ported for RISC-V (because there may be some builds in the queue to, these boards will offload some time), the ccache effect is not immediated visible in 4 attempts, but I have tested individually and so far, according to the stats, the ccache seems to be working (check this and this job builds too for ccache results in which I executed each job multiple times for checking ccache results).

Following RISC-V boards will be integrated once I get the token.

8x Milk-V Jupiter boards
1x Banana Pi F3 (BPI F3) - This is already integrated with runner name 87a313462bbe, and I will not add it again for duplication

For checking the resource utilization, I have set up grafana to track usage. This tracking site tracks the usage of the host machine and not the containers in which the builds will be running. Use the following links to view resource usage.

NOTE: Since our network engineer is out of the office for the next couple of days, jupiter-16G-2, jupiter-16G-4, jupiter-16G-5 are not added in usage tracking, so you can ignore their shown data in the tracking dashboards (those are placeholders right now). I will add this within the next two weeks and will comment here.

@ggerganov, please share a github runner token when you can, and I will use that to register all these boards for builds. The token will be valid for one hour after generation, so let me know as soon as you generate it. You can send it to me at my email.

Let me know if anyone has any questions 🙂

ggerganov

@alitariq4589 Sending you the token in a min

ggerganov · 2025-12-02T08:31:34Z

ci/run.sh

+<<<<<<< HEAD
+CMAKE_EXTRA="-DLLAMA_FATAL_WARNINGS=${LLAMA_FATAL_WARNINGS:-ON} -DLLAMA_CURL=ON"
+=======
 CMAKE_EXTRA="-DLLAMA_FATAL_WARNINGS=ON -DLLAMA_CURL=ON -DGGML_SCHED_NO_REALLOC=ON"
+>>>>>>> master


Let me check.

BTW, the background of this change is that, since there is a warning (in the very first comment), tests were failing so I had to turn the "warnings as errors" off for CI to pass. I think we also need to create an issue and ping the contributor for this change. Maybe the RVV1.0 intrinsics change somewhere which is causing this error.

@xctan Could you take a look at this compile warning?

In file included from ../../../ggml/src/ggml-cpu/arch/riscv/quants.c:6: ../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp16_to_fp32': ../../../ggml/src/ggml-cpu/simd-mappings.h:101:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic] 101 | _Float16 hf; | ^~~~~~~~ ../../../ggml/src/ggml-cpu/simd-mappings.h: In function 'riscv_compute_fp32_to_fp16': ../../../ggml/src/ggml-cpu/simd-mappings.h:108:9: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic] 108 | _Float16 hf = (_Float16)f; | ^~~~~~~~ ../../../ggml/src/ggml-cpu/simd-mappings.h:108:24: error: ISO C does not support the '_Float16' type before C23 [-Werror=pedantic] 108 | _Float16 hf = (_Float16)f; | ^~~~~~~~ cc1: all warnings being treated as errors

@ggerganov strangely I dont see conflicts after merging. I dont see conflicts here on the github ui too. Are you testing this on older commit? I just merged master branch to my branch and it seems okay.

You have committed the merge conflict text, it just needs cleaning up.

The _Float16 type seems to be the only way to get the compiler's built-in code generation to work for Zfh and Zvfh extensions. The catch is, this type is only available starting with the ISO C C23 standard. Otherwise, we'd have to resort to inline assembly, which isn't ideal for register usage. Plus, vector intrinsic functions also need this type, and float16_t is exclusively defined in C++ headers (since C++23). So, given all that, maybe we can just disable this diagnostic when _Float16 types are being used?

@CISC My bad. I have cleaned it up now.

So, given all that, maybe we can just disable this diagnostic when _Float16 types are being used?

Sounds good

ci/run.sh

alitariq4589 · 2025-12-02T10:14:29Z

@CISC the .github/workflows/build-riscv-native.yml CI is failing. But should I remove this since most of the things are already covered in the added tests, and it seems redundant?

@ggerganov Thanks for sharing the token. Can you please confirm the following added runners in the repository settings as online?

jupiter-16G-1
jupiter-16G-2
jupiter-16G-4
jupiter-16G-5
jupiter-16G-6
jupiter-16G-7
jupiter-16G-8
jupiter-16G-9
(and one previously added runner with name 87a313462bbe)

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

CISC · 2025-12-02T10:23:10Z

@CISC the .github/workflows/build-riscv-native.yml CI is failing. But should I remove this since most of the things are already covered in the added tests, and it seems redundant?

You tell me. :) If it is of no further value to you it can be removed.

…v1.0ci

alitariq4589 · 2025-12-02T10:36:09Z

@CISC the .github/workflows/build-riscv-native.yml CI is failing. But should I remove this since most of the things are already covered in the added tests, and it seems redundant?

You tell me. :) If it is of no further value to you it can be removed.

Since it was putting extra load on the runners, I have removed it. Everything is already covered in the newly added tests.

.github/workflows/build.yml

CISC · 2025-12-02T12:57:37Z

Lots of whitespace errors:
https://github.com/ggml-org/llama.cpp/actions/runs/19855590279/job/56892592496?pr=16682

alitariq4589 · 2025-12-02T13:16:37Z

Lots of whitespace errors:
https://github.com/ggml-org/llama.cpp/actions/runs/19855590279/job/56892592496?pr=16682

I had added those for verbosity, but I have now removed them. Can you please cancel other CI runs which passed in previous commits? Will save us a lot of time

CISC · 2025-12-02T13:20:50Z

Lots of whitespace errors:
https://github.com/ggml-org/llama.cpp/actions/runs/19855590279/job/56892592496?pr=16682

I had added those for verbosity, but I have now removed them. Can you please cancel other CI runs which passed in previous commits? Will save us a lot of time

It's not the linebreaks that's the issue, it's the indentation spaces.

CIs will autocancel when you push a new commit, however we have a general congestion atm...

.github/workflows/build.yml

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

alitariq4589 · 2025-12-02T13:38:22Z

It's not the linebreaks that's the issue, it's the indentation spaces.

I see. I have removed the trailing spaces now. I also ran the editor-config locally to check and seems good to go.

CISC · 2025-12-02T13:57:10Z

It's not the linebreaks that's the issue, it's the indentation spaces.

I see. I have removed the trailing spaces now. I also ran the editor-config locally to check and seems good to go.

Will merge once EditorConfig and your CIs finish.

CISC · 2025-12-02T14:39:14Z

Hmmm, it seems no tests are actually run?

Edit: Ah, you disable building tests. :)

.github/workflows/build.yml

CISC · 2025-12-02T14:49:18Z

I think ccache is not working properly as well, the builds take too long.

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

alitariq4589 · 2025-12-02T15:39:14Z

I think ccache is not working properly as well, the builds take too long.

That is expected right now because there are a total of 9 boards, and currently, there is no ccache recorded. For ccache to start hitting, every job should have been run at least once on each board. There are currently 6 jobs. If i am not wrong, the probability that the job in the next run will pick the same runner is 1/9. We need around (approx.) 50 runs to start seeing the ccache results because job picks up runner randomly out of 9 (pardon my probability knowledge if I'm mistaken somewhere 🥲)

alitariq4589 · 2025-12-02T15:50:16Z

@CISC You can see the llguidance took 10 minutes this time

CISC · 2025-12-02T19:01:22Z

Looks like git-lfs is not inited.

Also, test-thread-safety fails due to not being compiled with curl or openssl.

.github/workflows/build.yml

CISC · 2025-12-02T20:27:27Z

Looks good now, will merge once the rest are done, git-lfs issue can be resolved post-merge.

alitariq4589 added 5 commits October 20, 2025 13:27

Added RISC-V supported tests

6929268

Added default value for LLAMA_FATAL_WARNINGS and option to specify by…

b1431a4

… user

Added RISC-V supported tests

966cada

Added default value for LLAMA_FATAL_WARNINGS and option to specify by…

c4cfa50

… user

Synced with origin

9258742

alitariq4589 mentioned this pull request Oct 20, 2025

Add Trigger at PR creation #15386

Merged

Removed apt prompt

c64c960

github-actions bot added the devops improvements to build systems and github actions label Oct 20, 2025

alitariq4589 closed this Nov 29, 2025

alitariq4589 reopened this Nov 29, 2025

loci-dev mentioned this pull request Nov 29, 2025

UPSTREAM PR #16682: RVV1.0 Supported tests for RISC-V auroralabs-loci/llama.cpp#365

Open

alitariq4589 added 2 commits December 1, 2025 16:04

Resolved conflicts with master

40c4b63

alitariq4589 marked this pull request as ready for review December 2, 2025 07:07

alitariq4589 requested review from CISC and ggerganov as code owners December 2, 2025 07:07

ggerganov reviewed Dec 2, 2025

View reviewed changes

alitariq4589 added 2 commits December 2, 2025 14:17

Merge branch 'master' into rvv1.0ci

0fb6f21

Resolved the merge conflict and cleaned up run.sh

644b1e5

CISC reviewed Dec 2, 2025

View reviewed changes

ci/run.sh Outdated Show resolved Hide resolved

Update ci/run.sh

f862d15

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

alitariq4589 added 2 commits December 2, 2025 15:32

Removed previously added build ci for RISC-V

684e7be

Pulled branch 'rvv1.0ci' of github.com:alitariq4589/llama.cpp into rv…

6a38407

…v1.0ci

ggerganov approved these changes Dec 2, 2025

View reviewed changes

CISC approved these changes Dec 2, 2025

View reviewed changes

CISC reviewed Dec 2, 2025

View reviewed changes

.github/workflows/build.yml Outdated Show resolved Hide resolved

Removed trailing whitespaces

a2a24c6

alitariq4589 force-pushed the rvv1.0ci branch from 0c9f0a0 to a2a24c6 Compare December 2, 2025 13:36

corrected build name

7f4ecb1

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

cleanup

c6b17de

CISC reviewed Dec 2, 2025

View reviewed changes

alitariq4589 and others added 2 commits December 2, 2025 20:29

Enabled build tests (1)

d11925b

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

Enabled build tests (2)

2254708

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

CISC reviewed Dec 2, 2025

View reviewed changes

enable openssl

09d1891

CISC merged commit 4eba8d9 into ggml-org:master Dec 2, 2025
12 of 65 checks passed

RVV1.0 Supported tests for RISC-V #16682

RVV1.0 Supported tests for RISC-V #16682

Conversation

alitariq4589 commented Oct 20, 2025

Tests which are added for execution with RISC-V

Dependencies which are not yet supported for RISC-V:

Tests which are not added for RISC-V due to above unmet dependencies:

Additional Notes

Note 1

Note 2

Uh oh!

CISC commented Nov 27, 2025

Uh oh!

alitariq4589 commented Nov 27, 2025

Uh oh!

CISC commented Nov 27, 2025

Uh oh!

alitariq4589 commented Nov 27, 2025

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

ggerganov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

alitariq4589 commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025

Uh oh!

CISC commented Dec 2, 2025 •

edited

Loading