Does llvmlite support riscv64?

[yuzibo]

---

Feature request

Hi,

On Debian riscv64, I noticed the llvmlite never built due to test failed:

test_add_module (llvmlite.tests.test_binding.TestGlobalConstructors.test_add_module) ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_add_module_lifetime (llvmlite.tests.test_binding.TestGlobalConstructors.test_add_module_lifetime) ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_add_module_lifetime2 (llvmlite.tests.test_binding.TestGlobalConstructors.test_add_module_lifetime2) ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_close (llvmlite.tests.test_binding.TestGlobalConstructors.test_close) ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_emit_assembly (llvmlite.tests.test_binding.TestGlobalConstructors.test_emit_assembly)
Test TargetMachineRef.emit_assembly() ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_emit_object (llvmlite.tests.test_binding.TestGlobalConstructors.test_emit_object)
Test TargetMachineRef.emit_object() ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
ok
test_global_ctors_dtors (llvmlite.tests.test_binding.TestGlobalConstructors.test_global_ctors_dtors) ... WARNING: This target JIT is not designed for the host you are running.  If bad things happen, please choose a different -march switch.
LLVM ERROR: Unsupported code model for lowering
Aborted
E: pybuild pybuild:388: test: plugin distutils failed with: exit code=134: cd /<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_llvmlite/build; python3.11 -m unittest discover -v
...

From the commit, we can support riscv binding test, but only for riscv32, right? Is this a chance to add support riscv64? If yes, What direction should we go? Ihave real riscv64 hardware, please tell me if I can help here.

Sorry, I am very new to llvmlite so please correct me if I am misundering here.

[gmarkall]

Support for riscv64 is getting there but not quite complete yet (any use of it should be considered experimental at this stage). The status is:

• Building llvmlite and using the IR layer works out of the box, as does generating RISC-V code using the binding layer (though not necessarily from a RISC-V host).
• The rest of the binding layer mostly doesn't work OOTB, because MCJIT and RuntimeDyld (used by the current JIT implementation) don't support RISC-V.
• The orcjit-experimental branch can be used for JIT on riscv64 - See PR Experimental OrcJITv2 support #919. However, you will need to build LLVM 14 with a few extra backported patches for fixes to JITLink and to enable JITLink by default on riscv - you can grab a suitable branch from https://github.com/gmarkall/llvm-project/tree/riscv-llvm-14 - I documented the process of backporting and testing this branch here: https://big-grey.co.uk/2023/01/26/using-risc-v-jit-on-llvm-14/. With that patch, you should be able to run the lljit example on riscv64: https://github.com/gmarkall/llvmlite/blob/72b32eacf56c54bc9838bf4b61aa3a8adcd2288b/examples/lljit.py, and the test suite should run to completion successfully (though note adding an object file from the filesystem doesn't work, and I messed up the tests so that it spuriously passes - this shouldn't be an issue in most cases).

I'm quite keen to push forward on RISC-V support, but it is a just a personal project for me so I've not been able to spend a great deal of time on it. Initially I got things tested / brought up with qemu, but I've since been using a Starfive VisionFive 2. What hardware are you using?

From the commit, we can support riscv binding test, but only for riscv32, right? Is this a chance to add support riscv64? If yes, What direction should we go? Ihave real riscv64 hardware, please tell me if I can help here.

That commit tests assembly code generation for riscv32, it doesn't necessarily imply llvmlite runs on riscv32 (though it may well run as well as on riscv64). I haven't come across a Linux-capable riscv32 system yet (that's not in an FPGA or qemu) - have you seen one?

[yuzibo]

hi, sorry for the late reply.
Thanks for your share about supporting the riscv64 status. Very much appreciate for your work.

The rest of the binding layer mostly doesn't work OOTB, because MCJIT and RuntimeDyld (used by the current JIT implementation) don't support RISC-V

Here, could you give more info/hit? So maybe more people know where to start?

What hardware are you using?

I have a real riscv64 hardware(Unmatched board) by hand to test/build some Debian packages.
I am a Debian riscv64 porter so I have to face to many packages with build/test issues on Debian riscv64. llvmlite is one of these packages.:)

When we have a plan to push the patch that support RISC-V JIT on llvm-14 to upstream? Since I'm not familiar with the details of this, so my assumption might be wrong.

I will cost some time to test patch support RISC-V jit but I am very new to llvm. I want to help here but maybe cost a long time.:(

I haven't come across a Linux-capable riscv32 system yet (that's not in an FPGA or qemu) - have you seen one?
Very bad. I have setup riscv32 qemu via yocto. I think this is very inconvenient for developers.

Thanks again for your work!

[gmarkall]

Here, could you give more info/hit? So maybe more people know where to start?

• Use the branch from this PR: Experimental OrcJITv2 support #919
• This example shows how to JIT in a way that will work on RISC-V: https://github.com/gmarkall/llvmlite/blob/orcjit-experimental/examples/lljit.py
• I will add some test skips to the branch that skip all MCJIT tests on RISC-V, so that you can get it to pass without doing anything special. I'll post back here once that's done.

I hope this helps - please let me know if you need further instructions / clarification.

When we have a plan to push the patch that support RISC-V JIT on llvm-14 to upstream? Since I'm not familiar with the details of this, so my assumption might be wrong.

We don't need to do this - the branch I pointed to you is just me backporting fixes that are already in LLVM 15 to LLVM 14. llvmlite will move to LLVM 15 in the future.

I will cost some time to test patch support RISC-V jit but I am very new to llvm. I want to help here but maybe cost a long time.:(

I'd like to help support you in doing this - if you have any trouble with it, please reach out to me and I'll try to guide you.

Thanks again for your work!

Thanks for your interest!

[yuzibo]

Ok, I will try it and thanks for your help here.:)

[markdryan]

I'm wondering whether anyone could provide an update on what the current status of llvmlite on riscv64 is.

I tried building a riscv64 llvmlite 0.44.0 wheel locally (using llvm 15.0.7, patched appropriately as per the build instructions). The wheel builds fine but some of the tests fail, notably.

test_global_ctors_dtors (llvmlite.tests.test_binding.TestGlobalConstructors) ... LLVM ERROR: Unsupported code model for lowering
test_object_cache_getbuffer (llvmlite.tests.test_binding.TestMCJit) ... LLVM ERROR: Relocation type not implemented yet!
test_add_object_file (llvmlite.tests.test_binding.TestObjectFile) ... LLVM ERROR: Relocation type not implemented yet!

I also tried enabling the OrcJIT tests for riscv64 but they also failed

test_add_ir_module (llvmlite.tests.test_binding.TestOrcLLJIT) ... LLVM ERROR: Relocation type not implemented yet!

Are these tests expected to fail on riscv64 or did I mess up my build in some way? If the tests are expected to fail, I'm assuming that this means that neither llvmlite or numba currently support riscv64. Can anyone confirm this?

[seibert]

That's correct. We are not testing on RISC-V, and haven't been tracking if any of the LLVM JIT interfaces support RISC-V. We are working to update our LLVM from version 15 to version 20, but I don't know if LLVM 20 improves the JIT situation on RISC-V.

[markdryan]

That's correct. We are not testing on RISC-V, and haven't been tracking if any of the LLVM JIT interfaces support RISC-V.

Thanks for the confirmation.

We are working to update our LLVM from version 15 to version 20, but I don't know if LLVM 20 improves the JIT situation on RISC-V.

I guess I could give LLVM 20 a try. Is LLVM 20 likely to work with llvmlite 0.44.0 or is it too early to even attempt this? Is there a llvmlite patch set that needs to be applied to LLVM 20?

[seibert]

That work is happening in this PR: #1092

[markdryan]

That work is happening in this PR: #1092

Thanks. I'll try again when that PR is merged and will report back here.

[gmarkall]

I wrote a blog post detailing the required LLVM patches for LLVM 14 to use llvmlite on RISC-V with it passing the whole test suite. I thought none of those patches were required for LLVM 15, but I could have been wrong. The post is: https://big-grey.co.uk/2023/01/26/using-risc-v-jit-on-llvm-14/

Given the error messages you're getting I suspect at least [JITLink] Add R_RISCV_SUB6 relocation may be needed still with LLVM 15.