Get ramalama working on Framework laptop

[Djoust]

I'm trying to run a LLM on my AMD Ryzen AI 9 HX 370 w/ Radeon 890M framework laptop with 16gb of memory allocated to the gpu but running into the following error when loading any model:

Memory access fault by GPU node-1 (Agent handle: 0x3453c110) on address 0x7f4d86f5c000. Reason: Page not present or supervisor privilege.

I run:
ramalama run hf.co/Qwen/Qwen2.5-1.5B-Instruct-GGUF:Q2_K

[ehsanullahjan]

Two key pieces of info:

1. ramalama runs a rocm container on AMD systems.
2. rocm 6.4.4 is the first release with support for Ryzen AI 300 series APUs.

Since Bluefin (v43.20260106) ships rocm 6.4.2, rocm support isn't yet available for your APU (I got the same one). Once rocm 6.4.4 ships with Bluefin, I'd expect this to resolve.

In the meantime, you can use LM Studio (AppImage) or docker model run <model>. These work because they fallback to Vulkan which works perfectly.

[Djoust]

Thanks! did found out I can run it with docker, on the gpu!

[castrojo]

I've run into this, there's a variable you can set to have ramalama use vulkan instead but I can't seem to find it.

[shogimoto]

ramalama runs gemma4:12b successful on Dakota on my AMD Ryzen HX370 with 8GB / 24GB setting