AMD GPU/APU acceleration (CLBLAS), Nvidia GPU acceleration (CUBLAS)

[sammcj]

It would be awesome if Khoj worked with AMD GPU / APU acceleration given how popular Ryzen CPU/APUs are these days.

For reference - LocalAI does this with CLBLAS so it should be possible - https://localai.io/basics/build/#clblas

---

Somewhat related to #388

[sammcj]

I just tried the project out with an Nvidia card - it doesn't seem to use CUDA/CUBLAS with nvidia cards either - perhaps a config option is missing somewhere to enable this?

[debanjum]

For Khoj Chat, #494 has added support for running on Vulcan supported GPU+OS. I don't have access to such a device myself but if your Machine is Vulcan supported, give Khoj a try and report back if it still doesn't engage your AMD, Nvidia GPU

[debanjum]

Marking this issue as completed for now as Khoj now supports at least some Nvidia, AMD GPUs with Vulkan support added to GPT4All (our upstream dependency). But feel free to reopen the issue if it doesn't work for you