Multi-GPU

[kltye]

Is there a way to force which GPU CPAI runs on? I've used CUDA_VISIBLE_DEVICES in Windows but it doesn't seem to have any effect (models appear to run on the GPU I wish to exclude). Additionally, does CPAI load-balance between GPUs?

[m411b]

I'd like to know the same. Is there a way to get CPAI to run on a chosen GPU rather than a GPU it picks?

[ChrisMaunder]

GPU usage is specific to each module. It's up to the module (and module's author) how they handle GPUs and what options they provide to the server to pass back to the users

[m411b]

Weird. When I go to the CodeProject.AI-ObjectDetectionYOLOv8 module page the only person mentioned is "ChrisMaunder". Is Chris Maunder not the YOLOv8 author? If this is left up to the module (and module's author), then why can you not help us with this? I'd like to point out, though, that it doesn't matter what module is being used. CPAI always uses GPU 0. So I very highly doubt this is a module issue. Thanks for all your help, Chris..

[ChrisMaunder]

Yes, I wrapped the YOLOv8 code into a module, but I'm not the author of the YOLOv8 code per se.

The point I was making is this is a module-by-module issue, not a general server issue.

Adding multi-GPU support is (I'm pretty sure) fairly simple, but we just don't have a lot of spare time right now (so my apologies)