bug: Models loading onto wrong GPU in multi-GPU setup

[ericdfrazer]

Jan version

v0.5.6

Describe the Bug

When I select just the 4060 GPU, models load onto the 2070.
When I select just the 2070, models load onto the 4060.

Steps to Reproduce

No response

Screenshots / Logs

What is your OS?

• MacOS
• Windows
• Linux

[dan-menlo]

@vansangpfiev Assigning to you, as this will be closely related to Hardware API

[gabrielle-ong]

Cortex has implemented Hardware detection and activation API (Single GPU)
https://cortex.so/api-reference/#tag/hardware/post/v1/hardware/activate

@louis-jan might need your help to investigate further / implement in Jan