multiple mps for base X86 Mac with multiples gpus

[traderpedroso]

🚀 The feature, motivation and pitch

Overview

Your app can use multiple GPUs on an Intel-based Mac, including any built-in and external GPUs. Start by getting a list of all the system’s available GPUs, and then submit workloads to those appropriate for your app’s tasks.

Note

Mac computers with Apple silicon have a single, high-performance, and energy-efficient GPU.

Get a List of GPU Devices

Your app can get an array of MTLDevice instances, each of which represents an available GPU, by calling the MTLCopyAllDevices() function.

let devices = MTLCopyAllDevices()

However, that function provides a list of GPUs that are available at that moment in time. To get the current list and register for device update notifications, provide a handler to Metal by calling the MTLCopyAllDevicesWithObserver(_:_:) function.

let (devices, observer) = MTLCopyAllDevicesWithObserver() { (device, notification) in    self.device(device, issued: notification)}

Metal calls your handler to tell your app when the system adds or removes an MTLDevice from the system.

Note

Metal calls your app’s handler when a device may change its state in the future, such as when a person makes a safe disconnect request. For more information, see Handling External GPU Additions and Removals.

Your app can deregister its observer when it no longer needs GPU device updates from the system by calling the MTLRemoveDeviceObserver(_:) function.

MTLRemoveDeviceObserver(observer)

Identify Each GPU by Type

Each GPU on a Mac computer’s system can be one of three types: integrated, discrete, or external. You can identify each MTLDevice instance’s type by inspecting its isLowPower and isRemovable properties.

GPU Type	isLowPower	isRemovable
true	false
false	false
false	true

Docs Related

Alternatives

As you can see, there are 2021 Intel-based Mac Pros on the market with costs reaching up to $70,000 that demonstrate remarkable performance when training on a single GPU using PyTorch. There are also many in the community who will continue to use MacBook Pros for a significant period. As such, it would be incredible to add support for multiple GPUs. Despite the advent of new ARM-based models, there remains extensive activity in the Intel sector, and we can expect support from Apple to continue for quite some time. A very straightforward issue to check would be whether the ARM or ARM64 system is single GPU, and if it's x86, does it have access to multiple GPUs.

Additional context

No response

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev

[EricPHassey]

Shocked and dissapointed this didn't get anymore attention. The Mac Pro 2019 was "just" released with some versions costing close to $100k and there's no dual GPU/MPS options?