Adding mps support to base handler and regression test

[udaij12]

Description

Adding MPS as device type for Mac M1. #3022

When Model-config.yaml deviceType: "gpu" or deviceType is not specified then default to GPU:
MPS enabled

When Model-config.yaml deviceType: "cpu":
MPS not enabled

Test cases

Added 4 test cases.
Frontend:

1. test that number of gpu is greater then 0 when on Mac M1

Backend:

1. Set deviceType to GPU and test that device == "mps"
2. Set deviceType to CPU and test that device != "mps"
3. Dont set deviceType (remains at default) and test that device == "mps"

[agunapal]

Can we add mps here

serve/model-archiver/README.md

Line 174 in 1a99de4

deviceType: cpu # cpu, gpu, neuron

cc @lxning

[agunapal]

In terms of design, having one config for all deviceType probably makes sense?

[lxning]

1. CX still can specify deviceType (CPU or GPU) on Mac.
2. The PR should be able to support getNumber of GPUs on Mac.
3. The GPU device id is decided by frontend, which is as same as Linux.
4. Backend handler should be able to auto-detect Mac and apply MPS+deviceId. Essentially, there are no new config parameters in the model-config.yaml.

[udaij12]

MPS is auto detected in the base handler and getAvailableGpu() now supports Mac M1 gpu cores.

[agunapal]

Please add the User experience for both the cases in the PR/ add this in a README and add a pytest for MPS on/off for a model

[agunapal]

cc: @msaroufim

[lxning]

Can you add test cases?

1. define deviceType in model-config.yaml on Mac
2. cpu
3. gpu

• deviceId

2. define deviceType in model-config.yaml on gpu host

[lxning]

@udaij12 in general, LGTM. I just left a minor comments #3048 (comment). Thanks.

[agunapal]

@udaij12 Can you please add a document called apple_silicon_support.md in docs folder where you mention these details. also please include what is currently working

[agunapal]

LGTM

stale