add device config env for the accelerator

[shiyuan680]

Thank you for pr and @delock contribution of ideas.
As mentioned in this pr, each device has its own environmental variables.
We create visible_devices_envs() and set_visible_devices_envs() methods on the accelerator class to enable each accelerator to implement env settings within the interface , which is more generic to other accelerators.

this commit has tested on npu, each one has 8 ascend npus

[shiyuan680]

@microsoft-github-policy-service agree

[eigen2017]

great thanks to HW and Microsoft，as ascend npu customer, we'll continuously provide contributions.

[CurryRice233]

great thanks to HW and Microsoft，as ascend npu customer, we'll continuously provide contributions.

Thanks to you too :)

[CurryRice233]

@delock @nelyahu @mrwyattii we added an accelerator method to get the env of the visible device, and we don't know the env of other accelerators. Can you provide it?

[delock]

For XPU, the name of env is ZE_AFFINITY_MASK.

I think it would be better to define interface as set_visible_devices(list) and let each accelerator implement setting of env inside the interface. This will give each accelerator flexibility to set visible devices in different ways.

[BacharL]

For HPU HABANA_VISIBLE_MODULES

I think @delock proposed a good approach

[tjruwase]

I think it would be better to define interface as set_visible_devices(list) and let each accelerator implement setting of env inside the interface. This will give each accelerator flexibility to set visible devices in different ways.

@shiyuan680, thanks for creating this PR. Can you please add this interface suggested by @delock?

@delock and @BacharL, thanks for assisting with this PR.

[CurryRice233]

@loadams CI has passed, would you please help to review this PR and add to merge queue ?

[tjruwase]

@loadams CI has passed, would you please help to review this PR and add to merge queue ?

@CurryRice233, can you help to address my questions? Thanks!

[eigen2017]

@loadams CI has passed, would you please help to review this PR and add to merge queue ?

@CurryRice233, can you help to address my questions? Thanks!

yes it is unknown that how to set cpu and mps env，both setting CUDA_VISIBLE_DEVICES, i suppose it is means to keep the same logic to the old version of deepspeed for mps and cpu.

so is it better to add annotations over the code to explain why set CUDA_VISIBLE_DEVICES ?

[tjruwase]

so is it better to add annotations over the code to explain why set CUDA_VISIBLE_DEVICES ?

@eigen2017, yes annotation is reasonable for this. Thanks for your explanation.

[eigen2017]

so is it better to add annotations over the code to explain why set CUDA_VISIBLE_DEVICES ?

@eigen2017, yes annotation is reasonable for this. Thanks for your explanation.

i don't have write access to microsoft/DeepSpeed, could anyone help to add new commits to branch [shiyuan680:master], and add annotations to [mps_accelerator.py] and [cpu_accelerator.py] ? @tjruwase @CurryRice233 @loadams @shiyuan680

explainations can like:
"TODO: before mps' and cpu's visible envs are confirmed, keep the same old env setting as CUDA_VISIBLE_DEVICES "

[eigen2017]

@tjruwase thank you for helping us~~thank you~

[CurryRice233]

@eigen2017 thank you for your explanation, and @tjruwase thank you for the annotation. :)