[FIX] fix the wrong nproc_per_node in the multi gpu test

[faaany]

What does this PR do?

With the command pytest test_multigpu.py on my NV A100 device, I got a failed test for test_distributed_data_loop. Based on the code, I think it is intended to use 2 processes instead of the available device number.

I also change the require_multi_device to require_multi_gpu, because cuda_visible_devices=0,1 reveals that this test can only be applied to GPU.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Pls have a review, thanks!
@muellerzr or @pacman100

[muellerzr]

Does it pass with two GPUs? (Yes we only test on two gpus but I can't recall if that was meant to be two GPUs or not). I ask if it passes because we've had some people post timeout issues on a100's (due to some hardware issues on later torch versions) so just curious if it passes or not

[faaany]

Does it pass with two GPUs? (Yes we only test on two gpus but I can't recall if that was meant to be two GPUs or not). I ask if it passes because we've had some people post timeout issues on a100's (due to some hardware issues on later torch versions) so just curious if it passes or not

yes, it passed on my A100 and I use the latest pytorch 2.2.0.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[faaany]

@muellerzr BTW, do you mind if I add further XPU tests to this file? e.g.

 @require_multi_xpu
    def test_distributed_data_loop(self):
        print(f"Found {device_count} devices, using 2 devices only")
        cmd = ["torchrun", f"--nproc_per_node={device_count}", self.data_loop_file_path]
        cmd = ["torchrun", "--nproc_per_node=2", self.data_loop_file_path]
        with patch_environment(omp_num_threads=1, ze_affinity_mask="0,1"):
            execute_subprocess_async(cmd, env=os.environ.copy())

Or would it be better that I create a new file called "test_xpu.py" or "test_multixpu.py" in the test folder?

At the moment, when I run pytest test_multigpu.py on XPU, the test will fail. That's also the reason why it is better to change require_multi_device to require_multi_gpu in the current PR. But I am thinking about what would be the best way to add XPU-related tests...

[muellerzr]

Thanks!