test_ray.py::test_gpu_ids_num_workers sometimes fails on Buildkite

[maxhgerlach]

CUDA_VISIBLE_DEVICES seems to contain too many entries.

Example from PR #3261: https://buildkite.com/horovod/horovod/builds/7041#9a807189-938e-491f-9f83-c6bc31420a67

        hjob = RayExecutor(setting, num_workers=4, use_gpu=True)
        hjob.start()
        all_envs = hjob.execute(lambda _: os.environ.copy())
        all_cudas = {ev["CUDA_VISIBLE_DEVICES"] for ev in all_envs}
        assert len(all_cudas) == 1, all_cudas
>       assert len(all_envs[0]["CUDA_VISIBLE_DEVICES"].split(",")) == 4
E       assert 8 == 4
E         +8
E         -4

[ashahab]

@maxhgerlach thanks for the report, I'll take a look tonight.

[maxhgerlach]

Closing in favor of #3435