add rccl p2p variable

[fsx950223]

Checklist before submitting

• Did you read the contributor guide?
• Did you update the docs?
• Did you write any tests to validate this change?
• Did you update the CHANGELOG, if this change affects users?

Description

Fixes # (issue).

Review process to land

1. All tests and other checks must succeed.
2. At least one member of the technical steering committee must review and approve.
3. If any member of the technical steering committee requests changes, they must be addressed.

[maxhgerlach]

Hi @fsx950223,

thanks for your contribution! Could you please sign off your commit for DCO? See comments in https://github.com/horovod/horovod/pull/3654/checks?check_run_id=7836563465

If this enables hvd.alltoall() for you, we should be able to merge it quickly. As it stands, we do not have any automated tests with non-Nvida GPUs.

[maxhgerlach]

LGTM, I am going to merge immediately.

[romerojosh]

@fsx950223 Do all versions of RCCL support P2P? Shouldn't rccl.h define some equivalent to NCCL_P2P_SUPPORTED that we can depend on rather than defining this directly within Horovod?

[maxhgerlach]

For context: NCCL_P2P_SUPPORTED is defined in Horovod's nccl_operations.h:

#if HAVE_CUDA
#include <nccl.h>
#if NCCL_VERSION_CODE >= NCCL_VERSION(2, 7, 0)
#define NCCL_P2P_SUPPORTED
#endif
#elif HAVE_ROCM
// ..

We could add a similar version check for RCCL if Horovod should be compatible with older versions without P2P support.

[romerojosh]

Ah that is right, NCCL_P2P_SUPPORTED is our own define, just under a version check. I agree that a similar version check should be added for RCCL assuming there are versions that don’t support P2P operations.