Adding nica2a preset

[pierreantoineH]

Description

• Adding nica2a preset (NIC all-to-all over GPUs via NIC executors, multi-node): stride/device grouping and NIC planes.

  • STRIDE — Step size for stride orbits on rank-major devices (gcd with total device count); no traffic between different orbits.
  • GROUP_SIZE — Devices per subgroup inside each stride orbit (natural rank-major order); must divide orbit size (default: all devices per rank × GPUs).
  • NIC_A2A_SCOPE — intra: transfers only within the same device subgroup; inter: only between different subgroups (same stride orbit only).
  • NIC_A2A_NO_SAME_RANK — When non-zero, omit transfers where source and destination are the same rank.
  • NUM_NIC_PLANES — Split NIC endpoints into this many disjoint planes (rank-major index modulo planes); traffic only between NICs in the same plane.

Motivation

Need to stress scale-out network independently from intra node communication with Alltoall pattern

Technical Details

The pattern is defined to use all NICs, can be used in the case of several NICs per GPU .
The stride startegy differ from the one defined in PodAlltoall . Here natural order inside each stride orbit is used. Syntax close to the one of PodAlltoall.
The scope allows to considers running inside the same groups but the need was also to run only between groups.
Also an option in terms of number of planes have been added (can also be used in rail topology) to be able to restrict a2a pattern to have only peers connected through scale-out network.

Test Plan

Tested up to 6 nodes varying different variables.

Test Result

Expected patterns used in each case

[Copilot]

Pull request overview

Adds a new nica2a preset to TransferBench to generate NIC-executor-based all-to-all traffic across GPUs with configurable device stride/grouping, intra/inter-group scope, same-rank filtering, and NIC plane partitioning—intended to stress scale-out networking independently of intra-node GPU communication.

Changes:

• Register new nica2a preset in the preset map.
• Implement NicAllToAllPreset with STRIDE/GROUP_SIZE orbit grouping, NIC plane filtering, and scope controls.
• Document the new preset and its environment variables in the changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
src/client/Presets/Presets.hpp	Adds nica2a preset entry and includes the new preset header.
src/client/Presets/NicAllToAll.hpp	Implements the NIC all-to-all preset logic, transfer generation, and reporting table.
CHANGELOG.md	Documents the new preset and its configuration knobs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[AtlantaPepsi]

I'll test the new changes later today and tomorrow

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[nileshnegi]

@pierreantoineH was encountering build error, so pushed a fix.
after this, nica2a works for me. tested up to 8 nodes.
please let me know if these changes look okay.

[nileshnegi]

if i understand correctly, mpirun -np 8 ./TransferBench nica2a will do all possible node-NIC combinations, and using something like mpirun -np 8 -x STRIDE=8 ./TransferBench nica2a will keep all node-NIC combinations within the rail?

[pierreantoineH]

if i understand correctly, mpirun -np 8 ./TransferBench nica2a will do all possible node-NIC combinations, and using something like mpirun -np 8 -x STRIDE=8 ./TransferBench nica2a will keep all node-NIC combinations within the rail?

The STRIDE argument is making groups of GPUs every STRIDE GPUs and doing communications restricted to those GPUs. Inside 1 group of those GPUs (called orbit) , they still might be split into several subgroups (depending on the GROUP_SIZE) and the communications can be between gpus inside the same subgroup, or between GPUs not in the same subgroups (depending on the NIC_A2A_SCOPE). The STRIDE option is targeted to have groups of GPU like it might be used in some parallelization strategy in workloads. So correct for your question in the effect expected.
Nevertheless there is another option NUM_NIC_PLANES , which is really similar but which target to select GPUs on same planes or rails. NUM_NIC_PLANES target to adapt to physical topologies while STRIDE target to experiment different kind of communication patterns. And both options can be used simultaneously.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.