Added a precision for AllGather and ReduceScatter sizes since NCCL us…

…es the size per rank.
NVIDIA · Aug 17, 2018 · dcf8189 · dcf8189
1 parent eb4c43f
commit dcf8189
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/doc/PERFORMANCE.md b/doc/PERFORMANCE.md
@@ -78,6 +78,8 @@ And the Bus Bandwidth is therefore computed as :
 
  `B = S/t * (n-1)/n = algbw * (n-1)/n`
 
+Note that here, S is the size in bytes of the total array, which for NCCL is equal to `recvcount*sizeof(datatype)*n` as the `recvcount` argument is the count per rank.
+
 ### AllGather
 
 The AllGather operation requires only to perform the assignation part of the allReduce operation :
@@ -94,6 +96,8 @@ And the Bus Bandwidth is therefore computed as :
 
  `B = S/t * (n-1)/n = algbw * (n-1)/n`
 
+Note that here, S is the size in bytes of the total array, which for NCCL is equal to `sendcount*sizeof(datatype)*n` as the `sendcount` argument is the count per rank.
+
 ### Broadcast
 
 The broadcast operation representation is similar to allGather :