oneCCL specification defines the following reduction operations for Allreduce
, Reduce
and ReduceScatter
collective operations:
enum class reduction
{
sum = /* unspecified */,
prod = /* unspecified */,
min = /* unspecified */,
max = /* unspecified */,
custom = /* unspecified */
};
- reduction::sum
elementwise summation
- reduction::prod
elementwise multiplication
- reduction::min
elementwise min
- reduction::max
elementwise max
- reduction::custom
- specify user-defined reduction operation
the actual reduction function must be passed throughreduction_fn
operation attribute
operation_attributes