fix: skip compressed allreduce for empty tensors

[T1mn]

Handle empty buffers in compressed allreduce by early-return and clearing error buffers to avoid NaNs and needless communication.

[tohtana]

Hi @T1mn, thank you for your fix! As we need the same code in four files, can you create deepspeed/runtime/comm/utils.py and define a common logic to avoid duplicated code. You can have something like:

 def check_and_handle_empty_buffer(
      buffer_m: torch.Tensor,
      original_shape: torch.Size,
      original_size: int,
      worker_error: torch.Tensor,
      server_error: torch.Tensor,
  ) -> Optional[torch.Tensor]:
      if original_size == 0:
          if worker_error.numel():
              worker_error.zero_()
          if server_error.numel():
              server_error.zero_()
          if len(original_shape) > 1:
              return buffer_m.reshape(original_shape)
          return buffer_m
      return None

Then you can do

      result = check_and_handle_empty_buffer(
          buffer_m, original_shape, original_size, worker_error, server_error
      )
      if result is not None:
          return result

[T1mn]

Hi @tohtana , thanks for the detailed suggestion. I’ve followed your guidance and extracted the empty-buffer handling into deepspeed/runtime/comm/utils.py as check_and_handle_empty_buffer(...). The four backends now call this helper and return early when it produces a result, so the logic is centralized and consistent across files.
Please let me know if you’d like the helper placed elsewhere or want any additional adjustments.

[tohtana]

Thank you @T1mn! The looks good. Can you fix formatting?

[T1mn]

I’ve run formatting and pushed the updated commit. Please take another look when you have a chance.

Thank you @T1mn! The looks good. Can you fix formatting?

[T1mn]

Hi @tohtana , I may have missed some formatting details, I ran the formatter and pushed the update.
If you spot anything else to tweak, please let me know. Appreciate your time!