Skip to content

feat(ops): add Copy operator - #872

Merged
voltjia merged 1 commit into
masterfrom
feat/add-copy-operator
Aug 5, 2026
Merged

feat(ops): add Copy operator#872
voltjia merged 1 commit into
masterfrom
feat/add-copy-operator

Conversation

@voltjia

@voltjia voltjia commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a canonical Copy operator with the public C++ signature Copy(const Tensor src, const bool non_blocking, Tensor out).
  • Adds shared CUDA implementation coverage for NVIDIA, Iluvatar, MetaX, and Moore, including broadcasted and strided source layouts.
  • Keeps the deprecated RearrangeInfinilm API and tests intact while reusing CopyKernel directly.
  • Adds one parameterized test_copy covering non_blocking, broadcasting, non-contiguous layouts, and float32/float16/bfloat16.

Motivation

RearrangeInfinilm was introduced as an InfiniLM compatibility operator and is now deprecated. A normal operator aligned with an established open-source API is required before downstream callers can migrate away from the suffixed interface.

API Alignment

InfiniOps API Alignment target Mapping Evidence
Copy(const Tensor src, const bool non_blocking, Tensor out) PyTorch Tensor.copy_(src, non_blocking=False) -> Tensor src and non_blocking preserve the PyTorch order. PyTorch's mutable receiver/returned self is represented by trailing caller-provided out per the InfiniOps input-attribute-output convention. InfiniOps uses Copy/copy rather than an in-place _ suffix because mutation is expressed by out. The C++ API requires the defaulted boolean explicitly and does not add a convenience overload. PyTorch 2.13 torch.Tensor.copy_

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

NVIDIA A100-SXM4-80GB, accelerator-dev/nvidia:latest:

python3 -m pip install . --force-reinstall --no-build-isolation --no-deps \
  --config-settings=cmake.define.INFINI_RT_ROOT=/opt/infinirt \
  --config-settings=cmake.define.AUTO_DETECT_DEVICES=OFF \
  --config-settings=cmake.define.AUTO_DETECT_BACKENDS=OFF \
  --config-settings=cmake.define.WITH_NVIDIA=ON \
  --config-settings=cmake.define.WITH_CPU=OFF \
  --config-settings=cmake.define.CMAKE_CUDA_ARCHITECTURES=80 \
  --config-settings=cmake.define.INFINI_OPS_SMOKE_BUILD=ON
Successfully built InfiniOps
Successfully installed InfiniOps-0.1.0

python3 -m pytest tests -m smoke -q
64 passed, 34 skipped, 8246 deselected in 4.13s

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes smoke passed Focused copy + rearrange_infinilm: 54 passed, 54 skipped in 1.85s; wrapper generator: 22 passed in 1.58s
Iluvatar Yes Not run Backend hardware/toolchain unavailable
MetaX Yes Not run Backend hardware/toolchain unavailable
Cambricon No N/A N/A
Moore Yes Not run Backend hardware/toolchain unavailable
Ascend No N/A N/A

Benchmark / Performance Impact

N/A. This adds an API migration path and reuses the same elementwise copy kernel for the canonical and compatibility operators.

Notes for Reviewers

  • RearrangeInfinilm remains deprecated and its public signature and test coverage are unchanged.
  • The current CUDA providers require matching dtype and device. PyTorch's cross-dtype and cross-device behavior is implementation work outside this interface-migration PR.
  • For the currently supported same-device CUDA path, PyTorch documents that non_blocking has no effect; the parameter is retained to keep the canonical interface complete.
  • No torch_ops.yaml, CMake allowlist, generated artifact, or default-value overload is added.
  • Final formatting checks passed with clang-format 21.1.8 and ruff 0.15.22.

@voltjia voltjia changed the title feat(ops): add Copy operator feat(ops): add Copy operator Aug 5, 2026
@voltjia
voltjia force-pushed the feat/add-copy-operator branch from 185b4c9 to 5bae094 Compare August 5, 2026 07:27
@voltjia
voltjia marked this pull request as ready for review August 5, 2026 07:37
@voltjia
voltjia requested a review from a team August 5, 2026 07:37
@voltjia
voltjia merged commit 701795f into master Aug 5, 2026
18 of 20 checks passed
@voltjia
voltjia deleted the feat/add-copy-operator branch August 5, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant