Skip to content

refactor(cuda): adapt RearrangeInfinilm to Copy - #897

Merged
voltjia merged 1 commit into
masterfrom
refactor/adapt-rearrange-infinilm-to-copy
Aug 7, 2026
Merged

refactor(cuda): adapt RearrangeInfinilm to Copy#897
voltjia merged 1 commit into
masterfrom
refactor/adapt-rearrange-infinilm-to-copy

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Implement the deprecated CUDA-family RearrangeInfinilm provider as a thin adapter over canonical Copy.
  • Remove the duplicate metadata allocation, launcher, and direct kernel dispatch while preserving the legacy base interface and tests.

Motivation

RearrangeInfinilm(input, out) is the non_blocking=false subset of the existing Copy(src, non_blocking, out) operator. Keeping two provider implementations would duplicate the same strided copy behavior during the InfiniLM compatibility migration.

No issue is closed by this PR.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

NVIDIA validation used accelerator-dev/nvidia:latest on an A100 host and the current InfiniRT install prefix:

cmake -S /workspace/InfiniOps -B /workspace/InfiniOps/build-adapters \
  -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=80 \
  -DWITH_NVIDIA=ON -DGENERATE_PYTHON_BINDINGS=ON \
  -DINFINI_RT_ROOT=/opt/infinirt \
  -DINFINI_OPS_OPS=copy,rearrange_infinilm \
  -DFETCHCONTENT_SOURCE_DIR_CUTLASS=/tmp/cutlass-src
cmake --build /workspace/InfiniOps/build-adapters --target ops --parallel 8

python3 -m pytest tests/test_copy.py tests/test_rearrange_infinilm.py \
  --devices nvidia -q
54 passed in 2.39s

Repository checks:

clang-format 21.1.8 --dry-run --Werror: passed
ruff 0.15.22 check: passed
ruff 0.15.22 format --check: 148 files already formatted
python -X utf8 scripts/generate_wrappers.py \
  --devices nvidia iluvatar metax moore \
  --ops copy rearrange_infinilm --strict-ops
passed

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes focused ops build passed 54 focused tests passed on A100
Iluvatar Yes strict wrapper generation passed Native hardware not available; platform CI/owner validation required
MetaX Yes strict wrapper generation passed Native hardware not available; platform CI/owner validation required
Cambricon No N/A - not affected N/A - not affected
Moore Yes strict wrapper generation passed Native hardware not available; platform CI/owner validation required
Ascend No N/A - not affected N/A - not affected
Full focused `pytest` output
Running 54 items in this shard
......................................................                   [100%]
54 passed in 2.39s

Benchmark / Performance Impact

N/A. The adapter calls the same canonical copy kernel and this PR makes no performance claim.

Notes for Reviewers

Alignment table

Legacy interface Canonical interface Open-source target
RearrangeInfinilm(input, out) Copy(src=input, non_blocking=false, out) PyTorch Tensor.copy_(src, non_blocking=False)
  • InfiniOps retains its explicit out form so the output allocation remains caller-owned.
  • The legacy class, public signature, backend registrations, and tests remain in place and remain deprecated.
  • The adapter forwards the selected stream to the cached CudaCopy provider before every call.

@voltjia voltjia changed the title refactor(cuda): adapt rearrange infinilm to copy refactor(cuda): adapt RearrangeInfinilm to Copy Aug 7, 2026
@voltjia
voltjia marked this pull request as ready for review August 7, 2026 05:59
@voltjia
voltjia requested a review from a team August 7, 2026 05:59
@voltjia
voltjia merged commit 647f1ca into master Aug 7, 2026
16 of 20 checks passed
@voltjia
voltjia deleted the refactor/adapt-rearrange-infinilm-to-copy branch August 7, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant