Skip to content

feat(cuda): add canonical GELU and sigmoid providers - #889

Merged
voltjia merged 2 commits into
masterfrom
feat/add-canonical-activation-providers
Aug 7, 2026
Merged

feat(cuda): add canonical GELU and sigmoid providers#889
voltjia merged 2 commits into
masterfrom
feat/add-canonical-activation-providers

Conversation

@voltjia

@voltjia voltjia commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add canonical Gelu and Sigmoid CUDA providers for NVIDIA, Iluvatar, MetaX, and Moore.
  • Reuse the existing CUDA-compatible kernels and add focused Gelu coverage for both supported approximation modes.

Motivation

InfiniCore still uses the deprecated GeluInfinilm, GeluTanhInfinilm, and SigmoidInfinilm compatibility operators. Canonical provider coverage is required before those adapters can migrate to the PyTorch-aligned operator names.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Validated on ssh nvidia in accelerator-dev/nvidia:latest as part of the canonical-provider integration stack:

$ scripts/dev/build.sh nvidia --jobs 16
Build succeeded

$ python -m pytest tests/test_gelu.py tests/test_sigmoid.py --devices nvidia -q
148 passed

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes build passed 148 focused tests passed
Iluvatar Yes not run platform hardware unavailable; platform CI required
MetaX Yes not run platform hardware unavailable; platform CI required
Cambricon No N/A N/A - not affected
Moore Yes not run platform hardware unavailable; platform CI required
Ascend No N/A N/A - not affected

Benchmark / Performance Impact

N/A. The canonical providers reuse existing kernels.

Notes for Reviewers

The public schemas already existed; this PR adds native provider coverage and the metadata needed by the generic implementations.

InfiniOps API Alignment target Evidence
Gelu(input, approximate, out) PyTorch torch.nn.functional.gelu(input, approximate='none') PyTorch GELU documentation
Sigmoid(input, out) PyTorch torch.sigmoid(input, *, out=None) PyTorch sigmoid documentation

InfiniOps uses an explicit trailing output tensor for its C++ API. This PR does not add overloads or remove the deprecated InfiniLM compatibility classes. The vendor backend headers are thin wrappers around the shared CUDA implementations.

The NVIDIA result validates the NVIDIA provider and shared kernels. Iluvatar, MetaX, and Moore platform CI remain required.

@voltjia
voltjia force-pushed the feat/add-canonical-activation-providers branch from 498c9aa to 1722a08 Compare August 7, 2026 07:45
@voltjia
voltjia merged commit d0f3693 into master Aug 7, 2026
20 checks passed
@voltjia
voltjia deleted the feat/add-canonical-activation-providers branch August 7, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant