Skip to content

feat(nvidia): link vLLM topk_softmax provider - #906

Merged
voltjia merged 1 commit into
masterfrom
feat/linked-topk-softmax-provider
Aug 8, 2026
Merged

feat(nvidia): link vLLM topk_softmax provider#906
voltjia merged 1 commit into
masterfrom
feat/linked-topk-softmax-provider

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a linked vLLM _moe_C::topk_softmax implementation at NVIDIA slot 16.
  • Retain native slot 0 and the existing public API as the complete default path.
  • Reject is_padding at the linked boundary because the tested provider schema does not expose it.

Motivation

Vendor vLLM 0.20.2 exposes a six-argument topk_softmax C10 operator. Registering it as a separate implementation proves the _moe_C provider path while preserving InfiniOps behavior for the parameter that this provider version lacks.

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

python3 -m pytest tests -m smoke --devices nvidia -q
79 passed, 20 skipped, 7826 deselected

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Release operator-pruned build and smoke passed on A100 Focused suite: 83 passed. Provider matrix covered 3 input dtypes, 3 index dtypes, optional bias, and both renormalization modes.
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - additive provider slot; no performance claim.

Notes for Reviewers

  • Default dispatch remains native slot 0. Slot 16 is selected explicitly and has no automatic fallback.
  • is_padding is checked before tensor conversion and rejected with TORCH_CHECK, so Release and Debug behavior match.
  • The YAML records the exact six-argument schema registered by the tested Verl vLLM 0.20.2 DSO and requires the CUDA dispatch key.

@voltjia
voltjia marked this pull request as ready for review August 8, 2026 00:14
@voltjia
voltjia requested a review from a team August 8, 2026 00:14
@voltjia voltjia changed the title feat(nvidia): link vLLM topk softmax provider feat(nvidia): link vLLM topk_softmax provider Aug 8, 2026
@voltjia
voltjia merged commit 939386c into master Aug 8, 2026
20 checks passed
@voltjia
voltjia deleted the feat/linked-topk-softmax-provider branch August 8, 2026 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant