Skip to content

feat(linked): add vLLM grouped_topk provider - #908

Merged
voltjia merged 1 commit into
masterfrom
feat/linked-grouped-topk-provider
Aug 8, 2026
Merged

feat(linked): add vLLM grouped_topk provider#908
voltjia merged 1 commit into
masterfrom
feat/linked-grouped-topk-provider

Conversation

@voltjia

@voltjia voltjia commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add a linked NVIDIA slot 16 for vLLM _moe_C::grouped_topk.
  • Preserve the existing native slot 0 and public InfiniOps API.
  • Cover native slot 0 and vLLM slot 16 in the existing grouped_topk test module.

Motivation

InfiniOps currently carries a native grouped_topk implementation derived from
upstream serving kernels. The installed vLLM provider already exposes the
operation through the C10 Dispatcher, so this PR adds a thin linked adapter
without copying another provider kernel into the repository.

#906 is merged and provides the shared vllm/_moe_C.*.so library mapping.

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • Hygon (WITH_HYGON)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Remote environment: A100 with the Cornserve vLLM provider image, using an
operator-pruned Release build with WITH_LINKED=ON.

python -m pytest tests -m smoke --devices nvidia -q
79 passed, 19 skipped, 7807 deselected

The smoke allowlist is unchanged.

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes Release operator-pruned build and existing smoke passed on A100 Linked slot 16: 4 passed; native slot 0: 13 passed, 1 skipped
Iluvatar No N/A N/A
MetaX No N/A N/A
Cambricon No N/A N/A
Moore No N/A N/A
Ascend No N/A N/A

Benchmark / Performance Impact

N/A - no performance claim.

Notes for Reviewers

  • The adapter computes the vLLM provider inputs required by the registered
    schema (scores and scores_with_bias), calls the provider on the caller's
    stream, and copies the returned tuple into InfiniOps' output-tail tensors.
  • Slot 16 intentionally follows the vLLM provider's observable low-precision
    rounding, tie, and nonfinite behavior. It does not claim bitwise equivalence
    with native slot 0. The native implementation remains the default and keeps
    the existing InfiniOps contract.
  • Provider-specific cases in tests/test_grouped_topk.py explicitly select slot 16
    and compare against the loaded vLLM operator. Without that provider, only those
    cases skip; the native regression cases still run.

@voltjia
voltjia force-pushed the feat/linked-grouped-topk-provider branch from fbbb737 to 6c87a0e Compare August 7, 2026 17:57
Base automatically changed from feat/linked-topk-softmax-provider to master August 8, 2026 00:15
@voltjia
voltjia force-pushed the feat/linked-grouped-topk-provider branch 3 times, most recently from 7b32f24 to 36b8624 Compare August 8, 2026 01:04
@voltjia voltjia changed the title feat(linked): add vLLM grouped_topk provider feat(linked): add vLLM grouped_topk provider Aug 8, 2026
@voltjia
voltjia force-pushed the feat/linked-grouped-topk-provider branch from 36b8624 to 2a23f12 Compare August 8, 2026 02:52
@voltjia
voltjia marked this pull request as ready for review August 8, 2026 11:32
@voltjia
voltjia requested a review from a team August 8, 2026 11:32
@voltjia
voltjia merged commit 2d9e9ec into master Aug 8, 2026
20 checks passed
@voltjia
voltjia deleted the feat/linked-grouped-topk-provider branch August 8, 2026 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant