Skip to content

refactor(ops): migrate topksoftmax to canonical InfiniOps API - #1472

Closed
voltjia wants to merge 1 commit into
refactor/migrate-paged-caching-infiniopsfrom
refactor/migrate-topksoftmax-infiniops
Closed

refactor(ops): migrate topksoftmax to canonical InfiniOps API#1472
voltjia wants to merge 1 commit into
refactor/migrate-paged-caching-infiniopsfrom
refactor/migrate-topksoftmax-infiniops

Conversation

@voltjia

@voltjia voltjia commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • migrate the InfiniCore topksoftmax adapter from deprecated TopkSoftmaxInfinilm to canonical TopkSoftmax
  • provide the canonical auxiliary token_expert_indices output as plan-owned internal storage
  • keep the InfiniCore public two-output API and existing norm behavior unchanged

API alignment

InfiniCore adapter Previous InfiniOps API Canonical InfiniOps API Alignment target and evidence
Top-k softmax TopkSoftmaxInfinilm::Call(handle, config, gating_output, topk, renormalize, topk_weights, topk_indices) TopkSoftmax::Call(handle, config, gating_output, bias, is_padding, renormalize, topk_weights, topk_indices, token_expert_indices) vLLM Python API: topk_softmax(topk_weights, topk_ids, token_expert_indices, gating_output, renormalize=False, e_score_correction_bias=None, is_padding=None). InfiniOps groups inputs, attributes, and explicit outputs; bias maps to e_score_correction_bias, and topk_indices maps to topk_ids.

The existing InfiniCore public API does not expose bias, padding, or the auxiliary token-to-expert output. This adapter passes null optional inputs and allocates the required third output in plan-owned storage. Its shape retains the caller's existing topk selection, while the canonical operator derives topk from the output shape.

Dependency

This PR is based on refactor/migrate-paged-caching-infiniops / #1467 and advances the InfiniOps gitlink from e733e325 to 1a712d0c, the head of InfiniOps #891, which extends the canonical provider to the CUDA-compatible backends used by InfiniCore.

Against #1467, this PR changes one adapter file and the InfiniOps gitlink.

Validation

Validated commit 3199b309 remotely on ssh nvidia with accelerator-dev/nvidia:latest, using the exact InfiniOps #891 head 1a712d0c in a temporary integration merge:

  • built and installed _infinicore with the canonical topk_softmax provider in the wrapper allowlist
  • ran python3 test/infinicore/ops/topksoftmax.py --nvidia: 24/24 passed
  • clang-format 16.0.6 --dry-run --Werror passed for the changed C++ file
  • git diff --check passed

@voltjia

voltjia commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #1480, which consolidates the canonical InfiniOps adapter migrations while preserving each logical change as a separate commit.

@voltjia voltjia closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant