Skip to content

[Relax] Fix int64 row index cast in GPU multinomial sampling - #19902

Merged
tlopex merged 1 commit into
apache:mainfrom
MasterJH5574:relax-sampling-int64-cast
Jun 30, 2026
Merged

[Relax] Fix int64 row index cast in GPU multinomial sampling#19902
tlopex merged 1 commit into
apache:mainfrom
MasterJH5574:relax-sampling-int64-cast

Conversation

@MasterJH5574

Copy link
Copy Markdown
Contributor

After the tirx refactor, a T.let binding no longer implicitly casts
its right-hand side to the annotated dtype. In
gpu_multinomial_from_uniform, row_idx is annotated int64 but is
loaded from the row_indices buffer, whose dtype is the configurable
sample_indices_dtype and may be int32. Relying on the let annotation
to widen the value is no longer valid and yields a dtype mismatch.

Wrap the load in an explicit T.Cast("int64", ...) so the row index is
always converted to the annotated int64 type regardless of
sample_indices_dtype.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the parallel_sampling_from_prob function in python/tvm/relax/backend/gpu_generic/sampling.py to explicitly cast row_indices[bx, 0] to an int64 type using T.Cast when assigning it to row_idx. There are no review comments, and I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

After the tirx refactor, a `T.let` binding no longer implicitly casts
its right-hand side to the annotated dtype. In
`gpu_multinomial_from_uniform`, `row_idx` is annotated `int64` but is
loaded from the `row_indices` buffer, whose dtype is the configurable
`sample_indices_dtype` and may be `int32`. Relying on the let annotation
to widen the value is no longer valid and yields a dtype mismatch.

Wrap the load in an explicit `T.Cast("int64", ...)` so the row index is
always converted to the annotated int64 type regardless of
`sample_indices_dtype`, and update the expected module in
`test_dispatch_multinomial_from_uniform_gpu` to match.
@MasterJH5574
MasterJH5574 force-pushed the relax-sampling-int64-cast branch from 6ccfcde to de89fb1 Compare June 29, 2026 14:14
@tlopex
tlopex merged commit 7cbaa21 into apache:main Jun 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants