Use macro guard CUDA functions for back compatibility in grouped_topk_kernel.cu #25346
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@minosfuture has exported this pull request. If you are a Meta employee, you can view the originating diff in D82918389. |
houseroad
added
ready
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
The pull request introduces a compatibility wrapper is_finite to handle differences in isfinite support across CUDA versions. This is a good approach to maintain backward compatibility. My feedback focuses on improving the implementation of this new function for better robustness and readability.
Comment on lines
421
to
428
| template <typename T> | ||
| __device__ inline bool is_finite(const T val) { | ||
| #if (__CUDACC_VER_MAJOR__ * 10000 + __CUDACC_VER_MINOR__ * 100 >= 120800) | ||
| bool res = cuda::std::isfinite(val); | ||
| #else | ||
| bool res = isfinite(cuda_cast<float, T>(val)); | ||
| #endif | ||
| return res; | ||
| } |
There was a problem hiding this comment.
The implementation of is_finite can be improved for robustness and conciseness.
- The preprocessor check for the CUDA version can be made more robust and standard by using the
__CUDACC_VER__macro. This macro combines major, minor, and patch versions into a single integer (e.g.,120800for 12.8.0), which makes the check cleaner and correctly handles patch versions if needed in the future. The current check only considers major and minor versions. - The function body can be simplified by directly returning the result of the
isfinitecalls within the#if/#elsebranches, removing the need for the intermediateresvariable.
Here is a suggested improved version:
template <typename T>
__device__ inline bool is_finite(const T val) {
#if defined(__CUDACC_VER__) && __CUDACC_VER__ >= 120800
return cuda::std::isfinite(val);
#else
return isfinite(cuda_cast<float, T>(val));
#endif
}
There was a problem hiding this comment.
CUDACC_VER should be deprecated, rigth?
There was a problem hiding this comment.
the training data of gemini needs some refreshing :D
minosfuture
force-pushed
the
export-D82918389
branch
from
da22c1c to
c68fe6f
Compare
|
@minosfuture has exported this pull request. If you are a Meta employee, you can view the originating diff in D82918389. |
…_kernel.cu (vllm-project#25346) Summary: cuda::std::isfinite is not available with earlier CUDA versions. We guard it with macros and extract a device function for is_finite. Test Plan: build with 12.4 and 12.8 Reviewed By: houseroad Differential Revision: D82918389 Privacy Context Container: L1370295 Signed-off-by: Ming Yang <minos.future@gmail.com>
…ject#25250) Signed-off-by: Rahul Tuli <rtuli@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Ming Yang <minos.future@gmail.com>
minosfuture
force-pushed
the
export-D82918389
branch
from
3b5ebfd to
ccabe13
Compare
minosfuture
requested review from
simon-mo,
WoosukKwon,
youkaichao,
robertgshaw2-redhat,
mgoin,
tlrmchlsmth,
houseroad,
hmellor,
yewentao256 and
ProExpertProg
as code owners
houseroad
approved these changes
Sep 21, 2025
yewentao256
approved these changes
Sep 22, 2025
5 tasks
FeiDaLI
pushed a commit
to FeiDaLI/vllm
that referenced
this pull request
Sep 25, 2025
…_kernel.cu (vllm-project#25346) Signed-off-by: Ming Yang <minos.future@gmail.com> Signed-off-by: Rahul Tuli <rtuli@redhat.com> Co-authored-by: Rahul Tuli <rtuli@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com> Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com>
yewentao256
added a commit
that referenced
this pull request
Oct 3, 2025
…_kernel.cu (#25346) Signed-off-by: Ming Yang <minos.future@gmail.com> Signed-off-by: Rahul Tuli <rtuli@redhat.com> Co-authored-by: Rahul Tuli <rtuli@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Lu Fang <30275821+houseroad@users.noreply.github.com> Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com> Signed-off-by: yewentao256 <zhyanwentao@126.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
cuda::std::isfinite is not available with earlier CUDA versions. We guard it
with macros and extract a device function for is_finite.
Test Plan: build with 12.4 and 12.8
Reviewed By: houseroad
Differential Revision: D82918389