[CUDAGraph] [FIX] Fix CUDA error(700): 'cudaErrorIllegalAddress' in CascadeAppendW… #4218
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YuhanXu
changed the title
Collaborator
|
PR 标题建议带上 [CUDAGraph] 标签 |
YuhanXu
changed the title
carryyu
reviewed
Sep 23, 2025
| const uint32_t qkv_bias = bias % hidden_size; | ||
| const uint32_t hi = qkv_bias / head_size; | ||
| const uint32_t h_bias = qkv_bias % head_size; | ||
| const uint32_t ori_bi = batch_id_per_token[token_idx]; |
Contributor
Author
There was a problem hiding this comment.
不合适，因为uint32的取值范围大于int32。
不过单看这个kernel里ori_bi之后没有参与别的uint32的赋值，所以是可以的。
我改一下试试
…riteCacheKVQKV cache_kernel(). Continue when batch_id_per_token[token_idx] is default value -1.
YuhanXu
force-pushed
the
rope_cache_kernel_bugfix
branch
from
e09e898 to
db403bd
Compare
|
Thanks for your contribution! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
修复cache_kernel里的 CUDA error(700): 'cudaErrorIllegalAddress'。
由于int* batch_id_per_token默认值会被设成-1，
在代码
const uint32_t ori_bi = batch_id_per_token[token_idx];中，-1被转成uint32_t会导致越界报错。