Skip to content

fix(cuda): bound DeepSeek V4 prefill memory - #1757

Merged
AlpinDale merged 2 commits into
mainfrom
fix/sm89-prefill-memory
Aug 1, 2026
Merged

fix(cuda): bound DeepSeek V4 prefill memory#1757
AlpinDale merged 2 commits into
mainfrom
fix/sm89-prefill-memory

Conversation

@AlpinDale

Copy link
Copy Markdown
Member

Summary

  • write SM89 sparse-prefill output directly into the engine buffer
  • reserve one shared DeepSeek V4 padded-query workspace per worker
  • remove request-time padded-query allocation from the fused CUDA kernel

Validation

  • touched-file pre-commit checks pass
  • prior output-buffer fix survived 9 uncached 70k-token requests in 3 concurrent batches on 8x RTX 4090
  • PR wheel will be used to repeat the stress test with both allocation fixes

Signed-off-by: AlpinDale <alpindale@gmail.com>
Signed-off-by: AlpinDale <alpindale@gmail.com>
@AlpinDale

Copy link
Copy Markdown
Member Author

/build-wheels

@AlpinDale
AlpinDale merged commit 536b49b into main Aug 1, 2026
2 checks passed
@AlpinDale
AlpinDale deleted the fix/sm89-prefill-memory branch August 1, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant