Skip to content

[Fix] fix num_kv_split becomes 0#45

Merged
hannahli-nv merged 2 commits intomainfrom
jinmanx/fix
Jan 31, 2026
Merged

[Fix] fix num_kv_split becomes 0#45
hannahli-nv merged 2 commits intomainfrom
jinmanx/fix

Conversation

@xjmxyt
Copy link
Copy Markdown
Collaborator

@xjmxyt xjmxyt commented Jan 31, 2026

Description

Previously, we use NUM_KV_SPLITS = NUM_SMS // (batch_size * num_kv_heads), it has possibility to become 0.
Now change to max(1, NUM_SMS // (batch_size * num_kv_heads), it is at least 1.

CI Configuration

config:
  build: true
  # valid options are "ops" and "benchmark"
  test: ["ops", "benchmark"]

Checklist

  • Code formatted and imports sorted via repo specifications (./format.sh)
  • Documentation updated (if needed)
  • CI configuration reviewed

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Jan 31, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@xjmxyt
Copy link
Copy Markdown
Collaborator Author

xjmxyt commented Jan 31, 2026

/ok to test 008393e

@xjmxyt
Copy link
Copy Markdown
Collaborator Author

xjmxyt commented Jan 31, 2026

/ok to test 8245eda

@xjmxyt xjmxyt requested review from azazhu and hannahli-nv January 31, 2026 08:46
@hannahli-nv hannahli-nv merged commit b5ea56e into main Jan 31, 2026
18 checks passed
@hannahli-nv hannahli-nv deleted the jinmanx/fix branch January 31, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants