Enable head dimension 128 test cases for backward and forward equivalence #137
Conversation
Uncomments previously disabled test configurations for head dimension 128, allowing comprehensive testing of backward pass equivalence across various sequence lengths and causal modes. Moves limitation comment to head dimension 256 section where the restriction still applies due to splitkv branch constraints and shared memory limitations.
Uncomments previously disabled test cases for head dimension 128 configurations across various sequence lengths. Moves the comment about sm89 shared memory limitations to head dimension 256 section where it remains applicable.
LoserCheems
requested review from
Evanwu1125,
SNHuan,
Thanksyy,
Copilot,
ftgreat and
wubingheng111
LoserCheems
assigned
ftgreat,
Thanksyy,
SNHuan,
Evanwu1125,
wubingheng111,
LoserCheems and
Copilot
There was a problem hiding this comment.
Pull Request Overview
This PR enables testing for head dimension 128 in both forward and backward equivalence test suites by uncommenting previously disabled test cases. The changes clarify that shared memory limitations now only apply to head dimension 256, not 128.
Key changes:
- Uncommented 12 test cases for head dimension 128 across various sequence lengths (128-4096)
- Updated comments to specify that shared memory limitations apply to head dimension 256, not 128
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| benchmarks/forward_equivalence.py | Enabled head dimension 128 test cases and updated comments about sm89 limitations |
| benchmarks/backward_equivalence.py | Enabled head dimension 128 test cases and clarified comments about splitkv branch limitations |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| (1, 2, 1, 128, 128, 128, True), | ||
| (1, 2, 1, 128, 128, 128, False), |
There was a problem hiding this comment.
Line 603 should have False as the last parameter, but the original commented line 603 had True. This creates an inconsistency where there are two test cases with True and none with False for the (1, 2, 1, 128, 128, 128) configuration.
Uncomment test cases for head dimension 128 in both backward and forward equivalence tests, allowing for comprehensive validation across various sequence lengths. Adjust comments regarding shared memory limitations to reflect the current applicability for head dimension 256.