[BACKEND][AMD] Disable linear layout due to perf regression #4126
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have identified a 20% perf regression in our downstream flash attention perf kernel after switching to linear layout. Initial analysis shows register pressure is increased to cause spills. Further analysis is still ongoing. So this commit introduces a minimal way to selectively disable linear layout only on AMD backend to avoid affecting NVIDIA backend while contining bring it up on AMD side.
jlebar
approved these changes
Jun 12, 2024
There was a problem hiding this comment.
SGTM. I also observed something similar on nvidia, but never big enough to make a significant performance difference. I was never able to pin it down, though, because once I made smaller testcases the IR was simplified to exactly the same as without LLs. That is, I was seeing that there was something about "big testcases" that prevented some important optimization.
I tried things like increasing the recursion depth in instcombine and compute-known-bits but it didn't help for my testcase.
|
Yeah we are looking into this at the moment. Because in the down stream kernel (specifically, this one) it's already using lots of registers so the change in codegen causes spills. the flash attention tutorial is not as a big issue due to less registers used there. |
jataylo
pushed a commit
to ROCm/triton
that referenced
this pull request
Jun 19, 2024
…ang#4126) We have identified a 20% perf regression in our downstream flash attention perf kernel after switching to linear layout. Initial analysis shows register pressure is increased to cause spills. Further analysis is still ongoing. So this commit introduces a minimal way to selectively disable linear layout only on AMD backend to avoid affecting NVIDIA backend while continuing bring it up on AMD side. (cherry picked from commit e8bc45d)
jataylo
pushed a commit
to jataylo/triton
that referenced
this pull request
Jun 20, 2024
…ang#4126) We have identified a 20% perf regression in our downstream flash attention perf kernel after switching to linear layout. Initial analysis shows register pressure is increased to cause spills. Further analysis is still ongoing. So this commit introduces a minimal way to selectively disable linear layout only on AMD backend to avoid affecting NVIDIA backend while continuing bring it up on AMD side. (cherry picked from commit e8bc45d)
ptillet
pushed a commit
that referenced
this pull request
Jun 20, 2024
Cherry picks for release/3.0.x General: - e8bc45d [BACKEND][AMD] Disable linear layout due to perf regression (#4126) - 9a0a7c2 [AMD] Add basic verification to MFMA encoding (#4117) for RDNA: - 100e2aa [AMD][WMMA] Support dot3d (#3674) - 4a1ea8e [AMD][gfx11] Fix BF16 wmma instr generation (#4135) Proton HIP PRs: - 328b86d [PROTON] Refactor GPU profilers (#4056) - 60613fb [PROTON] Roctracer: convert agent id to gpu id for gpu ops (#4090) - c1776fa [PROTON][AMD] Add Proton HIP GPU Utilization Metrics (#4119) --------- Co-authored-by: Lei Zhang <antiagainst@gmail.com> Co-authored-by: Alexander Efimov <efimov.alexander@gmail.com> Co-authored-by: Ilya V <152324710+joviliast@users.noreply.github.com> Co-authored-by: Keren Zhou <kerenzhou@openai.com> Co-authored-by: mwootton <michael.wootton@amd.com> Co-authored-by: Corbin Robeck <corbin.robeck@amd.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have identified a 20% perf regression in our downstream flash attention perf kernel after switching to linear layout. Initial analysis shows register pressure is increased to cause spills. Further analysis is still ongoing.
So this commit introduces a minimal way to selectively disable linear layout only on AMD backend to avoid affecting NVIDIA backend while contining bring it up on AMD side.