feat(gemm_a8w8_blockscale_bpreshuffle): add GLM-5.2 tuned configs (gfx950)#4243
Open
Raiden-Makoto wants to merge 1 commit into
Open
feat(gemm_a8w8_blockscale_bpreshuffle): add GLM-5.2 tuned configs (gfx950)#4243Raiden-Makoto wants to merge 1 commit into
Raiden-Makoto wants to merge 1 commit into
Conversation
…x950) The a8w8_blockscale_bpreshuffle GEMM has per-model tuned configs for dsv3/mm2.5/qwen but none for GLM-5.2. Its 128-aligned MLA projections on gfx950 -- q_b_proj (4096,2048) and o_proj (6144,4096) -- miss the tuned lookup and fall back to the default solution. Add a model_configs file with tuner-generated entries (gfx950/cu256, prefill + decode M). Auto-merged by get_config_file's model_configs glob; no code changes.
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
yifehuan
reviewed
Jul 16, 2026
| @@ -0,0 +1,121 @@ | |||
| gfx,cu_num,M,N,K,libtype,kernelId,splitK,us,kernelName,tflops,bw,errRatio | |||
| gfx950,256,1024,4096,2048,asm,5,1,18.4361,_ZN5aiter43fp8gemm_bf16_blockscale_BpreShuffle_128x128E,931.86,1023.77,0.0 | |||
Contributor
There was a problem hiding this comment.
there are some duplicated cases from other model config csv (same a8w8 blockscale) make ci pipeline failed, please recheck
Author
|
Fixed in |
Oseltamivir
added a commit
to SemiAnalysisAI/InferenceX
that referenced
this pull request
Jul 18, 2026
The cookbook-verified v0.5.13.post1-rocm720-mi35x-20260618 build segfaults in AITER gemm_a8w8_blockscale_bpreshuffle mid-profiling under the agentic corpus (KV usage 0.95+, 5.3M pending tokens, untuned GLM-5.2 GEMM shapes falling back to default configs - gfx950 tuned configs are not upstream yet, ROCm/aiter#4243). Move to the newest rocm720 build for the latest AITER fallback fixes. Cap the conc list at 32 (was 64): the B300 sibling frontier showed no-offload points beyond ~c48 are KV-cliff-dominated on the 131k-token mean corpus, and MI355X prefill is slower still - c48/c64 only added crash surface and sweep time for dominated points. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The
a8w8_blockscale_bpreshuffleGEMM has per-model tuned configs underaiter/configs/model_configs/for dsv3, mm2.5, and qwen — but none for GLM-5.2.GLM-5.2's two 128-aligned MLA dense projections on gfx950 —
q_b_proj(N=4096, K=2048) and
o_proj(N=6144, K=4096) — therefore miss the tuned-configlookup and fall back to the default solution (aiter logs
not found tuned config in ... will use default config).Note aiter already ships
a8w8_bpreshuffle_tuned_gemm_glm5.2.csv, but that is theper-tensor bpreshuffle path (schema has
q_dtype_w). These projections use theblock-scale (128x128) path (
a8w8_blockscale_bpreshuffle), which has no GLM-5.2config. This PR adds one. Config coverage only — no code changes.
Technical Details
aiter/configs/model_configs/a8w8_blockscale_bpreshuffle_tuned_gemm_glm5.2.csvgfx950/cu_num=256covering both shapes(
q_b_proj4096,2048 ando_proj6144,4096).sizes (1–512).
a8w8_blockscale_bpreshuffle_tuned_gemm.csv:gfx,cu_num,M,N,K,libtype,kernelId,splitK,us,kernelName,tflops,bw,errRatio(libtype ∈ {asm, ck, cktile}).
AiterConfigs.get_config_file(...)globsmodel_configs/*a8w8_blockscale_bpreshuffle_tuned_gemm*.csvandupdate_config_files(...)merges + dedups (lowestusper shape) with the baseat runtime, so the new file is picked up automatically.
(
csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --preshuffle), nomanual edits.
Test Plan
fallback warning (i.e.
get_config_filemerges the new model_config).op_tests/test_gemm_a8w8_blockscale.pyon the added shapes.Test Result
not found tuned config … will use defaultwarningsfor
4096,2048and6144,4096no longer appear at load; both resolve to theirtuned entries.
Submission Checklist
black/ruff).