bench(ops): add matmul micro-benchmarks (dense vs sparse operand)#868
Merged
Conversation
Merging this PR will not alter performance
Performance Changes
Comparing Footnotes
|
FBumann
force-pushed
the
bench/kvl-matmul-idiom
branch
from
July 24, 2026 10:19
f4b4f21 to
a7232c0
Compare
FBumann
marked this pull request as draft
July 24, 2026 10:23
FBumann
force-pushed
the
bench/kvl-matmul-idiom
branch
from
July 24, 2026 10:23
a7232c0 to
6ecf331
Compare
The suite had no benchmark exercising @/dot: the kvl_cycles pattern
builds its constraint via the expanded (flow * C).sum('branch'), which
bypasses __matmul__ entirely, and ops.py had no contraction op. A
sparse-aware matmul kernel (#748/#867) would land invisible to CI.
Add a matmul op group contracting the profile's large dim against a
(1000 x 100) constant: expr_matmul_dense (fully nonzero — stays on the
dense kernel) and expr_matmul_sparse (incidence-shaped, ~3 nonzeros per
column — the case a sparse-aware kernel collapses). New ops carry no
CodSpeed history, so this changes no existing baseline; kvl_cycles is
left untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
FBumann
force-pushed
the
bench/kvl-matmul-idiom
branch
from
July 24, 2026 10:27
6ecf331 to
d38c0a4
Compare
FBumann
marked this pull request as ready for review
July 24, 2026 10:29
Collaborator
Author
|
I will merge this as soon as tests finish |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We are not yet tracking sparse matmul in the ops. To optimize it, im adding a baseline benchmark here.
Sorry for the churn. I wanted to keep it in the kvl benchmark, but It regressed with matmul, so decided to keep the existing one.
Note
The following content was generated by AI (Claude Code), prompted and reviewed by @FBumann.
The suite has no benchmark exercising
@/dot: thekvl_cyclespattern — written to show what a sparse-aware matmul would win (#748) — builds its constraint via the expanded(flow * C).sum("branch"), which bypasses__matmul__entirely, andops.pyhas no contraction op. A sparse-aware kernel (#867) would land invisible to CI.An earlier revision of this PR switched
kvl_cyclesitself to the@operator, but that surfaced as a flat +31% memory "regression" across all severities: on the current kernel,Variable.__matmul__goes throughto_linexpr()(ones coefficients) plus a full-size multiply, allocating an extra(time × branch × cycle)float64 temporary that the expansion avoids. Real information about the@idiom's current cost, but the wrong place to change measurement history — so instead this adds a dedicated matmul op group toops.py, following the suite's own op-vs-pattern philosophy ("an op benchmark says which path got heavier"):expr_matmul_dense— fully nonzero (1000 × 100) operand; stays on the dense kernel, guards it.expr_matmul_sparse— incidence-shaped operand at 0.3% density (the@/dotagainst a sparse matrix densifies the result to full_term#748 KVL case, ~3 of 1000 entries per column); flat today, collapses under a sparse-aware kernel.New ops carry no CodSpeed history, so no existing baseline moves and nothing is flagged.
kvl_cyclesis untouched. Landing this before #867 gives that PR a proper baseline: its CodSpeed run should showexpr_matmul_sparsedropping sharply whileexpr_matmul_densestays put.🤖 Generated with Claude Code