Repository navigation
Perf Tracker 2026 11 08
Nightly run 42394161005, commit 31c92ddaf, NVIDIA H200, CUDA 13.2, driver 580.178.04, torch 2.13.0+cu132, TileLang 0.1.12, image ghcr.io/tile-ai/tileops-ci-runner:local-runner-2.338.2. Times are CUPTI device_busy_ms.
A row is behind when its fastest comparator's time over TileOPs' time is below 0.99. A row whose fastest comparator takes under 10 us is behind only if it was also below that ratio the night before. An op is labelled by its worst row: P0 < 0.50, P1 < 0.80, P2 < 0.90, P3 < 0.95, P4 < 0.99.
| Measure | Value |
|---|---|
| Ops behind | 12 of 206 |
| Rows behind | 25 of 2333 |
| By label | P0 0, P1 0, P2 2, P3 0, P4 10 |
| Failed rows | 0 |
Since nightly run 42351460931:
- Newly behind:
ReciprocalFwdOp,WhereFwdOp - Caught up:
BMMFwdOp,GEMMFP8FwdOp,GQAVarlenFwdOp,RsqrtFwdOp - Not measured tonight: none
Nightlies behind counts the consecutive nightlies, up to 30, in which the op had a row behind.
| # | Op | Category | Label | Worst ratio | Rows behind | Nightlies behind |
|---|---|---|---|---|---|---|
| 1 | GroupedGEMMFwdOp | gemm | P2 | 0.8587 | 3 of 10 | 4 |
| 2 | GEMMFwdOp | gemm | P2 | 0.8710 | 6 of 27 | 5 |
| 3 | RMSNormFwdOp | norm | P4 | 0.9660 | 4 of 34 | 6 |
| 4 | MoEGroupedGEMMFP8FwdOp | moe | P4 | 0.9731 | 1 of 18 | 4 |
| 5 | NeFwdOp | elementwise | P4 | 0.9782 | 2 of 13 | 4 |
| 6 | SoftmaxFwdOp | reduction | P4 | 0.9787 | 2 of 42 | 5 |
| 7 | MaxPool2dIndicesFwdOp | pool | P4 | 0.9824 | 1 of 9 | 3 |
| 8 | GtFwdOp | elementwise | P4 | 0.9853 | 1 of 13 | 3 |
| 9 | LogicalOrFwdOp | elementwise | P4 | 0.9855 | 2 of 18 | 4 |
| 10 | RemainderFwdOp | elementwise | P4 | 0.9860 | 1 of 14 | 5 |
| 11 | ReciprocalFwdOp | elementwise | P4 | 0.9886 | 1 of 5 | 1 |
| 12 | WhereFwdOp | elementwise | P4 | 0.9887 | 1 of 8 | 1 |
Benchmark benchmarks.ops.bench_grouped_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
qwen3-30b-ep8-skew-down-bfloat16 |
48.145 | quack | 41.344 | 0.8587 |
qwen3-30b-ep8-skew-up-bfloat16 |
83.984 | quack | 80.031 | 0.9529 |
qwen3-30b-ep8-routed1k-bfloat16 |
30.977 | quack | 30.592 | 0.9876 |
Benchmark benchmarks.ops.bench_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
qwen2.5-7b-qkv-b64-bfloat16 |
13.152 | cublaslt-best | 11.456 | 0.8710 |
ds-v3-m96-qkv-a-bfloat16 |
14.816 | cublaslt-best | 14.304 | 0.9654 |
ds-v3-m2-qkv-a-bfloat16 |
12.416 | cublaslt-best | 12.002 | 0.9666 |
qwen3-8b-gate-up-b32-bfloat16 |
49.312 | cublaslt-best | 48.128 | 0.9760 |
ds-v3-m64-down-bfloat16 |
9.984 | cublaslt-best | 9.856 | 0.9872 |
ds-v3-prefill-down-bfloat16 |
170.321 | cublaslt-best | 168.337 | 0.9884 |
Benchmark benchmarks.ops.bench_norm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
quack-throughput-8192-float16 |
256.193 | torch-compile | 247.473 | 0.9660 |
quack-throughput-8192-bfloat16 |
255.633 | torch-compile | 251.089 | 0.9822 |
quack-throughput-256-float16 |
8.800 | torch-compile | 8.672 | 0.9855 |
quack-throughput-256-bfloat16 |
8.800 | torch-compile | 8.673 | 0.9856 |
Benchmark benchmarks.ops.bench_moe_grouped_gemm_fp8.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
qwen3-30b-ep8-skew-up |
48.832 | deepgemm | 47.520 | 0.9731 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
linear-bias-bfloat16 |
4.384 | torch-compile | 4.288 | 0.9782 |
linear-bias-float16 |
4.384 | torch-compile | 4.320 | 0.9854 |
Benchmark benchmarks.ops.bench_softmax.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
quack-throughput-512-bfloat16 |
16.512 | quack | 16.160 | 0.9787 |
quack-throughput-2048-bfloat16 |
63.680 | quack | 62.688 | 0.9844 |
Benchmark benchmarks.ops.bench_pool.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
vgg-block-float16 |
9.088 | torch-compile | 8.928 | 0.9824 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
linear-bias-bfloat16 |
4.352 | torch-compile | 4.288 | 0.9853 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
linear-bias-bfloat16 |
4.416 | torch-compile | 4.352 | 0.9855 |
linear-bias-float16 |
4.416 | torch-compile | 4.352 | 0.9855 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
cnn-feat-broadcast-float32 |
23.888 | torch-compile | 23.553 | 0.9860 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
30.784 | torch-compile | 30.432 | 0.9886 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
attn-causal-mask-float32 |
62.416 | torch-compile | 61.713 | 0.9887 |