Repository navigation
Perf Tracker 2026 11 09
Nightly run 42502722407, commit ba31dd42f, NVIDIA H200, CUDA 13.2, driver 580.178.04, torch 2.13.0+cu132, TileLang 0.1.12, image ghcr.io/tile-ai/tileops-ci-runner:local-runner-2.338.2. Times are CUPTI device_busy_ms.
A row is behind when its fastest comparator's time over TileOPs' time is below 0.99. A row whose fastest comparator takes under 10 us is behind only if it was also below that ratio the night before. An op is labelled by its worst row: P0 < 0.50, P1 < 0.80, P2 < 0.90, P3 < 0.95, P4 < 0.99.
| Measure | Value |
|---|---|
| Ops behind | 9 of 206 |
| Rows behind | 12 of 2337 |
| By label | P0 0, P1 0, P2 0, P3 1, P4 8 |
| Failed rows | 0 |
Since nightly run 42394161005:
- Newly behind:
BMMFwdOp,DivFwdOp,ErfFwdOp - Caught up:
GtFwdOp,MaxPool2dIndicesFwdOp,MoEGroupedGEMMFP8FwdOp,NeFwdOp,ReciprocalFwdOp,WhereFwdOp - Not measured tonight: none
Nightlies behind counts the consecutive nightlies, up to 30, in which the op had a row behind.
| # | Op | Category | Label | Worst ratio | Rows behind | Nightlies behind |
|---|---|---|---|---|---|---|
| 1 | DivFwdOp | elementwise | P3 | 0.9232 | 1 of 20 | 1 |
| 2 | GEMMFwdOp | gemm | P4 | 0.9654 | 2 of 27 | 6 |
| 3 | GroupedGEMMFwdOp | gemm | P4 | 0.9778 | 1 of 10 | 5 |
| 4 | SoftmaxFwdOp | reduction | P4 | 0.9786 | 2 of 42 | 6 |
| 5 | RMSNormFwdOp | norm | P4 | 0.9818 | 1 of 34 | 7 |
| 6 | ErfFwdOp | elementwise | P4 | 0.9849 | 1 of 5 | 1 |
| 7 | LogicalOrFwdOp | elementwise | P4 | 0.9853 | 2 of 18 | 5 |
| 8 | BMMFwdOp | gemm | P4 | 0.9857 | 1 of 16 | 1 |
| 9 | RemainderFwdOp | elementwise | P4 | 0.9866 | 1 of 14 | 6 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
ragged-chan-broadcast-float16 |
15.008 | torch-compile | 13.856 | 0.9232 |
Benchmark benchmarks.ops.bench_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
ds-v3-m96-qkv-a-bfloat16 |
14.784 | torch-cublas | 14.272 | 0.9654 |
qwen3-8b-gate-up-b32-bfloat16 |
49.072 | cublaslt-best | 48.096 | 0.9801 |
Benchmark benchmarks.ops.bench_grouped_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
qwen3-30b-ep8-skew-up-bfloat16 |
82.240 | quack | 80.417 | 0.9778 |
Benchmark benchmarks.ops.bench_softmax.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
quack-throughput-512-bfloat16 |
16.480 | quack | 16.128 | 0.9786 |
quack-throughput-2048-bfloat16 |
63.584 | quack | 62.704 | 0.9862 |
Benchmark benchmarks.ops.bench_norm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
quack-throughput-256-float16 |
8.768 | torch-compile | 8.608 | 0.9818 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
31.776 | torch | 31.296 | 0.9849 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
linear-bias-float16 |
4.416 | torch-compile | 4.351 | 0.9853 |
linear-bias-bfloat16 |
4.416 | torch-compile | 4.352 | 0.9855 |
Benchmark benchmarks.ops.bench_bmm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
square-512-float16 |
11.168 | quack | 11.008 | 0.9857 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
cnn-feat-broadcast-float32 |
23.872 | torch-compile | 23.553 | 0.9866 |