Repository navigation
Perf Tracker 2026 11 11
Nightly run 42860420524, commit 9bf4bf56c, NVIDIA H200, CUDA 13.2, driver 580.178.04, torch 2.13.0+cu132, TileLang 0.1.12, image ghcr.io/tile-ai/tileops-ci-runner:local-runner-2.338.2. Times are CUPTI device_busy_ms.
A row is behind when its fastest comparator's time over TileOPs' time is below 0.99. A row whose fastest comparator takes under 10 us is behind only if it was also below that ratio the night before. An op is labelled by its worst row: P0 < 0.50, P1 < 0.80, P2 < 0.90, P3 < 0.95, P4 < 0.99.
| Measure | Value |
|---|---|
| Ops behind | 10 of 206 |
| Rows behind | 11 of 2337 |
| By label | P0 0, P1 0, P2 0, P3 0, P4 10 |
| Failed rows | 0 |
Since nightly run 42660464876:
- Newly behind:
AbsFwdOp,ErfFwdOp,ExpFwdOp,GEMMW4A16FwdOp,MaskedFillScalarFwdOp,WhereFwdOp - Caught up:
BitwiseAndFwdOp,FloorFwdOp,GEMMFwdOp - Not measured tonight: none
Nightlies behind counts the consecutive nightlies, up to 30, in which the op had a row behind.
| # | Op | Category | Label | Worst ratio | Rows behind | Nightlies behind |
|---|---|---|---|---|---|---|
| 1 | SoftmaxFwdOp | reduction | P4 | 0.9744 | 2 of 42 | 8 |
| 2 | GroupedGEMMFwdOp | gemm | P4 | 0.9792 | 1 of 10 | 7 |
| 3 | ErfFwdOp | elementwise | P4 | 0.9829 | 1 of 5 | 1 |
| 4 | MaskedFillScalarFwdOp | elementwise | P4 | 0.9887 | 1 of 8 | 1 |
| 5 | GEMMW4A16FwdOp | gemm | P4 | 0.9888 | 1 of 15 | 1 |
| 6 | ClampTensorFwdOp | elementwise | P4 | 0.9892 | 1 of 18 | 2 |
| 7 | AbsFwdOp | elementwise | P4 | 0.9896 | 1 of 5 | 1 |
| 8 | ExpFwdOp | elementwise | P4 | 0.9896 | 1 of 5 | 1 |
| 9 | SigmoidFwdOp | elementwise | P4 | 0.9896 | 1 of 5 | 2 |
| 10 | WhereFwdOp | elementwise | P4 | 0.9898 | 1 of 8 | 1 |
Benchmark benchmarks.ops.bench_softmax.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
quack-throughput-32768-float16 |
1051.060 | quack | 1024.180 | 0.9744 |
quack-throughput-32768-bfloat16 |
1078.610 | quack | 1065.040 | 0.9874 |
Benchmark benchmarks.ops.bench_grouped_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
qwen3-30b-ep8-skew-down-bfloat16 |
42.336 | quack | 41.456 | 0.9792 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
31.872 | torch | 31.328 | 0.9829 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
33.919 | torch-compile | 33.536 | 0.9887 |
Benchmark benchmarks.ops.bench_gemm.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
prefill-mlp-up-float16 |
103.857 | torch-dequantized-matmul | 102.689 | 0.9888 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
cnn-feat-channel-bounds-float32 |
23.680 | torch-compile | 23.425 | 0.9892 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
30.784 | torch-compile | 30.464 | 0.9896 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
30.880 | torch-compile | 30.560 | 0.9896 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
16M-float32 |
30.880 | torch-compile | 30.560 | 0.9896 |
Benchmark benchmarks.ops.bench_elementwise.
| Workload | TileOPs (us) | Fastest comparator | Comparator (us) | Ratio |
|---|---|---|---|---|
attn-causal-mask-float32 |
62.464 | torch-compile | 61.824 | 0.9898 |