v0.1.0 — Growing an operator library with agents and an AI compiler

TileOPs 0.1.0 is the first release of an operator library redesigned for the agentic era.

TileOPs is built with TileFoundry, an AI compiler designed for coding agents. Each of its 206 LLM operators starts as a YAML spec that declares the signature, the workloads and the roofline. Coding agents develop and tune the kernels, increasingly with TileFoundry, and the tests and benchmarks derived from the spec decide what enters the library. On H200, the result is at or ahead of the fastest available implementation on 2,262 of 2,265 benchmark rows.

This release fixes the public API: later 0.x releases only add to it.

pip install tileops==0.1.0

Highlights

• Built with TileFoundry. 195 merged PRs carry the foundry label: their kernels were generated with TileFoundry and passed the same CI as every other PR.
• 206 spec-defined operators. Every op has a manifest entry in tileops.manifest with its signature, the workloads used by tests and benchmarks, and a roofline formula. A validator checks each entry against its implementation in CI. (#2279, #2167)
• A stable public API. Ops are imported as tileops.<family>.<Op>. docs/stability.md lists the public modules, and scripts/ci/check_api_compat.py fails any pull request that breaks a call written against the latest release. (#2741, #2742, #2743, #2758)
• Ops follow their reference APIs. Each op names the API whose semantics it follows (ref_api), and backward ops are paired with their forward ops. (#2751)
• At or ahead of the fastest comparator on 2,262 of 2,265 benchmark rows (ratio ≥ 0.99), with a geometric mean of 1.37x on H200. See Performance.

Features

• torch.compile(fullgraph=True) works on every op whose manifest entry has a call-time tensor input and no composition. (#1749, #1751, #1752)
• CUDA Graph capture is validated on H200 for the ops in the graph test group.
• Ops autotune after op.request_tune(). Compiled kernels and tuning results are cached under TILELANG_CACHE_DIR and reused across processes.
• tileops.backend lets a backend distribution serve whole ops on its own device through register_kernel_builder. (#1883, #2743)
• tileops.authoring lets code outside TileOPs write a new op by subclassing Op with its own spec. (#2743)
• tileops.trace records an in-kernel timeline of a TileLang kernel. (#1640)

Performance

The nightly benchmark compares each TileOPs kernel with every other implementation of the same op on the same GPU. A row's ratio is the fastest comparator's time divided by TileOPs' time; a ratio above 1 means TileOPs is faster.

Family	Rows	Geometric mean	Rows ≥ 0.99
elementwise	527	1.09x	525
linear_attention	415	1.54x	415
attention	284	1.42x	284
gemm	235	1.28x	234
reduction	178	1.33x	178
moe	145	1.37x	145
norm	131	1.28x	131
quantization	71	1.45x	71
pool	65	1.51x	65
convolution	55	1.86x	55
mamba	54	1.61x	54
sampling	48	2.95x	48
rope	23	1.16x	23
sequence_modeling	17	4.00x	17
fft	10	1.23x	10
transform	7	1.28x	7
All	2,265	1.37x	2,262

Comparators include torch, torch.compile, FlashAttention 2 and 3, FlashInfer, FlashMLA, vLLM, DeepGEMM, flash-linear-attention, FlagGems, QuACK, Mamba, Marlin and cuBLASLt. The three rows below 0.99 trail torch.compile or the fastest GEMM comparator by 1.1–1.7%: Where and ClampScalar on 16M float32 elements, and the FP8 GEMM on the DeepSeek-V3 prefill output projection.

Measured by nightly run 40344177035 on the release commit: one H200, SM clock locked at 1500 MHz, CUDA 13.2, PyTorch 2.13, TileLang 0.1.12, device busy time with the L2 cache flushed before each call, 25 ms warmup and 100 ms of timed repeats. The benchmark dashboard shows every row.

Requirements

• Linux and an NVIDIA GPU with compute capability 9.0 (SM90), with its driver.
• CUDA Toolkit 13.2 with nvcc on PATH: TileLang compiles kernels at run time.
• Python 3.12 or 3.13. pip installs PyTorch 2.13, TileLang 0.1.12 and PyYAML from PyPI.

The release workflow builds the wheel twice and checks that the builds match, then installs that wheel from PyPI dependencies on H200 and runs it on Python 3.12 and 3.13 before publishing. (#2359, #2718, #2746)

Some kernels also have SM80, SM86 or SM89 paths. Those paths are tested when run on those devices but are not part of the release verification.

Known issues

• FusedAddLayerNormFwdOp loses precision on rows far from zero and overflows on rows near 1e36. (#2773)
• On a host with GPUs of different architectures, a kernel compiled for one GPU can run on another. (#2302)
• Bwd ops are called directly and do not plug into torch.autograd.
• An op instance supports one thread and one CUDA stream at a time. docs/stability.md lists the full support scope.

Looking ahead

0.1.0 is a starting point.

TileOPs and TileFoundry are being built together as one platform for the agentic era: an AI compiler that gives coding agents the hardware facts they need, and an operator library that agents grow on top of it. Kernel optimization is an iterative search, where an agent changes a kernel, measures it and decides the next step. TileFoundry grounds each step in facts: which instructions can implement an operation, what each choice costs in compute and memory traffic, and how far a kernel is from its roofline. With that feedback, agents converge on fast kernels instead of trying changes blindly, and writing and tuning operators becomes work that scales with the library.

The first results suggest how much room is left. Measured against the fastest of widely used libraries such as torch.compile, FlashAttention 3, FlashInfer, vLLM and DeepGEMM, TileOPs reaches a geometric mean of 1.37x over 2,265 benchmark rows on H200. These rows cover single operators on one GPU, but they indicate that well-established AI infrastructure still has performance to recover, and that compiler-guided agents are one way to recover it.

The next releases extend this approach from single operators toward the shape of real workloads:

1. Complex fused operators: fusion across operator boundaries, such as attention with its surrounding projections and normalization, whole MoE layers, and quantization inside GEMM epilogues.
2. Whole models: complete prefill and decode passes built from TileOPs, measured end to end against serving engines.
3. Compute–communication fusion: computation fused with collective communication across GPUs, such as tensor-parallel GEMM with all-reduce and expert-parallel MoE with all-to-all.

The compiler, the operator library and the way agents work with them will keep evolving from release to release. Feedback, issues and contributions are welcome on GitHub.

Thanks

Thanks to @FeathBow for 98 merged PRs in this release, including the SM89 kernel paths and fixes across attention, Mamba, GEMM and MoE.

Full changelog: CHANGELOG.md