v0.16.0

New features

Preliminary support for signing

This release adds preliminary support for signing kernels. Kernel signing is currently in an experimental phase and the details may still change. For this reason, signatures are currently not yet validated when downloading a kernel. Kernel verification consists of two parts:

• A list of files and their hashes (digest) is added to a kernel's metadata.json. During verification, all files must be present and have the correct hash. Verification also fails when there are files that are not specified in the digest.
• The (digest inside the) metadata is protected using a detached signature in metadata.json.sigstore. The signature is made using sigstore, which signs artifacts using ephemeral signing keys, reducing the impact of key theft. During verification, the signature is used to validate that metadata.json was not tampered with and that the signature was made from a trusted repository and workflow.

Since the kernel is not verified yet during retrieval during the experimental stage, we provide the kernels verify-signature command-line utility that can be used to verify a kernel on the Hub. The kernel and kernel version to verify should be provided as arguments:

$ kernels verify-signature kernels-community/flash-attn4 0
✅ torch-cuda: kernel metadata is correctly signed

All kernels-community kernels are signed. If you want to experiment with kernel signing yourself, there are two changes you need to make:

• Run nix flake update to get the latest version of kernel-builder. The latest version embeds the kernel digest in the metadata.
• Update your build workflow to run cosign to sign the kernel. This cannot be done as part of the build itself, since signing using ephemeral kernels requires internet access and the kernel build sandbox does not provide internet access. You can use the kernels-community workflow as an example of how to set up metadata signing.

CPU kernel skill

kernel-builder now supports the cpu-kernels skill for writing, optimizing, and benchmarking C++ kernels using AVX2/AVX512. For example, to add the skill to Claude, use:

$ kernel-builder skills add --skill cpu-kernels --claude

Simplified handling of kernel functions

use_kernel_func_from_hub, FuncRepository, LocalFuncRepository, and LockedFuncRepository are now deprecated.

To make a function extensible by a layer, you can now use the same decorator as for layers (use_kernel_forward_from_hub). This makes it clearer that the function is actually replaced by a layer. We have also added the use_kernelized_func decorator to attach such a function to the layer wherein it is used to make it discoverable by kernelize. Here is a full example:

# Make silu_and_mul replaceable with a kernel layer registered as `silu_and_mul`.
@use_kernel_forward_from_hub("silu_and_mul")
def silu_and_mul(x: torch.Tensor) -> torch.Tensor:
    d = x.shape[-1] // 2
    return F.silu(x[..., :d]) * x[..., d:]

# Attach the function to the layer where it is used to make it discoverable by `kernelize`.
@use_kernelized_func(silu_and_mul)
class FeedForward(nn.Module):
  def __init__(self, in_features: int, out_features: int):
      self.linear = nn.Linear(in_features, out_features)

  def forward(self, x: torch.Tensor) -> torch.Tensor:
      return silu_and_mul(self.linear(x))

The FuncRepository, LocalFuncRepository, and LockedFuncRepository classes will not be replaced. They allowed using an arbitrary function from a kernel as a layer. However, this was easily misused and did not have a clean way of marking such a function as supporting torch.compile or backwards passes. Going forward, they should be made available as regular kernel layers that can be used with LayerRepository and its local/locked versions.

For more information, see the layer documentation.

Curated extra dependencies

Kernels support a small set of curated Python dependencies, such as einops, nvidia-cute-dsl, and apache-tvm-ffi. These dependencies are now also provided as extras of the kernels package, curated for CUDA and curated-xpu for XPU:

# CUDA
$ pip install 'kernels[curated]'
# XPU
$ pip install 'kernels[curated-xpu]'

This can be used to install all dependencies that a kernel might use.

Documentation improvements

kernel-builder architecture overview

The documentation now provides an overview of the kernel-builder architecture.

What's Changed

• [docs] add a pinning note. by @sayakpaul in #604
• [ci] don't trigger test_kernels on changes to builder by @sayakpaul in #594
• Set version to 0.16.0.dev0 by @danieldk in #603
• [docs] add an overview document for builder design by @sayakpaul in #606
• vouch: vasqu by @danieldk in #610
• [FuncRepository] Add ability of detecting flags by @vasqu in #607
• [ci] add concurrency by @sayakpaul in #600
• ci: build forCache to ensure that we have every variant cached by @danieldk in #615
• [packaging] curated package installation by @sayakpaul in #613
• [core] feat: refactor validate dependency by @sayakpaul in #609
• Add jiqing-feng to the vouch list by @danieldk in #617
• python3Packages.triton(-xpu): update hashes to final release by @danieldk in #619
• kernel-builder: add hash subcommand and hook up in Nix by @danieldk in #618
• Improve security of handling pyc files by @danieldk in #623
• nix-builder: remove the MKL override by @danieldk in #624
• Unify bytecode handling across downloads/builds/hashing by @danieldk in #625
• [docs] remove stale entry from cli doc. by @sayakpaul in #608
• Add CPU kernel skills by @jiqing-feng in #614
• Add source to metadata by @drbh in #572
• feat: check latest symbol files and update if necessary by @sayakpaul in #629
• add device properties to the API reference. by @sayakpaul in #628
• nix-builder: update to final 2.12.0 by @danieldk in #633
• [ci] remove unsupported 3.13t for maturin by @sayakpaul in #638
• feat: implement load telemetry. by @sayakpaul in #637
• Add support for verifying kernels by @danieldk in #635
• Document convention to use version 0 for alpha/beta kernels by @danieldk in #639
• Use binary wheels for triton-rocm by @danieldk in #616
• Revert "feat: implement load telemetry." by @sayakpaul in #641
• Modify workflow to refresh ABI symbols and add token step by @paulinebm in #642
• kernels: tighten up code signing security by @danieldk in #644
• kernel-abi-check: refresh vendored ABI symbol files by @hf-kernels-bot[bot] in #643
• Add acknowledgment for the xpu-kernels skill / Xe-Forge by @jiqing-feng in #627
• [docs] add an example usage of get_kernel_variants by @sayakpaul in #631
• skills: update cuda skills to make the generated kernels compliant with the builder by @sayakpaul in #630
• kernels: verify using the signer URIs by @danieldk in #649
• don't close a PR from an unvouched contributor when it's opened by a maintainer by @sayakpaul in #646
• kernels: add verify-signature subcommand by @danieldk in #650
• add a contributing doc. by @sayakpaul in #647
• ci: run audits on PRs and update to Opus 4.8 by @danieldk in #654
• kernels: do not skip malformed bundles when using --filter-unsigned by @danieldk in #653
• kernels: also require OIDCSourceRepositoryURI by @danieldk in #652
• ci: better gating in the security workflow and be more explicit about untrusted user input by @danieldk in #656
• Support pyext in torch-noarch by @danieldk in #658
• kernel-builder hash: do not try to join two full paths by @danieldk in #663
• feat: provide helpful guidance to users when they don't have kernel repo publishing rights by @sayakpaul in #662
• kernels: allow kernels signed by the migration workflow by @danieldk in #664
• [tests] fix get variant tests. by @sayakpaul in #667
• feat: add docs for building on hf jobs vis gh actions by @drbh in #636
• python3Packages.triton: fix ldconfig use by @danieldk in #670
• [docs] update docs structure by @stevhliu in #645
• chore: update security-audit.yml by @hf-security-analysis[bot] in #568
• chore: enable Dependabot weekly GitHub Actions bumps by @hf-dependantbot-rollout[bot] in #585
• feat: enable cutlass 4.5 on nix builder by @mfuntowicz in #665
• Support different ptxas version than CUDA version by @danieldk in #671
• docs: remove Discord links by @danieldk in #673
• kernel-builder: support specifying backend archs for noarch kernels by @danieldk in #668
• Simplify handling of kernelized functions by @danieldk in #622

New Contributors

• @vasqu made their first contribution in #607
• @jiqing-feng made their first contribution in #614
• @hf-kernels-bot[bot] made their first contribution in #643
• @stevhliu made their first contribution in #645
• @hf-security-analysis[bot] made their first contribution in #568
• @hf-dependantbot-rollout[bot] made their first contribution in #585

Full Changelog: v0.15.1...v0.16.0