Skip to content

v0.17.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 14 Sep 09:20
· 16 commits to main since this release
v0.17.0
7a11103

New features

Device architecture checks

get_kernel now raises an exception if the kernel does not support the architecture of the GPU in the machine. For instance, if a kernel is only compiled for Hopper (CUDA capability 9.0) is loaded on a Blackwell GPU with CUDA capability 12.0 an exception is raised. Previously, an incompatible kernel would load fine and then (loudly or quietly) fail at kernel launch time.

In some cases, an AOT kernel may support more architectures than the metadata declares, for instance when the kernel has a Triton fall-back path. For such kernels, the check can be disabled:

from kernels import get_kernel, has_kernel

has_kernel("kernels-community/flash-attn3", version=1, check_arch=False)
flash_attn3 = get_kernel("kernels-community/flash-attn3", version=1, check_arch=False)

Kernel dependencies (experimental)

Kernels can now depend on other kernels from the Hub. Dependencies are declared in the general section of build.toml. For example:

[general]
# ...
kernel-depends = [
    { repo-id = "kernels-community/activation", version = 1 },
    { repo-id = "kernels-community/einops", version = 1 },
]

The kernel dependencies can than be retrieved when the kernel is imported using the new get_kernel_dep function:

import kernels

activation = kernels.get_kernel_dep("kernels-community/activation")
einops = kernels.get_kernel_dep("kernels-community/einops")

Warning: using kernel dependencies breaks support for kernels<0.17. For this reason, we recommend you to only start using kernel dependencies when version 0.17.0 has been out for a while. The kernels minimum version metadata described below will be ported to kernels 0.16 and will help with this and future migrations.

kernels minimum version

To introduce new features more gracefully in the future, using newer kernels features (such as kernel dependencies) will write the minimum required version to the kernel metadata. The kernels client will use this to verify that it is compatible with the kernel and, if not, suggest what version of kernels to install.

Conditional kernelization of layers

use_kernel_forward_from_hub now accepts an optional condition argument. This condition is applied to the layer when kernelize is called. This is useful when a layer supports multiple configurations, but kernelization is only supported for one configuration:

@use_kernel_forward_from_hub(
    "SwiGLUMLP",
    condition=lambda module: module.config.hidden_act == "silu",
)
class MyMLP(nn.Module):
    ...

Experimental TPU backend

kernel-builder now has experimental support for building TPU torch_tpu/Pallas kernels. backends = ["tpu"] produces a torch-tpu noarch variant.

Intel Crescent Island (cri) support

The XPU/SYCL build supports Intel Crescent Island.

Helion support

The Helion DSL is now supported through the new helion Python dependency (python-depends = ["helion"]). See the blog post Helion x 🤗 HF Kernels: Building and Shipping Out-of-the-box Performant Kernels for more information.

Trusting specific repositories

kernels only loads kernels from a curated set of trusted publishers. Until now, the only way to load a kernel from another publisher was to opt in with trust_remote_code=True, which disables the check entirely. You can now pass a list of repository IDs instead, so that only the given repositories are allowed:

get_kernel(
    "some-other-org/my-kernel",
    version=1,
    trust_remote_code=["some-other-org/my-kernel"],
)

kernels info

The new kernels info subcommand describes a kernel from a Hub repo ID or a local path, printing its name, version, license, upstream/source repositories, Python dependencies, and supported backends:

$ kernels info kernels-community/activation
Repository: kernels-community/activation
Revision: v1
Name: activation
Version: 1
License: Apache-2.0
Upstream: -
Source: -
Python dependencies: -
Backends: cuda, metal

Build provenance and dirty builds

Kernel metadata now records the provenance of a build in a provenance object. The provenance information contains the Git commit of the kernel and the builder and whether both were in a clean or dirty state.

Backend-scoped stable ABI

kernels 0.15.1 added experimental support for the Torch stable ABI. The stable ABI was enabled by setting torch.stable-abi to the stable ABI version that the kernel should be compiled with. Unfortunately, the stable ABI is not complete enough yet to be supported on all backends. For this reason, we have decided to change the torch.stable-abi option to take a table with versions per-backend:

[torch.stable-abi]
cuda = "2.11"
rocm = "2.9"

Backends that are not listed are compiled without the stable ABI. This makes it possible to opt-in to the stable ABI per backend and use different stable ABI versions per backend.

Torch 2.13 and 2.14

This release adds support for Torch 2.13 and 2.14 and removes Torch 2.11 and 2.12.

Major documentation improvements

Breaking changes

  • stable-abi is now a table rather than a single Torch version. build.toml files can be migrated to edition 5 with kernel-builder update-build.
  • Device architecture checks are enabled by default. Kernels that support more architectures than they declare (e.g. through a Triton fallback) must be loaded with check_arch=False.
  • The compat module is not generated anymore. Kernel tests must use get_kernel instead of importing the kernel directly.

Packaging

kernels-data is not a separate Python package anymore, but is directly integrated into the kernels package.

What's Changed

  • [docs] update agents guide to use kernel-builder build subcommand by @sayakpaul in #677
  • doc: add a migration section for deprecated kernel functions by @danieldk in #679
  • Set version to 0.17.0.dev0 by @danieldk in #678
  • kernel-builder: support cxx-flags for Torch and tvm-ffi sources by @danieldk in #680
  • nix-builder: redesign kernel testing by @danieldk in #676
  • skills: update to use new testing methodology by @danieldk in #683
  • Add support for Torch 2.13 by @danieldk in #681
  • kernel-builder: ensure that HIP_ARCHITECTURES gets properly cleared by @danieldk in #684
  • feat: implement kernels info by @sayakpaul in #685
  • feat: scope stable abi to backend by @drbh in #682
  • nix-builder: some Torch 2.13 XPU fixes by @danieldk in #687
  • add link to pytorch registration of custom ops by @sayakpaul in #688
  • triton-kernels: add vendor-neutral Triton skill by @jaygala223 in #675
  • python3Packages.torch_2_13: rc10 -> rc15 by @danieldk in #697
  • Remove mention of kernels upload by @sayakpaul in #695
  • feat: append builder info to detect builds made with uncommitted changes in the builder by @sayakpaul in #666
  • fix: flat layers.py should also be documented in the kernel card. by @sayakpaul in #699
  • Update nixpkgs by @danieldk in #702
  • feat: add a page for links to blog posts by @drbh in #692
  • feat: implement kernel uploads through PRs. by @sayakpaul in #694
  • Upload should only look for cards inside the directory where variants… by @sayakpaul in #704
  • build(deps): bump the actions group across 1 directory with 15 updates by @dependabot[bot] in #689
  • chore: update security-audit.yml by @hf-security-analysis[bot] in #709
  • fix: show all variants in e2e test by @drbh in #708
  • Add Helion as a supported Python dependency by @sayakpaul in #706
  • vouch: add tengomucho by @drbh in #712
  • feat: warn for dirty builds. by @sayakpaul in #703
  • Add XPU guidance for register-spill-aware autotune pruning to the `xp… by @sywangyi in #690
  • feat: add ability to output upload results as json by @drbh in #716
  • fix incompatible variants problem on mac. by @sayakpaul in #726
  • Update to final Torch 2.13.0, cleanup by @danieldk in #732
  • add comment on trust_remote_code=True by @sayakpaul in #735
  • kernel-builder: stop generating the compat module by @danieldk in #736
  • kernel-builder: fix collision of static variables between kernels by @danieldk in #740
  • Add tip for kernel requirements reference by @sayakpaul in #730
  • use kernel-builder in actions and jobs by @sayakpaul in #738
  • Add CUTLASS 4.4.2 dependency and fix 4.5.2 source hash by @LiangSu8899 in #741
  • build(deps): bump the actions group across 1 directory with 11 updates by @dependabot[bot] in #728
  • kernel-abi-check: refresh vendored ABI symbol files by @hf-kernels-bot[bot] in #723
  • Add TPU backend support (torch_tpu / Pallas kernels) by @tengomucho in #714
  • python3Packages.nvidia-cutlass-dsl: 4.5.0 -> 4.6.1 by @danieldk in #739
  • Kernel loading untangling by @danieldk in #745
  • Untangle kernel loading, part2 by @danieldk in #746
  • Untangle kernel loading, part 3 by @danieldk in #747
  • chore: update security-audit.yml by @hf-security-analysis[bot] in #743
  • Add kernel-depends option to general options in build.toml by @danieldk in #749
  • include shutdown notice in migration notice by @sayakpaul in #759
  • [docs] add more detailed notes about versioning with layers. by @sayakpaul in #755
  • Request IEEE fp32 division for SYCL kernels by @jiqing-feng in #752
  • fix: pin hf-hub post create pr patch by @drbh in #761
  • kernels-data: replace GitHash by GitStatus by @danieldk in #765
  • kernel-abi-check: refresh vendored ABI symbol files by @hf-kernels-bot[bot] in #763
  • feat(nix-builder): add ARM64 target for CUDA backend on Windows by @mfuntowicz in #760
  • fetchFromHuggingFace: add by @danieldk in #766
  • kernel-builder: move build.toml parsing to kernels-data by @danieldk in #768
  • Add locking data structures to kernels-data by @danieldk in #771
  • feat: set the endpoint and token when building hf client by @drbh in #772
  • fix: avoid skipping dirty arg in gitstatus by @drbh in #767
  • Crescent Island (cri) support for the XPU/SYCL build by @jiqing-feng in #734
  • feat: add an example on using autotuning. by @sayakpaul in #754
  • Add kernel resolver infrastructure by @danieldk in #775
  • chore: document telemetry env var beter. by @sayakpaul in #781
  • Add kernel dependency tree resolver by @danieldk in #779
  • kernel-builder: support capabilities 10.3 and 12.1 by @danieldk in #783
  • Add support for Torch 2.14 (2.14-rc6) by @danieldk in #784
  • kernels: switch over to the dependency solver by @danieldk in #785
  • Small Torch 2.14 fixes by @danieldk in #787
  • kernel-builder: sycl-tla hotfix by @danieldk in #788
  • hotfix: fix sycl-tla typo by @danieldk in #790
  • kernel-builder: upload README to version branch as well by @danieldk in #791
  • nix-builder: wire up kernel dependencies by @danieldk in #786
  • kernels-data: strongly type version length by @danieldk in #794
  • {rocm,xpu}Packages: read version manifests from directory by @danieldk in #797
  • feat: implement capability check in main by @sayakpaul in #798
  • fix: cleanup failed imports by @drbh in #803
  • feat: support and check minver by @drbh in #801
  • nix-builder: add support for ROCm 7.14 by @danieldk in #807
  • kernels: make kernels version check a validator by @danieldk in #808
  • feat: allow passing a list of trusted repo ids. by @sayakpaul in #806
  • fix: point get-kernel-check at the built variant by @drbh in #802
  • fix: format nix arch files by @drbh in #810
  • remove extra-data example to save test build times. by @sayakpaul in #811
  • kernels: support conditions on use_kernel_forward_from_hub by @danieldk in #796
  • Small validator refactorings by @danieldk in #809
  • feat: implement user_agent in layers by @sayakpaul in #805
  • feat: turn dirty flag validation into a Validator. by @sayakpaul in #812
  • chore: move to using logger.warning from warnings.warn by @sayakpaul in #815
  • feat: suggest compatible variants when available in newer versions by @sayakpaul in #817
  • Update to Torch 2.14.0 and remove Torch 2.12. by @danieldk in #816
  • Move kernels-data Python binding into kernels by @danieldk in #818
  • Rename kernels-data to kernels-common by @danieldk in #821

New Contributors

Full Changelog: v0.16.1...v0.17.0

Contributors

danieldk, mfuntowicz, and 8 other contributors
Assets 62
Loading