Skip to content

Releases

cti-kernels 0.0.1

Choose a tag to compare

@JerryGJX JerryGJX released this 24 Oct 05:39

First public release of cti-kernels: PyTorch GPU operations for continuous training and inference. See the README for modules, dependencies and supported devices.

Assets:

  • cti_kernels-0.0.1-py3-none-any.whl. Check its SHA256 against the digest GitHub lists for the asset. Install it with python -m pip install "./cti_kernels-0.0.1-py3-none-any.whl[cute,gdn]".
  • flash_attn-2.8.3.post1-cp312-cp312-linux_x86_64.whl, sha256 3e8f6edd3c2f20ec3a668f7c60c6da8ec0adfb5517979bb1ee0b50e06927e3b7.
  • flash_attn-2.8.3.post1-cp312-cp312-linux_aarch64.whl, sha256 d29013ab8bbcfab10f1df2b9e0d834f5ff807e6d4004fd1b0a4ec5c0bb235493.

The flash-attn wheels serve the varlen extra:

  • Built for Python 3.12, Torch 2.13.0 and CUDA 13.0.
  • They contain sm_120 code only, so they run on RTX 5090, RTX PRO 6000 and GB10 and on no other GPU.
  • Tested: x86_64 on RTX PRO 6000 (varlen forward and backward, BF16 and FP16, causal and non-causal); aarch64 on GB10 (varlen).
  • On other setups, build flash-attn from source.

Pins: torch 2.13.0, triton >=3.7.1,<3.8, flashinfer-python 0.7.0.post1, quack-kernels 0.6.5. Extras: cute = nvidia-cutlass-dsl[cu13] 4.8.0, gdn = fla-core 0.5.2, varlen = flash-attn 2.8.3.post1.

Gated devices: RTX 5090, RTX PRO 6000 Server Edition and GB10.

License: MIT. Adapted third-party code keeps its upstream license; see THIRD_PARTY_NOTICES.