Skip to content

v1.3.1 — Multi-GPU Auto-Detect Build (T4 / A100 / L4 / RTX / H100)

Choose a tag to compare

@KiyoEditz KiyoEditz released this 24 Sep 22:42
· 22 commits to master since this release

Added

  • Fat multi-arch CUDA build: Compiles native binaries for T4 (sm_75), A100 (sm_80), RTX 30-series (sm_86), L4/RTX 40-series (sm_89), and H100 (sm_90), plus an sm_90 PTX forward-compatibility entry.
  • Runtime GPU auto-detection: Collab-Llama.py and Kaggle-Llama.py now detect the attached GPU architecture automatically via nvidia-smi—no manual CUDA_ARCH editing required.
  • Coverage-aware prebuilt verification: Tarball verification now checks if the attached GPU is natively supported or JIT-compatible before extraction, falling back to source build if not.
  • Enhanced VERSION.txt metadata: Added cuda_archs and cuda_archs_human for quick inspection.

Fixed

  • Shell argument truncation: Quoted -DCMAKE_CUDA_ARCHITECTURES="{CUDA_ARCH}" to prevent semicolons from splitting cmake arguments into separate shell commands.
  • CUDA driver library resolution: Expanded search paths to include container paths (/usr/local/nvidia/lib64, /usr/local/nvidia/lib), LD_LIBRARY_PATH, and versioned CUDA directories on Colab and Kaggle.
  • VMM fallback configuration: Cleaned up fallback flags to properly bypass CUDA::cuda_driver linkage when VMM is disabled.

Changed

  • Asset naming updated to llama-cuda-multiarch-{tag}.tar.gz (one package serves all supported GPUs).
  • GITHUB_RELEASE_URL in the notebook scripts should be updated to point to this new multi-arch asset.