Skip to content

dasllama-v0.6.7

Choose a tag to compare

View all tags
@borisbat borisbat released this 31 Oct 03:48
· 364 commits to master since this release
dasllama-v0.6.7
56d13cf
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

dasllama 0.6.7 - the row-major race profiles.

The OpenAI-compatible server, the command line, the benchmark and the watchdog, as one package per platform, every [tune] kernel shipped as one clone per CPU class from the release-17 class profiles (arm-neon, arm-i8mm, arm-sve, x86-avx2, x86-vnni512, x86-amx), every one of them re-minted on the row-major GEMV race.

Since 0.6.6: the CUDA home serves host experts, the NextN head and the K/V ring (--gpu cuda), the Vulkan tier serves wave64 devices (Radeon on RADV, pinned at 32 lanes), a box whose class profile covers it only partly mints the rest at first start, and the fat exe starts on every class profile.

Install from the assets below, or through the package channels:

  • brew install --cask borisbat/daslang/dasllama - the Mac app
  • brew install borisbat/daslang/dasllama - the commands, macOS and Linux
  • scoop bucket add daslang https://github.com/borisbat/scoop-daslang then scoop install dasllama
  • pip install dasllama
  • the .deb for Debian and Ubuntu, or the .rpm for Fedora, on this page

The archives also refresh the rolling dasllama-server release. Benchmarks: https://dasllama.io/ladder.html

Assets 26
Loading