llamafile v0.10.5

What's Changed

0.10.5 is a release that focused on making the "sync with upstream llama.cpp" task simpler and better. In the span of 2 weeks we performed three syncs and took advantage of these iterations to improve the llamafile skill/plugin for AI agents. The result is a better process that provides more reliable code, requiring fewer iterations to go from an auto-generated draft PR to the final merged code. As a byproduct, llamafile now supports very recent models such as Ternary Bonsai 27B and Laguna-S-2.1.

Thanks to our community's contributions, this release also brings significant documentation updates clarifying the new help system, the difference between different lllamafile executables, and up-to-date llamafile GPU support. Many thanks everyone for the huge help!

• Update llama.cpp to b10052 (b2dd28a) by @aittalam in #1021
• Update llama.cpp to b10083 (846e991) by @aittalam in #1026
• Improve llamacpp update skill by @aittalam in #1027
• Update llama.cpp to b10103 (c588c4f) by @aittalam in #1030
• docs(quickstart): clarify llamafile vs llamafile-thin and other release binaries (#1022) by @Anai-Guo in #1023
• build(install): install transcribefile so it ships as a release artifact (#1035) by @Anai-Guo in #1036
• Fix typo in README.md for 'architectures' by @shawntax in #1031
• docs: Add docs for cli args and tools by @angpt in #1037
• docs(support): document the Vulkan GPU backend (#1029) by @Anai-Guo in #1033

New Contributors

• @shawntax made their first contribution in #1031

Full Changelog: 0.10.4...0.10.5

llamafile v0.10.4

What's Changed

The most exciting feature that this release of llamafile brings is the first version of transcribefile! 🎉 Our transcribefile is nothing more than a cosmocc-build of the CLI tool from transcribe.cpp, @cjpais' ggml-based Speech-to-Text library that supports 16+ model families 💙 For more details, see CJ's post and Mozilla.ai's annoucement.

In addition to this, a new sync with upstream llama.cpp (which will be soon replaced by a more recent one, as I let too much time pass before the release 😬); few improvements to Vulkan and ROCm acceleration; HTTPS support (both for the server and to download models from HF given their IDs); and pledge/SECCOMP sandboxing.

Thanks everyone for improving llamafile with issues and PRs!

• Probe GPU device count out-of-process on Windows (#988 follow-up) by @aittalam in #994
• docs: redirect GitHub Pages docs to GitBook by @angpt in #993
• ROCm shell script improvements by @aittalam in #995
• docs(support): add GPU backend status table & acceleration verification (#996) by @Anai-Guo in #997
• skill: guardrail llama.cpp update workflow with patch commands by @aittalam in #998
• Update llama.cpp to b9747 (d6d8995) by @aittalam in #1000
• chore: upgrade and digest pin all action digests by @RichardoC in #1002
• CPU / GPU fixes + harness for upstream ggml's test-backend-ops by @aittalam in #1004
• vulkan: clearer message when --gpu vulkan is explicitly unavailable by @aittalam in #1006
• Fixes CTRL+C bug in chat mode by @aittalam in #1009
• transcribefile: CPU build of transcribe.cpp + sync to df1a4ad by @aittalam in #1007
• Updated runner to 16-core by @aittalam in #1005
• transcribefile: Metal GPU support via llamafile's runtime loader by @aittalam in #1010
• Add HTTPS support via cpp-httplib's mbedtls backend by @aittalam in #1011
• feat(help): list all accepted llama.cpp options in --help by @RichardoC in #1003
• Bring back pledge/SECCOMP sandboxing (#930) by @aittalam in #1015

New Contributors

• @Anai-Guo made their first contribution in #997
• @RichardoC made their first contribution in #1002

Full Changelog: 0.10.3...0.10.4

llamafile v0.10.3

What's Changed

This release is mainly a bugfix to a regression introduced in 0.10.2 (many many thanks to @niutech for opening the issue! 🙏)

As releases become more frequent, we are working to better automate the process and making this more open. This PR brings a new llamafile/release.sh script that's the one used to create all the new release files (we'll be testing it until the next release before updating the RELEASE.md doc).

• Update release scripts by @aittalam in #990
• Fix uncaught SIGSEGV when GPU init fails, restore CPU fallback (#988) by @aittalam in #989

Full Changelog: 0.10.2...0.10.3

llamafile v0.10.2

What's Changed

• Add gpu accel probe by @aittalam in #953
• Added executable dir to list of dylib valid paths by @aittalam in #959
• Update README with Windows executable size note by @ParthProLegend in #962
• Update llama.cpp submodule to 7b8443ac7 by @aittalam in #951
• Reduce cuda dylibs size by @aittalam in #963
• Clarify Linux GPU offload diagnostics by @adminlip in #967
• docs: rename example_llamfiles to pre-built-llamafiles for better seo by @angpt in #972
• ggml-cpu: re-hook llamafile MoE matmul on x86 (partial fix for #973) by @aittalam in #974
• CPU flash-attention fixes for #975 (workaround + AVX-512 helpers + simd_gemm) by @aittalam in #978
• common.cpp: fix Apple Silicon physical-core detection under cosmocc (#976) by @aittalam in #977
• Updated version to 0.10.2 by @aittalam in #984
• Modernise Diffusionfile Support by @cbk-res in #970
• Update llama.cpp submodule to dbe9c0c (+ embed real web UI) by @aittalam in #983

New Contributors

• @ParthProLegend made their first contribution in #962
• @adminlip made their first contribution in #967
• @cbk-res made their first contribution in #970

Full Changelog: 0.10.1...0.10.2

llamafile v0.10.1

What's Changed

Summary:

• Added support for vulkan dylibs
• Added windows build scripts -> we now have cuda, rocm, vulkan both as .so and as .dll libraries
• Updated llama.cpp submodule to 5e9c63546 -> we now have llama.cpp support for new models (e.g. gemma-4, bonsai, Qwen3.6) and functionalities (e.g. llama.cpp internal tools for agents)

Details:

• Fix README_0.10.0 by @aittalam in #918
• Add support for vulkan dylibs by @aittalam in #892
• Add tinyblasStrsmBatched kernel by @aittalam in #923
• Fix: GGUF Q5_1 quant crashes llamafile on aarch64 cpu by @aittalam in #928
• fix broken link 'feel free to choose' -> example llamafiles by @bquast in #927
• Fix block-size assumption by @aittalam in #935
• Add windows build scripts for CUDA by @aittalam in #924
• Fix whisperfile documentation link in README by @martin0258 in #939
• Update llama.cpp submodule to 5e9c63546 by @aittalam in #941
• Migrate docs from MkDocs/GitHub Pages to GitBook by @angpt in #946
• Win build improvements by @aittalam in #940
• Fix cuda: /lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.32' not found` by @aittalam in #948
• Docs Updates by @angpt in #949
• New release version: update version.h and docs by @aittalam in #952

New Contributors

• @bquast made their first contribution in #927
• @martin0258 made their first contribution in #939
• @angpt made their first contribution in #946

Full Changelog: 0.10.0...0.10.1

llamafile v0.10.0

llamafile versions starting from 0.10.0 use a new build system, aimed at keeping our code more easily
aligned with the latest versions of llama.cpp. This means they support more recent models and functionalities,
but at the same time they might be missing some of the features you were accustomed to (check out this doc for a high-level description of what has been done).

If you liked the "classic experience" more, you will always be able to access the previous versions from our releases page. Our pre-built llamafiles show which version of the server they have been bundled with (0.9.* example, 0.10.* example), so you will always know which version of the software you are downloading.

What's Changed

• Accept array in chat message content field by @henfiber in #760
• chore: Update README.md to include call for community feedback on llamafile by @njbrake in #812
• chore: integrate whisper.cpp as a submodule by @njbrake in #813
• chore: convert stable diffusion to submodule by @njbrake in #818
• chore: llama.cpp as submodule by @njbrake in #819
• feat: move docs to mkdocs by @njbrake in #824
• chore: Add update-llama-cpp workflow. by @daavoo in #846
• fix(update-llama-cpp): Use new_build_wip as base ref. by @daavoo in #850
• Fixed broken llamafile URL in docs by @aittalam in #873
• update supported OpenBSD versions by @sthen in #897
• llamafile reloaded (v0.10.0) by @aittalam in #867

New Contributors

• @henfiber made their first contribution in #760
• @njbrake made their first contribution in #812
• @daavoo made their first contribution in #846
• @sthen made their first contribution in #897

Full Changelog: 0.9.3...0.10.0

llamafile v0.9.3

What's Changed

• Fix link to troubleshooting guide by @rsanheim in #740
• Preserve URL path when building relative URLs in JS by @dmcardle in #733
• Add Plaintext output option to LocalScore + Respect NO_COLOR env var by @cjpais in #742
• Update README.md, fix llama 8B table stats by @cjrh in #745
• Add phi4 support by @cjpais in #744
• Qwen3 Support by @cjpais in #743

New Contributors

• @rsanheim made their first contribution in #740
• @dmcardle made their first contribution in #733
• @cjrh made their first contribution in #745

Full Changelog: 0.9.2...0.9.3

llamafile v0.9.2

Llamafile

Llamafile v0.9.2 is a significant release. It adds support for:

• DeepSeek Distil R1 Models
• Gemma 3
• IBM Granite

LocalScore

In addition 0.9.2 introduces LocalScore, a benchmarking utility and website.

LocalScore is an open-source tool that both benchmarks how fast Large Language Models (LLMs) run on your specific hardware and serves as a repository for these results. We created LocalScore to provide a simple, portable way to evaluate computer performance across various LLMs while making it easy to share and browse hardware performance data.

LocalScore is now part of the release of Llamafile under the new CLI utility localscore

You can run it ./localscore -m <model>. It is also included in every llamafile so you can benchmark models on your hardware easily using ./llamafile --localscore.

LocalScore was created with support from Mozilla Builders.

What's Changed

• [llamafiler] doc/v1_chat_completions.md: remove duplicate entry by @mseri in #607
• Update server readme with code completion (FIM) example by @heaversm in #637
• URL constructor to get a clean url_prefix (fix #640) by @sizvix in #641
• Fix translation bug from cpp to js in TS highlight by @emilbayes in #681
• Add whisperfile server documentation by @alonsosilvaallende in #707
• Unify button look and rearrange buttons to make them more compact by @corebonts in #712
• add stable-diffusion.cpp to install target (fix #580) by @rgroesslinger in #635
• Improve OpenAI compatibility for /v1/* endpoints by @corebonts in #713
• Update WSL troubleshooting in README.md by @halter73 in #585
• Granite three support by @gabe-l-hart in #608
• Initial support for Gemma 3 models by @corebonts in #717
• Add copy and info buttons to the chat window and improve small screen UX by @corebonts in #719
• Avoid streaming incomplete UTF-8 characters by @corebonts in #727
• Introduce LocalScore CLI by @cjpais in #734

New Contributors

• @mseri made their first contribution in #607
• @heaversm made their first contribution in #637
• @sizvix made their first contribution in #641
• @emilbayes made their first contribution in #681
• @alonsosilvaallende made their first contribution in #707
• @corebonts made their first contribution in #712
• @rgroesslinger made their first contribution in #635
• @halter73 made their first contribution in #585
• @gabe-l-hart made their first contribution in #608

Full Changelog: 0.9.1...0.9.2

llamafile v0.9.1

This release adds support for DeepSeek Distil models. It improves some documentation, and fixes a segfault when running with an Nvidia GPU.

What's Changed

• Update Makefile: Fix PHONY from check to cosmocc and cosmocc-ci respectively by @mofosyne in #683
• Updated README to reflect WSL 2 command for Windows 11 by @peteski22 in #685
• Add Support for DeepSeek-R1 models by @Xydane in #687
• Revert Cosmopolitan to 3.9.7 by @cjpais in #705

New Contributors

• @peteski22 made their first contribution in #685
• @Xydane made their first contribution in #687

Full Changelog: 0.9.0...0.9.1

llamafile v0.9.0

We've solved all the known issues with the new llamafiler server, and
improved its web gui. In addition to the llamafiler binary below, the
new server is available as llamafile --server --v2. Its manual can be
accessed via llamafile --server --v2 --help.

• e64c7e2 Include llamafiler in llamafile binary
• a8fd4d2 Improve management of multiple slots
• 4158265 Show progress bar for prompt processing in web ui
• 38677b5 Support relocating matching suffixes in KV cache
• 956e62c Visually indicate messages truncated by context
• 08e7a21 Forget old messages when running out of context
• 59a5d97 8fa1702 Make pledge() security not break things
• 1fc35e2 Add upload button and support text files
• 43bc1eb fe514ef Improve buttons in web ui

This change upgrades our cosmocc toolchain, whose recent release has
fixed all known issues and made performance improvements to memory
allocation. See the cosmopolitan releases page.

• c293359 Upgrade to Cosmopolitan v4.0.2

The following improvements have been made to the terminal --chat bot.

• f51e535 Fix emoji editing in chatbot

Additional changes:

• 9b03e32 Call appropriate hip api's (#651)