dist : releases and versioning of ggml-org projects

[ggerganov]

Overview

Below is a summary of the release process of the ggml-org projects. The document should be useful for downstream project to decide how to build and distribute them.

Note

Work in progress - suggestions are welcome. The llama.cpp semantic versioning is not yet implemented

Projects

ggml-org/ggml

• Core library used by the rest of the projects

• Release builds use semantic versioning: vX.Y.Z

  • Git tags: vX.Y.Z

  Downstreams

  • Homebrew: ggml package, formula
  • Debian:
    • ggml package, packaging repository
    • ggml-cuda package (separate build because of non-free CUDA dependencies)

ggml-org/llama.cpp

• Includes an internal copy of ggml that may deviate from the official released version

• The internal copy is used for nightly development builds from the master branch

• The nightly builds use commit number [NUM] for version:

  • Git tags: b[NUM]
  • Build with CMake option -DLLAMA_BUILD_IS_DEV=ON

• Official release builds use semantic versioning: vX.Y.Z

  • Git tags: vX.Y.Z
  • Build with CMake option -DLLAMA_BUILD_IS_DEV=OFF
  • The release tags are created on the condition that the internal ggml copy is identical to the released ggml copy. This guarantees that a downstream building from the release tags and linking to a system-wide ggml will be compatible.

  Downstreams

  • ggml-org: llama.app webpage, packaging repository
    • Does not link to a system-level ggml (i.e. packs the internal ggml copy)
    • Publishes the nightly builds instead of the official releases (TODO: switch to official releases?)
  • Homebrew: llama.cpp package, formula
    • Depends on the ggml package and links to the installed system-wide ggml
    • TODO: how does the package manager know which version of ggml is required by llama.cpp? Right now it looks like it installs the latest ggml, which might not be compatible with the latest llama.cpp
  • Debian: llama.cpp package, packaging repository
    • Depends on the ggml and ggml-cuda packages and links to the installed system-wide ggml
  • Winget: llama.cpp workflow, manifests
    • Does not link to a system-level ggml (i.e. packs the internal ggml copy)
    • Publishes the nightly builds instead of the official releases (TODO: switch to official releases?)

ggml-org/whisper.cpp

• Includes an internal copy of ggml that may deviate from the official released version

• There are no nightly builds - only official releases

• Official release builds use semantic versioning: vX.Y.Z

• The release tags are created on the condition that the internal ggml copy is identical to the released ggml copy. This guarantees that a downstream building from the release tags and linking to a system-wide ggml will be compatible.

  Downstreams

  • Homebrew: whisper-cpp package, formula
    • TODO: rename to whisper.cpp?
    • TODO: how does the package manager know which version of ggml is required by whisper.cpp?
  • Debian: whisper.cpp package, packaging repository

ggml-org/llama.cpp/gguf-py

• Release builds use semantic versioning: vX.Y.Z

  • Git tags: gguf-vX.Y.Z

  Downstreams

  • pip: gguf package, workflow

ggml-org/llama.vim

• TODO: not sure about vim packaging/distribution practices, help appreciated in order to spec this section

[ggerganov]

FYI @ggml-org/ci @ckastner @p-linnane

[CISC]

May be worth mentioning gguf-py here as well?

[ckastner]

Also llama.vim?

[ggerganov]

Added, though I'm not sure how to spec it at the moment.

[ckastner]

This is very useful, thanks!

Super-happy to see semantic versioning for llama.cpp coming. That will also enable maintaining a SOVER. And that will allow us in Debian to ship it as a library in public search paths, because we can now guarantee backwards-compatibility (breaking can be communicated and maintained via SOVER bumps).

For llama.cpp and whisper.cpp release builds, it would be great if they could somehow specify the minimum ggml version they need, something like find_package(ggml 0.18 REQUIRED), so that packaging tools can make use of this.

Debian links:

• ggml package, packaging repository
• ggml-cuda package (separate build because of non-free CUDA dependencies)
• llama.cpp package, packaging repository
• whisper.cpp package, packaging repository
• llama.vim package, packaging repository

We also have HIP and CUDA builds using AMD's/NVIDIA's repositories, for bleeding edge hardware and toolkit support. I'd add that in an increment, though.

Below just FYI:

In Debian, we currently tend to update llama.cpp in tandem with new ggml release, since that was the point where both were expected to be ~"in sync":

• If ggml had a minor x.y bump, something key was added, and we increase llama.cpp's dependency on ggml version accordingly
• If ggml had a patch x.y.z bump, we assume a bugfix, and increase llama.cpp's dependency on ggml version only if a build shows it's needed
• We always verify this in practice, with various rebuilds and test runs

With this new release strategy, things are much more deterministic and plannable.

[ggerganov]

For llama.cpp and whisper.cpp release builds, it would be great if they could somehow specify the minimum ggml version they need, something like find_package(ggml 0.18 REQUIRED), so that packaging tools can make use of this.

Right now this can be deduced from the included internal copy of ggml. For example, for llama.cpp: https://github.com/ggml-org/llama.cpp/blob/master/ggml/CMakeLists.txt#L1-L10

But maybe this is too inconvenient for the package managers? We'll look into the find_package suggestion.

[ckastner]

find_package is just a bit more obvious as the build should already fail at the configuration stage, with a useful error message. And this works even outside of managed packages.

[ngxson]

I think the semantic version for llama.cpp makes perfect sense. Two things I want to bring to the discussion:

First: it will probably be better to have a list of requirements to bump major/minor/patch, to avoid forever sticking with something like 0.1.1234

I'm suggesting a initial version of the list for further discussions:

• Patch number bump when:
  • small bug fixes
  • vendor updates
• Minor bump when:
  • new model support
  • perf improvements
• Major bump when:
  • major changes in pubic API (i.e. new array of API is added, deprecated old one)
  • features added to server (which also changes the public REST API)

Second: we can setup a specific pipeline that tests the release against common models before releasing it

• To decide: what is the list of models? Maybe last N well known / most downloaded on HF, plus a fixed list of hand-picked models
• We may not have enough hw capacity to test on all backend, probably starts with CUDA & Metal for now?

[ngxson]

Interesting. At first glance, the 1.0.0 and 2.0.0 just look like additions to the REST API, that doesn't really breaking anything though, right? The 4.0.0 bump was because the /metrics endpoint got new fields, not sure about that.

Right, in general we rarely have breaking changes for server API, only addition (with backward compat), so it's true that the past 100 commit doesn't bump major number at all.

However, that would still mean at least ~50 minor bumps for a time span of 5 days. And if (for example) we choose to release once per week, that would mean (almost) always one minor bump per week, almost no patch bump (unless in the rare case of force triggering the release workflow).

So assuming 1 release per week, the releases will look almost like this:

0.1.0 --> first week
0.2.0 --> second week
0.3.0 --> third week
0.3.1 --> "force" release because 0.3.0 is broken
... (rarely a patch bump)
0.123.0 --> 123-th week after semver was adopted

That got me thinking about home assistant's calver scheme:

2026.1.0
2026.2.0
2026.2.1  --> "force" release for bug fixes
...
2026.8.0  --> latest

[ngxson]

If we choose to release every N commits instead of every week, the homebrew releases is a nice place to measure the speed: https://github.com/Homebrew/homebrew-core/pulls?q=sort:updated-desc+is:pr+llama.cpp+in:title

Things that I observed about this approach:

• We are merging about 10-20 commits per day, so if N less than 20 then it will be more frequent than nightly (which we don't want)
• Minor version bump almost always happen for a span of 10-20 commits, we rarely have 10-20 patches (bug fixes) in a row

[ckastner]

And if (for example) we choose to release once per week, that would mean (almost) always one minor bump per week, almost no patch bump (unless in the rare case of force triggering the release workflow).

One minor bump per week wouldn't really shock me that much, TBH. I wouldn't do more frequent than that, because at some point one just lands back at the nightlies. Perhaps "weekly" is even too strict, something like ~weekly, but shorter if some release needs it, or longer if stuff isn't ready yet.

The calver scheme you linked also looks nice. One can't really tell breaking changes from it, but for the library there is the SOVER to signal that independently, and for the tools -- well, maybe they don't need that signal?

[ngxson]

Agree, I think weekly release the as good as we can guaranteed for now. So these point are pretty much aligned for now:

• Most releases will be minor bump
• The release frequency is not strictly fixed, we can force-release. For example, some cases that I can think of:
  • If a model support is added, it's likely mean a new release --> likely be a minor bump
  • If the release is broken somehow --> likely be a patch bump
• Occasionally major release on these changes:
  • Breaking changes in ABI. Note that we usually try to have function calls backward-compat, meaning new libllama calls don't mean breaking change; however adding a new enum value will still be a breaking change
  • Do we cover llama-ext.h header? or only the llama.h?
  • For server API: only breaking change to inference endpoints (completions, embeddings) are considered major bump. Breaking changes to other things: router API, tools, CORS proxy are patches
  • Changes to components like llama-cli, llama-ui, llama-quantize are always (but not strictly) minor bump

I'll try to run another simulation tomorrow

[ggerganov]

Do we cover llama-ext.h header? or only the llama.h?

The purpose of the llama-ext.h is to be able to experiment with some API changes without affecting the official API of the library and therefore without affecting the minor and major versions. The llama-ext.h is not public so we can change it without announcing this with a version bumps.

[mbaudier]

Will this versioning policy also cover mtmd?

More generally, is it planned that libmtmd would be maintained as a library, like libllama? (vs. in tools/ as is currently the case)
Or will it be merged into libllama at some point?

[ggerganov]

I wish one day we will merge it in libllama, but it might take a while. For now it seems much simpler to remain separate. Though we probably should treat its API the same way as libllama's API and reflect changes in the semantic version accordingly. @ngxson WDYT?

[ngxson]

Good point. I don't have a good idea for now, but I'm considering about this:

Since our goal is to merge libmtmd and libllama into one package, it might be important to always keep the libmtmd version less than libllama until we merge them together.

One idea I'm thinking about: libmtmd will have versioning with major = 0 for now, and I will only bump the minor+patch. Then when we merge it to libllama, the major version will be increase to match libllama (in other words: if we bump mtmd major number right now, when we merge it, we will end up with mtmd's major > llama's major, which will be a messy situation)

[ggerganov]

I meant that we can have a single semantic version that tracks both libllama and libmtmd. So basically as if the llama.h and mtmd.h were part of the same API. Changes in either API results in version update.

[danbev]

I've opened ggml-org/llama.cpp#26839 and libmtmd in which both use the same semantic version.

[mbaudier]

As a consumer (vs. as a packager) of the libraries only, that is, developing directly on top of libllama and libmtmd, this is indeed one same API. The separation currently makes sense because libmtmd is, as stated in the doc, slightly less stable than libllama, and that makes it easier to track the dependencies (simply check the headers). +1 for using the same semantic version, as it will be easier for everyone to manage.

[ggerganov]

Next week will try to implement some automation with @danbev according to the spec above to start semver tagging of llama.cpp and utilizing find_package for discovering the correct ggml version. Until then we can iterate on the spec and discuss.

[danbev]

I've made a start on find_package which can be found here #1582

[danbev]

@ggerganov Regarding the release branches mentioned above. I was thinking of using the current build flow but with the addition of verifying that the ggml version in llama.cpp matches the released version for llama.cpp release builds, and do this in a github action. Basically not having a separate release branch is what I'm suggesting/asking, but perhaps I'm overlooking some additional benefit of using a separate release branch?

[ggerganov]

Yes, I also realized that we don't need the release branches. I will update the spec accordingly to simplify.

[p-linnane]

For Homebrew we perform regression tests of dependents when updating formulae. If a new version of ggml was released that llama.cpp didn't work with, CI would fail. Likewise, as seen in our latest bump, if a version of llama.cpp was released that didn't work with the current ggml, CI would fail.