Skip to content

embeddinggemma.c v0.2.2

Choose a tag to compare

View all tags
@ehartford ehartford released this 20 Jul 04:12
v0.2.2
0329be6

This patch release replaces the SM86-only CUDA artifact with a portable CUDA
fat binary. Release assets are raw executables, not tarballs. GitHub provides
the source .tar.gz and .zip archives separately.

Binaries

  • embeddinggemma-darwin-arm64-cpu: Apple Silicon CPU server.
  • embeddinggemma-darwin-arm64-metal: Apple Silicon Metal server with all
    Metal kernels embedded in the executable.
  • embeddinggemma-linux-x86_64-cpu: Linux x86-64 CPU server.
  • embeddinggemma-linux-x86_64-cuda: Linux x86-64 CUDA server containing
    native code for every architecture supported by CUDA 12.9, plus PTX for its
    newest target.
  • SHA256SUMS: SHA-256 checksums for the four executables.

Downloaded executables may need chmod +x <filename> before use. Model weights
are not included.

The Darwin binaries target macOS 14.0 or newer and are ad-hoc signed. The Linux
binaries were built on Ubuntu 24.04 and dynamically use system libcurl. The
CUDA binary additionally requires CUDA 12 libraries and an NVIDIA driver; it
was built with CUDA 12.9 and validated on an RTX 3090.

CUDA Portability

The CUDA executable contains native cubins for SM50, SM52, SM53, SM60, SM61,
SM62, SM70, SM72, SM75, SM80, SM86, SM87, SM89, SM90, SM100, SM101, SM103,
SM120, and SM121, plus compute_121 PTX for forward compatibility.

Packed Q4/Q8 projection uses dp4a on compute capability 6.1 and newer and a
scalar integer fallback on older targets. The optional native packed-Q4 MMA
path now reports a clear error below compute capability 8.0 instead of exposing
Ampere-only instructions to older GPUs. Architectures other than SM86 were
compile-validated but were not hardware-tested for this release.

Validation

  • CPU, Metal, and CUDA pass all 10 llama.cpp golden embeddings.
  • CPU, Metal, and CUDA pass packed-batch parity.
  • HTTP Matryoshka dimensions 768, 512, 256, and 128 pass using every stripped
    release executable.
  • Metal passes the full backend parity suite without a metallib sidecar.
  • CUDA attention, QKV epilogue, Q4 MMA, and Q8 latency route parity pass.
Assets 7
Loading