Skip to content

embeddinggemma.c v0.3.1

Latest
Latest

Choose a tag to compare

View all tags
@ehartford ehartford released this 23 Jul 08:15
v0.3.1
19a5da8

What's new

  • Adds an OpenAI-compatible POST /v1/embeddings endpoint with exact token usage, OpenAI error envelopes, Matryoshka dimensions, float/base64 encodings, and cache isolation from /api/embed.
  • Adds a persistent exact-embedding cache via --persistent-cache-path for repeat, re-index, and evaluation workloads.
  • Enables fused Metal up/gate/GELU by default.
  • Adds opt-in high-token CUDA W4A8 projection and int8 attention paths (EI_CUDA_W4A8_GEMM, EI_CUDA_INT8_ATTN).
  • Adds opt-in XPU attention-output half fusion (EI_XPU_FUSE_ATTN_OUTPUT_HALF).
  • Corrects llama.cpp benchmark ordering/thermal bias with both-order measurement support and documents the measured optimization ceiling.

Assets

This release contains raw executables for Darwin ARM64 CPU and Metal, plus Linux x86_64 CPU, CUDA, ROCm, and XPU. SHA256SUMS covers exactly those six binaries. No model weights are included.

Validation

Built from commit 19a5da8e8499 and tested on Apple M5 Max, NVIDIA RTX 3090 (GPU 1), AMD MI300X, and Intel Arc Pro B60. All backend golden, route-parity, packed-batch, and native/OpenAI HTTP tests passed. CUDA includes native cubins for SM 5.0 through SM 12.1 plus SM 12.1 PTX; ROCm includes gfx908, gfx90a, gfx942, and gfx950; XPU includes PVC and BMG device images.

Assets 9
Loading