eeANE 1.4.0

eeANE 1.4.0 closes two gaps in what eeane compile can faithfully reproduce — sentence-transformers Dense projection modules and plain-RoBERTa architectures — and one gap in how it judges its own work: the accuracy self-check no longer measures every model against fixtures in a single fixed language. It also adds the OpenAI-compatible dimensions parameter to /v1/embeddings, and grows the verified catalogue from 51 to 60 models.

The self-check now speaks three languages

The compile-time accuracy check compares the compiled fp16 artifact against an fp32 reference on a small fixed set of sentences. Until now that set came in one language per backend — English for BERT, Japanese for the others, an accident of which model family each backend was born for. For a model whose vocabulary barely covers that language, the fixtures encode to little more than unknown-token fragments, and the amplified fp16-vs-fp32 difference reflects the input rather than the model: an English-only ModernBERT model could fail the check without anything being wrong with it.

eeane compile now scores three fixed sets — English, Japanese and Chinese — and accepts a variant as soon as any one of them clears the unchanged threshold. The log names every set's result and the one the variant was accepted on, and the same numbers are recorded in the artifact metadata:

      s512: sanity : pass (best=en 0.99883; ja 0.98737, zh 0.99236)

The existing fixtures were inherited word for word, so accuracy numbers recorded for previously verified models remain directly comparable.

Dense projection modules are compiled in

Some sentence-transformers models apply a linear projection after pooling — declared as a Dense module in the model directory's modules.json — and their published embeddings only exist on the far side of it. Earlier releases did not read that file, so such a model compiled without its projection: the self-check still passed, because both sides of it made the same omission, while the served vectors disagreed with the published model in both meaning and width.

eeane compile now reads the declared module chain, supports Transformer -> Pooling -> Dense (any number) -> Normalize (optional, trailing), and bakes each Dense stage (Identity or Tanh activation) into both the compiled graph and the fp32 reference — the embedding width follows the projection, so mdbr-leaf-mt correctly serves 1024-dimensional vectors from a 384-wide backbone. A chain the conversion cannot reproduce is refused before any weight is read, instead of silently compiling something with a different meaning. Dense checkpoints go through the same safetensors-first policy as the main weights, the declaration is recorded in the artifact metadata, and a changed declaration triggers reconversion. The trailing Normalize module is not baked in: the server applies L2 normalization according to each model's normalize setting, as before.

RoBERTa architectures

Models whose config.json declares a RobertaModel architecture are now routed to the XLM-RoBERTa backend. transformers implements RoBERTa and XLM-RoBERTa as the same encoder, differing only in vocabulary, so the existing backend — position offsets included — applies unchanged.

Truncatable embeddings

/v1/embeddings accepts the OpenAI-compatible dimensions parameter: the embedding is truncated to its first dimensions components and then re-normalized (when normalization is enabled), returning a smaller vector without re-running inference. This is meaningful for models trained with a Matryoshka representation learning objective, whose embeddings stay useful at any prefix length. Requesting more dimensions than the model's embedding width returns a 400.

Nine newly verified models

Each compiles with a single eeane compile <HF ID>, passed its self-check with unchanged thresholds, and had its output compared against the reference sentence-transformers / CrossEncoder implementation on an M2 Mac (embedding per-text cosine 0.99994–0.99999, reranker sigmoid difference at most 0.0013, Neural Engine placement 90.7–98.1%):

• MongoDB/mdbr-leaf-mt — 23M, BERT with a Dense projection to 1024 dimensions
• MongoDB/mdbr-leaf-ir — 23M, Dense projection to 768 dimensions
• sentence-transformers/LaBSE — 471M, 109-language, CLS pooling with a Tanh Dense head
• ibm-granite/granite-embedding-30m-english — 30M, RobertaModel
• ibm-granite/granite-embedding-125m-english — 125M, RobertaModel
• nomic-ai/modernbert-embed-base — 149M; failed 1.3.0's Japanese-only self-check (that measurement reproduces bit-for-bit under the new per-set report) and passes on the English set, where this English model belongs
• hotchpotch/japanese-reranker-cross-encoder-xsmall-v1 — 107M Japanese cross-encoder reranker
• hotchpotch/japanese-reranker-cross-encoder-small-v1 — 118M Japanese cross-encoder reranker
• hotchpotch/japanese-bge-reranker-v2-m3-v1 — 568M Japanese reranker built on bge-reranker-v2-m3

The Verified models table in the README lists all 60.

Verification

• 1136 tests pass, up from 961. The new ones cover per-set sanity evaluation and every way it can go wrong, the Dense declaration reader and its refusal paths, a full conversion of a synthetic Dense model, RoBERTa dispatch, and the dimensions parameter end to end.
• No regression: a recompiled cl-nagoya/ruri-v3-310m matches its pre-change comparison against sentence-transformers to the exact same values, and the default configuration passes its end-to-end server verification with a rerank median of 5.602 s, in line with previous releases.
• Eleven models were served at once, with each one's HTTP responses matched against direct Core ML inference (embedding cosine 1.00000000, reranker score difference 0.0), and dimensions was exercised over HTTP, base64 encoding included.

Install

uv tool install eeane

Full changelog: v1.3.0...v1.4.0