Backend: dm_rerank

The dm_rerank is an ensemble style backend that uses a decision model service such as Jev to re-score/re-rank the candidate subjects proposed by the source backends. It supports any decision model with a /v1/systemone API endpoint implemented in recent versions of llama.cpp

For each document and each candidate subject, the backend asks the service a binary ("true or false?") predicate question about the subject and the document, using the document text as the shared input. The default proposition is a centrality ("sharp") question, "Is '{label}' a central subject of this document — one a librarian would assign as a primary heading — not merely a passing or incidental mention?", which can be replaced with any custom template via the instruction parameter.

Each candidate is then re-scored as

raw   = source * ((p + eps) / (1 - p + eps))^model-strength
    * sigmoid((p - gate-threshold) / 0.1)
score = raw / (1 + raw)

where source is the (unmodified) source score, p is the decision model score returned by the service (the probability that the predicate is true), and eps is a small damping constant (0.01).

Setting up the decision model service

See the blog post for details. Good models to try include lev 4B, Kev-4B, Clef-Flash 9B and OpenJev 27B. Here is an example llama.cpp command line that worked well for a V100 32GB GPU:

llama serve -m Kev-4B-Q8_0.gguf -lv 4 --parallel 50 -ub 2048

For good performance, you should use a high --parallel value, ideally matching the max-candidates setting (or half of it), but this will limit context size for individual requests. You can also set --fit-target 128 to leave less VRAM unused (default is 1024 MB).

Example configuration

A minimal configuration that relies on default values:

[yso-dm-fi]
name = "YSO DM Rerank"
language = "fi"
backend = "dm_rerank"
sources = "yso-mllm-fi"
endpoint = "http://localhost:9931/v1/systemone"
model-strength = 0.3
input-rules = true
limit = 100
vocab = "yso"

Backend-specific parameters

Parameter	Description
endpoint	URL of the reranking service endpoint. Defaults to http://localhost:9931/v1/systemone.
model	Model name of the reranking service, if it supports several models. When empty the model parameter is omitted from the request payload. Defaults to "".
model-strength	Exponent of the odds-ratio term — how strongly the decision model's yes-probability influences the final score. 0.0 ignores the model entirely (pure source order); values in (0, 1) apply a damped blend, and values > 1 let the model boost confident "yes" candidates above their source score. Defaults to 0.3.
gate-threshold	Center of a soft sigmoid gate (fixed width 0.1) applied to the raw decision model score; this gives the decision model veto powers over the suggested subjects. A sufficiently negative value (<= -1.0) makes the gate exactly 1 for all scores, which turns it fully off — combined with model-strength=0.0 the blend then reproduces the pure source scores exactly. Defaults to -1.
input-rules	Prepend the built-in indexing-policy rules (the format/genre rule) to the document input for every question. Defaults to false.
input-prefix	Custom text prepended to the document input for every question; overrides input-rules when both are set. Defaults to "".
input-transform	A transform applied to the document text only before it is sent to the decision model as the shared input, on top of the project transform (e.g. input-transform = "limit(10000)"). The source projects still run on the project-transformed text, so this keeps the input within the model's context length without weakening the source candidates; important when the source benefits from more text than the model can accept (e.g. long documents with a small-context decision model). Defaults to pass.
retries	Number of retries with exponential backoff (1s, 2s, …) on transient request failures (5xx, 408, 429, timeouts). HTTP 400 responses are not retried with backoff; instead the input is automatically reduced (75%, then 50% of the original) and the request is repeated. Defaults to 2.
timeout	Request timeout in seconds. Defaults to 60.
max-candidates	Maximum number of top candidates sent to the decision model. 0 scores all merged candidates from the sources; a positive value scores only the top N by combined source score and the remaining candidates are excluded from the output. Defaults to 0.
instruction	Question template. Must contain {label} (the subject's preferred label in the project language). Defaults to the sharp centrality predicate (see above).

Hyperparameter optimization

The two blend parameters can be tuned per corpus and decision model with the standard annif hyperopt command (e.g. annif hyperopt yso-dm-fi --trials 100 validation.tsv): the decision model is queried once per document in the corpus, and the model-strength and gate-threshold parameters are then searched over the cached scores with TPE (Optuna), so only the source suggestions and one model pass cost a significant amount of computing resources. The search space is model-strength 0–2 and gate-threshold −1.0…0.5. The pure-source configuration (0, −1.0), which reproduces the source order exactly, is always evaluated as an enqueued baseline trial (counting towards --trials), so the recommendation can fall back to the source ranking if the decision model scores carry no signal (e.g. by breaking ties in the source scores, every sampled configuration can be worse than the source ranking even though the neutral configuration is better).

Usage

Load a vocabulary:

annif load-vocab yso /path/to/Annif-corpora/vocab/yso-skos.ttl

Training is not necessary or possible. Test the model with a single document:

cat document.txt | annif suggest yso-dm_rerank-en

Evaluate a directory full of files in fulltext document corpus format:

annif eval yso-dm_rerank-en /path/to/documents/

---

← nn_ensemble | HTTP →