-
Notifications
You must be signed in to change notification settings - Fork 0
AI and Model Validation
AI is entirely optional. Inspectarr scores indexers with a deterministic weighted formula on its own, and detection and remediation never consult a model at all. Ollama adds an advisory overlay on top of indexer scoring, plus natural-language notification digests.
Leave the Ollama URL blank and everything else works unchanged.
Everything lives under Settings → AI.
| Setting | Purpose |
|---|---|
| Ollama URL | Base URL of your Ollama server, e.g. http://192.168.1.10:11434. Blank disables all AI features. |
| Timeout | Hard cutoff per scoring run, in seconds. Allow extra on the first run after a model swap — the model has to load. |
| Model | Which model performs scoring. Populated live from Ollama. |
| Cache TTL | How long an AI result is reused for identical input. |
| Model update check | How often to check whether your model has been replaced. 0 disables. |
| System prompt | The scoring prompt. Loads with the built-in default; edit to customise. |
A Test Connection button confirms the URL before you rely on it.
prowlarr:
ollama:
url: "http://192.168.1.10:11434"
model: "qwen2.5:7b-instruct"
timeout: 300
cache_ttl_hours: 24
update_check_hours: 24
system_prompt: "" # empty = built-in defaultNot every model can do this job, and the ways they fail are quiet rather than loud. A model with too small a context window does not error — it loses the scoring instructions and returns fabricated data that looks like a valid score. Others return the right shape with the wrong key names, or echo the input back as "reasoning".
Validation catches all of that before a model can be selected.
1. Discrimination. One unambiguously good indexer and one unambiguously bad one, scored in a single call. The model must rate the good one ≥ 70, the bad one ≤ 40, and separate them by at least 25 points.
The margin matters: a model scoring them 55 and 50 orders them correctly while being useless. This also catches models that return a constant, and models with an inverted scale.
2. Schema compliance. Every returned entry must carry a numeric health_score in range and a non-empty reasoning string that is not simply the input echoed back.
3. Context capacity. Synthetic indexers matching your actual indexer count are sent in one prompt, and every one must come back with no invented IDs.
This is the test that matters most. Because it is sized to your deployment, "passed" means passed here — not in the abstract.
Settings → AI → Validate. Selecting a model in the dropdown chooses a candidate; it is not applied until you click Apply.
Validation takes one to a few minutes: a cold model load plus two scoring calls, one at full indexer count. It runs in the background — you can leave the page. Only one validation runs at a time.
A model that has not passed cannot be applied by accident. Apply on an unvalidated model does not start a validation run; it explains the situation and offers Cancel or Apply anyway.
Applying anyway records the model as forced — distinct from failed, because you chose it knowingly. A later successful validation supersedes that.
Each model shows as ✓ validated, ✗ failed, ⚠ forced, or — untested in the dropdown, with per-test detail below it and a comparison table of every model you have tested — model, per-test results, average response time, and date.
That table is the practical way to answer "which of these models actually works here", especially when hunting for one model that serves several applications.
If the model behind a name is replaced (a re-pull of the same tag), the stored digest no longer matches and Inspectarr flags it so you can re-validate. It checks the local digest only and never pulls — the check is free and changes nothing on your Ollama host.
When a URL and model are configured, Inspectarr sends all indexer data in one batch and the returned scores replace the deterministic ones. See Prowlarr Indexer Scoring.
Results are cached by a hash of the payload plus the model and system prompt, so changing either always rescores rather than silently reusing the previous model's answers.
Measured against 39 indexers, the scoring prompt is roughly 5,700 tokens. A 4k-context model cannot hold it and will fail validation's context test. 8k is the practical minimum; 16k is comfortable.
Independently of scoring, Ollama can narrate notification digests and periodic summaries — notifications.digest.use_ollama and notifications.summary.use_ollama. Both controls are disabled on the Notifications page until an Ollama URL is configured. See Notifications.
- Prowlarr Indexer Scoring — what the AI score overlays
-
Configuration — every
prowlarr.ollamakey