Skip to content

Filter models

Abishek Narasimhan edited this page Nov 15, 2026 · 3 revisions

Setting up filter models

Call filters steps 3–5 need a model. RavensPort never downloads one itself: you download it from Hugging Face, point RavensPort at the folder, and it runs inside RavensPort on the CPU or a GPU. This page covers:

  • GLiNER PII: finds names, addresses, phone numbers and the like (step 3, Entities)
  • Embedding models: catch calls that mean something you deny, taught by example (step 4, Meaning)
  • Laya: answers your yes/no statements (step 5, Decision)
  • A zero-shot model: the simpler alternative for step 5

An .onnx file holds only the network. The tokenizer and config files sit beside it on Hugging Face, and RavensPort needs those too. The commands below download exactly the files needed.


Before you start

Install the Hugging Face command-line tool once (Python 3.9 or newer):

pip install -U "huggingface_hub[cli]"

It gives you hf download. Without Python, download the same files from the model's Files and versions tab in a browser, keeping the folder layout shown here.

The examples use C:\Models. Any folder works, and on Linux the same commands work with Linux paths.


GLiNER PII — step 3, Entities

onnx-community/gliner_multi_pii-v1 is a GLiNER model tuned for personal data. It reads German and English, and finds whatever labels you write in plain words: person, address, phone number, medical condition…

1. Download (about 1.1 GB):

hf download onnx-community/gliner_multi_pii-v1 onnx/model.onnx spm.model gliner_config.json added_tokens.json --local-dir C:\Models\gliner_multi_pii-v1

You get:

C:\Models\gliner_multi_pii-v1\
    spm.model
    gliner_config.json
    added_tokens.json
    onnx\model.onnx

For half the size, download onnx/model_fp16.onnx (550 MB) instead. Don't use the int8, uint8, q4 or other quantized files: they find almost nothing.

2. Add it. Filters tab → Models → Entity models (step 3):

Field Value
Name GLiNER PII
Model file or folder C:\Models\gliner_multi_pii-v1 (or the .onnx file itself, e.g. …\onnx\model_fp16.onnx)
Runs on CPU (see CPU or GPU)

Press Add entity model, then Test on its row.

3. Use it. In a profile, set 3 · Entities to a level, pick the model, and fill in Look for. Use suggested labels fills in a sensible set. Every label is a chance for a false alarm, so keep to what you want stopped.

It takes about 1.2 GB of RAM while RavensPort runs.


Embedding models — step 4, Meaning

An embedding model turns text into a vector, so texts that mean the same land close together. Step 4 compares each field of a call with what you deny, and blocks or logs the call when it comes close enough.

Why use one

  • You teach it by example. Under Deny calls that…, Examples to deny and Look-alikes to allow, you list what to stop (rm -rf /, DROP TABLE users) and what is fine although it looks similar (rm -rf ./build, SELECT * FROM users). A text closer to an allowed look-alike than to anything denied always goes through. That is the most direct way to fix a false alarm, and step 5 has nothing like it.
  • It doesn't depend on exact wording. "wipe the disk", "erase everything on C:" and Remove-Item -Recurse C:\ all land near "delete files or whole disks". Step 5 needs carefully worded statements; step 4 needs good examples.
  • It is cheap. Only the call's own text is embedded on each call; your deny and allow lists are embedded once and kept. all-MiniLM-L6-v2 is 86 MB, and stella-en-400M-v5 measured about 70 ms per call on a CPU.
  • Scores you can read. Closeness is measured against everyday text (file names, weather questions, issue lists): 0 means "like an ordinary call", 1 means "identical to a deny entry". Try it shows the score and which field and entry it came from, so the threshold is easy to set.
  • It runs wherever you already have one. An ONNX model inside RavensPort, or Ollama, llama.cpp, LM Studio, OpenAI or Voyage AI, with no export step.

What it can't do: it measures similarity, not intent. "Don't delete anything" sits close to "delete files", and it can't weigh "deletes a temp file" against "deletes customer data". Step 5 reasons about the call and handles those. Use step 4 to catch close copies of known-bad calls cheaply, and step 5 for judgement calls. Both can run in the same profile.

An ONNX model in RavensPort

Only models with a BERT-style tokenizer (a vocab.txt) are supported: MiniLM, bge, e5, gte, stella-en-400M. These come with ONNX files ready to use:

Model Size Notes
sentence-transformers/all-MiniLM-L6-v2 86 MB Small and fast; a good first choice
BAAI/bge-small-en-v1.5 126 MB Stronger than MiniLM at a similar size
BAAI/bge-base-en-v1.5 415 MB Stronger again, slower

All three are English. For other languages, run bge-m3 on a local server: multilingual models use a SentencePiece tokenizer, which RavensPort's ONNX runner doesn't read. NovaSearch/stella_en_400M_v5 works but has no ONNX files, so it has to be exported first.

1. Download, e.g. MiniLM:

hf download sentence-transformers/all-MiniLM-L6-v2 onnx/model.onnx vocab.txt tokenizer_config.json 1_Pooling/config.json --local-dir C:\Models\all-MiniLM-L6-v2

1_Pooling/config.json tells RavensPort how to pool the model's output, and tokenizer_config.json whether it lowercases text. For bge, swap in its repo name; the files are the same.

2. Add it. Filters tab → Models → Embedding models (step 4):

Field Value
Where it runs ONNX model in RavensPort (local)
Name MiniLM
Model file or folder C:\Models\all-MiniLM-L6-v2. Use the folder: vocab.txt is looked for next to what you point at, so pointing at onnx\model.onnx won't find it
Runs on CPU

Press Add embedding model, then Test.

Through a local server: llama.cpp, LM Studio or Ollama

A server on this PC runs models that RavensPort's ONNX runner can't, such as the multilingual bge-m3, without anything leaving the PC. Pick whichever you have; the first needs nothing else installed.

llama.cpp (no Ollama):

winget install ggml.llamacpp
llama-server -hf ggml-org/bge-m3-Q8_0-GGUF --embeddings --port 8080

-hf downloads the model from Hugging Face on first start (about 600 MB) and keeps it. Other choices: nomic-ai/nomic-embed-text-v1.5-GGUF (English, smaller) or ggml-org/embeddinggemma-300M-GGUF. On Linux, use a llama.cpp release or build it yourself. llama-server has to keep running while RavensPort checks calls, for example from a scheduled task at logon.

Add an embedding model with:

Field Value
Where it runs llama.cpp / OpenAI-compatible server
API base URL http://127.0.0.1:8080/v1 (filled in)
Model bge-m3 (llama-server serves the one model it loaded, whatever this says)
API key (none)

LM Studio (no Ollama): download an embedding model such as bge-m3 or nomic-embed-text in LM Studio, start its local server (Developer → Start Server), and add it the same way with base URL http://127.0.0.1:1234/v1 and the model name LM Studio shows.

Ollama:

ollama pull nomic-embed-text      # or bge-m3 for many languages, mxbai-embed-large

Add it with Where it runs Ollama (local). The base URL http://127.0.0.1:11434/v1 is filled in; set Model to nomic-embed-text and leave the API key empty.

OpenAI and Voyage AI work too, with their API key added on the Credentials tab. RavensPort warns on these: every call step 4 checks is sent to that API (minus what steps 1–3 replaced), so use one only if you trust it with your agents' calls.

Using it

In a profile, set 4 · Meaning, pick the model, and fill in the three lists. Write examples as plain text, not as tool-call JSON; examples count for more than descriptions. Start on Watch, check your own calls with Try it, and adjust the threshold (empty follows the level: Relaxed 0.55, Balanced 0.40, Strict 0.25) before switching to Block.


Laya — step 5, Decision

Laya (Apache-2.0) is a decision model: it answers yes/no questions about a text with a calibrated probability. RavensPort asks it your step 5 statements.

Laya is published only as PyTorch weights, so it has to be exported to ONNX once with Laya's own exporter. If you'd rather not, run Laya as a server instead: see Laya without exporting.

Checkpoint Reads ONNX size
english English, up to 512 tokens ~1.6 GB
multilingual 51 languages, up to 1,024 tokens ~1.2 GB

1. Export (about 10 minutes on a CPU; needs about 2.5 GB free per checkpoint):

git clone https://github.com/NandhaKishorM/laya
cd laya
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install "torch>=2.9" onnx onnxscript onnxruntime transformers safetensors huggingface_hub numpy
pip install -e .
python laya-dotnet/tools/export_onnx.py --model multilingual --out C:\Models\laya

Use --model english for the English checkpoint, and add --verify to check the export against Laya's reference outputs. The script downloads the weights from Hugging Face by itself. Full details are in Laya's MODELS.md.

You get:

C:\Models\laya\multilingual\
    model.onnx
    model.onnx.data          keep it beside model.onnx
    rl_agent_config.json
    tokenizer\tokenizer.json
    tokenizer\tokenizer_config.json

2. Add it. Filters tab → Models → Decision models (step 5):

Field Value
Kind Laya, ONNX in RavensPort (local)
Name Laya multilingual
Model folder (or .onnx file) C:\Models\laya\multilingual
Runs on CPU

Press Add decision model, then Test.

3. Use it. In a profile, set 5 · Decision, pick the model, and write statements about what to stop: "This text is about deleting files or data.". Laya tends to be overconfident, so start between Strict and Balanced and check your own calls with Try it before switching to Block.

RavensPort runs Laya with its own code rather than Laya's .NET package. It has not yet been checked against Laya's reference outputs.

Laya without exporting

Laya's own server speaks the Jev protocol, which RavensPort also supports:

pip install "laya[serve]"
laya-serve

Then add a decision model with Kind Jev API (TypeSafe Jev, or laya-serve), base URL http://127.0.0.1:8000/v1, and no API key. Laya runs in Python on this PC; nothing leaves it.


A zero-shot model — step 5, Decision

For step 5 you don't need Laya: any zero-shot (NLI) or cross-encoder model in ONNX works, and some come ready-made. MoritzLaurer/deberta-v3-base-zeroshot-v2.0 is the one RavensPort was measured with: about 50 ms per call for three statements on a CPU.

hf download MoritzLaurer/deberta-v3-base-zeroshot-v2.0 --include "onnx/*" --local-dir C:\Models\deberta-v3-base-zeroshot-v2.0

Add it with Kind Zero-shot / cross-encoder, ONNX in RavensPort (local) and the folder C:\Models\deberta-v3-base-zeroshot-v2.0. Start statements with "This text is about …": that wording scored a money transfer 1.00, while "The call sends money." scored it 0.01.

For other languages, MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 works too. It has no ONNX files, so export it once:

pip install "optimum[onnxruntime]"
optimum-cli export onnx --model MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 --task zero-shot-classification C:\Models\mdeberta-xnli

CPU or GPU

Every model has a Runs on setting.

  • CPU is the default, and for calls of a few lines it is usually the faster choice. In RavensPort's measurements it beat the GPU on both an RTX 4060 and an integrated Radeon.
  • GPU runs through DirectML on Windows, on any DirectX 12 GPU, listed as Windows sees it. It is worth trying for long calls or large models.
  • On Linux, models run on the CPU: the Linux build has no GPU support for them.

If the GPU cannot start a model, RavensPort runs it on the CPU and the activity log says so.


When it doesn't work

What you see What to do
No spm.model or gliner_config.json in … Download them into the folder above onnx\, beside the model
No vocab.txt next to the model in … Point the embedding model at its folder, not at onnx\model.onnx; or the model has no BERT-style tokenizer, so run it on a local server (llama.cpp, LM Studio or Ollama)
No model.onnx in that folder or its onnx/ folder Point at the .onnx file itself, e.g. …\onnx\model_fp16.onnx
no rl_agent_config.json beside the model — is this a Laya export? Point at the export's checkpoint folder (…\laya\multilingual), not its parent
no tokenizer.json beside the model Download tokenizer.json; for Laya it may also sit in tokenizer\
GLiNER finds (almost) nothing You are using a quantized file; use model.onnx or model_fp16.onnx
Every call blocked at step 3 or 5 The model failed to load. Steps set to Block fail closed. Test on the model's row shows why

Clone this wiki locally