Repository navigation
Filter models
Call filters steps 3–5 need a model. RavensPort never downloads one itself: you download it from Hugging Face, point RavensPort at the folder, and it runs inside RavensPort on the CPU or a GPU. This page covers:
- GLiNER PII: finds names, addresses, phone numbers and the like (step 3, Entities)
- Embedding models: catch calls that mean something you deny, taught by example (step 4, Meaning)
- Laya: answers your yes/no statements (step 5, Decision)
- A zero-shot model: the simpler alternative for step 5
An .onnx file holds only the network. The tokenizer and config files sit beside it on Hugging
Face, and RavensPort needs those too. The commands below download exactly the files needed.
Install the Hugging Face command-line tool once (Python 3.9 or newer):
pip install -U "huggingface_hub[cli]"It gives you hf download. Without Python, download the same files from the model's Files and
versions tab in a browser, keeping the folder layout shown here.
The examples use C:\Models. Any folder works, and on Linux the same commands work with Linux paths.
onnx-community/gliner_multi_pii-v1
is a GLiNER model tuned for personal data. It reads German and English, and finds whatever labels you
write in plain words: person, address, phone number, medical condition…
1. Download (about 1.1 GB):
hf download onnx-community/gliner_multi_pii-v1 onnx/model.onnx spm.model gliner_config.json added_tokens.json --local-dir C:\Models\gliner_multi_pii-v1You get:
C:\Models\gliner_multi_pii-v1\
spm.model
gliner_config.json
added_tokens.json
onnx\model.onnx
For half the size, download onnx/model_fp16.onnx (550 MB) instead. Don't use the int8, uint8, q4
or other quantized files: they find almost nothing.
2. Add it. Filters tab → Models → Entity models (step 3):
| Field | Value |
|---|---|
| Name | GLiNER PII |
| Model file or folder |
C:\Models\gliner_multi_pii-v1 (or the .onnx file itself, e.g. …\onnx\model_fp16.onnx) |
| Runs on | CPU (see CPU or GPU) |
Press Add entity model, then Test on its row.
3. Use it. In a profile, set 3 · Entities to a level, pick the model, and fill in Look for. Use suggested labels fills in a sensible set. Every label is a chance for a false alarm, so keep to what you want stopped.
It takes about 1.2 GB of RAM while RavensPort runs.
An embedding model turns text into a vector, so texts that mean the same land close together. Step 4 compares each field of a call with what you deny, and blocks or logs the call when it comes close enough.
-
You teach it by example. Under Deny calls that…, Examples to deny and Look-alikes
to allow, you list what to stop (
rm -rf /,DROP TABLE users) and what is fine although it looks similar (rm -rf ./build,SELECT * FROM users). A text closer to an allowed look-alike than to anything denied always goes through. That is the most direct way to fix a false alarm, and step 5 has nothing like it. -
It doesn't depend on exact wording. "wipe the disk", "erase everything on C:" and
Remove-Item -Recurse C:\all land near "delete files or whole disks". Step 5 needs carefully worded statements; step 4 needs good examples. - It is cheap. Only the call's own text is embedded on each call; your deny and allow lists are embedded once and kept. all-MiniLM-L6-v2 is 86 MB, and stella-en-400M-v5 measured about 70 ms per call on a CPU.
- Scores you can read. Closeness is measured against everyday text (file names, weather questions, issue lists): 0 means "like an ordinary call", 1 means "identical to a deny entry". Try it shows the score and which field and entry it came from, so the threshold is easy to set.
- It runs wherever you already have one. An ONNX model inside RavensPort, or Ollama, llama.cpp, LM Studio, OpenAI or Voyage AI, with no export step.
What it can't do: it measures similarity, not intent. "Don't delete anything" sits close to "delete files", and it can't weigh "deletes a temp file" against "deletes customer data". Step 5 reasons about the call and handles those. Use step 4 to catch close copies of known-bad calls cheaply, and step 5 for judgement calls. Both can run in the same profile.
Only models with a BERT-style tokenizer (a vocab.txt) are supported: MiniLM, bge, e5, gte,
stella-en-400M. These come with ONNX files ready to use:
| Model | Size | Notes |
|---|---|---|
sentence-transformers/all-MiniLM-L6-v2 |
86 MB | Small and fast; a good first choice |
BAAI/bge-small-en-v1.5 |
126 MB | Stronger than MiniLM at a similar size |
BAAI/bge-base-en-v1.5 |
415 MB | Stronger again, slower |
All three are English. For other languages, run bge-m3 on a local server: multilingual
models use a SentencePiece tokenizer, which RavensPort's ONNX runner doesn't read.
NovaSearch/stella_en_400M_v5 works but has no ONNX files, so it has to be exported first.
1. Download, e.g. MiniLM:
hf download sentence-transformers/all-MiniLM-L6-v2 onnx/model.onnx vocab.txt tokenizer_config.json 1_Pooling/config.json --local-dir C:\Models\all-MiniLM-L6-v21_Pooling/config.json tells RavensPort how to pool the model's output, and
tokenizer_config.json whether it lowercases text. For bge, swap in its repo name; the files are
the same.
2. Add it. Filters tab → Models → Embedding models (step 4):
| Field | Value |
|---|---|
| Where it runs | ONNX model in RavensPort (local) |
| Name | MiniLM |
| Model file or folder |
C:\Models\all-MiniLM-L6-v2. Use the folder: vocab.txt is looked for next to what you point at, so pointing at onnx\model.onnx won't find it |
| Runs on | CPU |
Press Add embedding model, then Test.
A server on this PC runs models that RavensPort's ONNX runner can't, such as the multilingual bge-m3, without anything leaving the PC. Pick whichever you have; the first needs nothing else installed.
llama.cpp (no Ollama):
winget install ggml.llamacpp
llama-server -hf ggml-org/bge-m3-Q8_0-GGUF --embeddings --port 8080-hf downloads the model from Hugging Face on first start (about 600 MB) and keeps it. Other
choices: nomic-ai/nomic-embed-text-v1.5-GGUF (English, smaller) or
ggml-org/embeddinggemma-300M-GGUF. On Linux, use a
llama.cpp release or build it yourself.
llama-server has to keep running while RavensPort checks calls, for example from a scheduled task
at logon.
Add an embedding model with:
| Field | Value |
|---|---|
| Where it runs | llama.cpp / OpenAI-compatible server |
| API base URL |
http://127.0.0.1:8080/v1 (filled in) |
| Model |
bge-m3 (llama-server serves the one model it loaded, whatever this says) |
| API key | (none) |
LM Studio (no Ollama): download an embedding model such as bge-m3 or nomic-embed-text in
LM Studio, start its local server (Developer → Start Server), and add it the same way with
base URL http://127.0.0.1:1234/v1 and the model name LM Studio shows.
Ollama:
ollama pull nomic-embed-text # or bge-m3 for many languages, mxbai-embed-largeAdd it with Where it runs Ollama (local). The base URL http://127.0.0.1:11434/v1 is filled
in; set Model to nomic-embed-text and leave the API key empty.
OpenAI and Voyage AI work too, with their API key added on the Credentials tab. RavensPort warns on these: every call step 4 checks is sent to that API (minus what steps 1–3 replaced), so use one only if you trust it with your agents' calls.
In a profile, set 4 · Meaning, pick the model, and fill in the three lists. Write examples as plain text, not as tool-call JSON; examples count for more than descriptions. Start on Watch, check your own calls with Try it, and adjust the threshold (empty follows the level: Relaxed 0.55, Balanced 0.40, Strict 0.25) before switching to Block.
Laya (Apache-2.0) is a decision model: it answers yes/no questions about a text with a calibrated probability. RavensPort asks it your step 5 statements.
Laya is published only as PyTorch weights, so it has to be exported to ONNX once with Laya's own exporter. If you'd rather not, run Laya as a server instead: see Laya without exporting.
| Checkpoint | Reads | ONNX size |
|---|---|---|
english |
English, up to 512 tokens | ~1.6 GB |
multilingual |
51 languages, up to 1,024 tokens | ~1.2 GB |
1. Export (about 10 minutes on a CPU; needs about 2.5 GB free per checkpoint):
git clone https://github.com/NandhaKishorM/laya
cd laya
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install "torch>=2.9" onnx onnxscript onnxruntime transformers safetensors huggingface_hub numpy
pip install -e .
python laya-dotnet/tools/export_onnx.py --model multilingual --out C:\Models\layaUse --model english for the English checkpoint, and add --verify to check the export against
Laya's reference outputs. The script downloads the weights from Hugging Face by itself. Full
details are in Laya's
MODELS.md.
You get:
C:\Models\laya\multilingual\
model.onnx
model.onnx.data keep it beside model.onnx
rl_agent_config.json
tokenizer\tokenizer.json
tokenizer\tokenizer_config.json
2. Add it. Filters tab → Models → Decision models (step 5):
| Field | Value |
|---|---|
| Kind | Laya, ONNX in RavensPort (local) |
| Name | Laya multilingual |
| Model folder (or .onnx file) | C:\Models\laya\multilingual |
| Runs on | CPU |
Press Add decision model, then Test.
3. Use it. In a profile, set 5 · Decision, pick the model, and write statements about what to stop: "This text is about deleting files or data.". Laya tends to be overconfident, so start between Strict and Balanced and check your own calls with Try it before switching to Block.
RavensPort runs Laya with its own code rather than Laya's .NET package. It has not yet been checked against Laya's reference outputs.
Laya's own server speaks the Jev protocol, which RavensPort also supports:
pip install "laya[serve]"
laya-serveThen add a decision model with Kind Jev API (TypeSafe Jev, or laya-serve), base URL
http://127.0.0.1:8000/v1, and no API key. Laya runs in Python on this PC; nothing leaves it.
For step 5 you don't need Laya: any zero-shot (NLI) or cross-encoder model in ONNX works, and some
come ready-made.
MoritzLaurer/deberta-v3-base-zeroshot-v2.0
is the one RavensPort was measured with: about 50 ms per call for three statements on a CPU.
hf download MoritzLaurer/deberta-v3-base-zeroshot-v2.0 --include "onnx/*" --local-dir C:\Models\deberta-v3-base-zeroshot-v2.0Add it with Kind Zero-shot / cross-encoder, ONNX in RavensPort (local) and the folder
C:\Models\deberta-v3-base-zeroshot-v2.0. Start statements with "This text is about …": that
wording scored a money transfer 1.00, while "The call sends money." scored it 0.01.
For other languages, MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 works too. It has
no ONNX files, so export it once:
pip install "optimum[onnxruntime]"
optimum-cli export onnx --model MoritzLaurer/mDeBERTa-v3-base-xnli-multilingual-nli-2mil7 --task zero-shot-classification C:\Models\mdeberta-xnliEvery model has a Runs on setting.
- CPU is the default, and for calls of a few lines it is usually the faster choice. In RavensPort's measurements it beat the GPU on both an RTX 4060 and an integrated Radeon.
- GPU runs through DirectML on Windows, on any DirectX 12 GPU, listed as Windows sees it. It is worth trying for long calls or large models.
- On Linux, models run on the CPU: the Linux build has no GPU support for them.
If the GPU cannot start a model, RavensPort runs it on the CPU and the activity log says so.
| What you see | What to do |
|---|---|
| No spm.model or gliner_config.json in … | Download them into the folder above onnx\, beside the model |
| No vocab.txt next to the model in … | Point the embedding model at its folder, not at onnx\model.onnx; or the model has no BERT-style tokenizer, so run it on a local server (llama.cpp, LM Studio or Ollama) |
| No model.onnx in that folder or its onnx/ folder | Point at the .onnx file itself, e.g. …\onnx\model_fp16.onnx
|
| no rl_agent_config.json beside the model — is this a Laya export? | Point at the export's checkpoint folder (…\laya\multilingual), not its parent |
| no tokenizer.json beside the model | Download tokenizer.json; for Laya it may also sit in tokenizer\
|
| GLiNER finds (almost) nothing | You are using a quantized file; use model.onnx or model_fp16.onnx
|
| Every call blocked at step 3 or 5 | The model failed to load. Steps set to Block fail closed. Test on the model's row shows why |