Models and Providers

Models & Providers

SenseTree is model-agnostic. Five AI tasks are configured independently, and each points wherever you want: the built-in local engine, a runner on this machine, a GPU box on your LAN, or a hosted API. Mix freely.

The five slots

Slot	Used for	Runs on	Default
Embedding	Vectorizing documents and queries — the whole search index	Local fastembed/ONNX or /v1/embeddings	Local multilingual-e5-small, 384-d
Reasoning	Chat agent, action plans, qualification, folder classification, context guessing	/v1/chat/completions	llama3.1:8b on Ollama, enabled
Vision	Image captions, scanned-PDF OCR	Multimodal /v1/chat/completions	moondream, disabled
Transcription	Speech in audio and video	/v1/audio/transcriptions (multipart)	faster-whisper-large-v3 on :8000, disabled
Video	Visual description of videos	/v1/chat/completions with video_url	Qwen2.5-VL-7B, disabled

Two more models run locally and unconfigurably: the cross-encoder reranker (model choice only) and CLIP for visual search. Both are fastembed/ONNX, downloaded on first use.

Only embedding is required for search. Reasoning unlocks the chat, plans and qualification. The rest are opt-in because each assumes a server you run.

Local engine vs HTTP server

Local engine (embedding only). fastembed runs an ONNX model in-process — no server, no network, no setup. ONNX Runtime is provisioned on first use; CUDA is used when use_gpu is on and an NVIDIA driver is present, with a graceful CPU fallback. This is why semantic search works the moment you install the app.

OpenAI-compatible server. Point a base_url at Ollama, LM Studio, vLLM, llama.cpp, LocalAI, a machine on your LAN, or an external API. Required for reasoning, vision, transcription and video; optional for embedding.

Typical endpoints

Runner	base_url	Installing a model
Ollama	http://localhost:11434/v1	ollama pull <name>, or the in-app catalog
LM Studio	http://localhost:1234/v1	lms get <name> (enable its local server)
vLLM / LocalAI / llama.cpp	your port + /v1	per that server's docs
Home server / LAN	http://<ip>:11434/v1	on that machine
External API	provider URL + api_key	n/a

Ollama gets three extra behaviours through its native API, all optional and all degrading silently elsewhere: bounded embedding context (large VRAM savings), on-demand model unloading, and live inspection of what's loaded. See AI Server Protocol.

The live model catalog

Rather than guessing model names, open the catalog for a slot. Nothing in it is hardcoded — a model released today shows up today.

Sources:

• Embedding — the official MTEB leaderboard API: official aggregates, ranks and specs. You choose which boards matter, including a 1037-language multilingual board next to per-language ones. A model with no score on a board reads as not evaluated, never as zero — conflating those would make an untested model look bad.
• Reasoning and vision — OpenCompass boards (academic reasoning; OpenVLM for vision), normalised to the same 0–1 format with per-board ranks.
• The Ollama library — the full official library, scraped from its server-rendered pages: descriptions, capabilities (vision / tools / thinking), pull counts, absolute update dates, and per-tag quantizations with their real published sizes.

What you can do with it:

• Pick the quantization yourself. 9b-q4_K_M at 6.6 GB versus 9b-q8_0 at 11 GB is the difference between fitting an 8 GB card and not. The catalog shows every tag with its size, and can filter to what actually fits your VRAM.
• Sort by benchmark, popularity or recency.
• One-click download, with a progress bar, resolving the correct Ollama (hf.co/<repo>:<quant>) or LM Studio install name automatically — via a real Hugging Face GGUF lookup, so the name is verified rather than guessed. A model with no community GGUF is reported as such instead of pretending.
• Delete a model from the server to reclaim disk.
• After download, the model appears in the slot's dropdown and selecting it switches the endpoint to where it was installed.
• Filters narrow, they never hide. No hardcoded denylist.

Everything is cached (7 days for benchmarks, 24 h for the Ollama library) and falls back to cache when offline, so the catalog is usable without a connection.

Recommendations by hardware

Starting points, not rules — the catalog exists so you can compare on live data.

8 GB VRAM (e.g. RTX 2080 Super)

• Embedding: stay on the built-in multilingual-e5-small, or move up to multilingual-e5-base if your corpus is large and mostly prose. Multilingual matters more than benchmark rank if your files aren't in English — see the warning below.
• Reasoning: a 7–8B instruct model (qwen2.5:7b, llama3.1:8b). Instruction-following matters most here, since SenseTree asks for strict JSON and tool calls — prefer solid IFEval over maths or code specialists, and prefer a model whose catalog entry advertises tools.
• Vision: something small that co-resides — moondream, qwen2.5vl:3b, minicpm-v.
• Transcription: faster-whisper (small or medium) on CPU is perfectly usable if you aren't in a hurry.

One GPU running reasoning + vision

An 8 GB GPU cannot hold two 7–8B models at once. The runner swaps them on every alternation, making the first call after a swap slow or timed out. Options:

• put reasoning and vision on different machines or GPUs;
• use a small vision model that co-resides;
• switch pipeline_mode to batch, which groups all LLM work before the embedding phase and cuts swaps by a factor of batch_files — SenseTree even unloads the LLMs explicitly between phases;
• or just accept it: transient failures are retried, and only exhaust into a contextual fallback (Troubleshooting).

A second machine with a real GPU

The best setup. Put reasoning, vision and transcription there, keep embedding local (or remote too), and your laptop stays cool while the index fills. Nothing leaves your network.

Gotchas

• Multilingual is invisible in a model name. Only the E5 family is multilingual among the built-in embedding models; the others collapse on a non-English corpus. Measured: English-first models score ~0.2 on Korean where a true multilingual scores ~0.67. The catalog flags this explicitly.
• dimensions must match your embedding model. Test connection reports the real vector length.
• Vision in Ollama needs a real multimodal model with its projector — /api/show must list vision in capabilities. A plain text GGUF pulled from hf.co/... will not do vision.
• Only one Ollama instance. A stale ollama serve reports freshly pulled models as missing (Troubleshooting).
• Ollama does not transcribe. The transcription slot's default deliberately does not point at port 11434.
• Changing the embedding model re-indexes everything. Changing anything else does not.
• External APIs send content off-device. Everything else is local; an external endpoint is the one exception, by your choice.