Models and Providers

Models & Providers

SenseTree is model-agnostic. Three tasks — embedding, reasoning, vision — are configured independently. Each can run on the built-in local engine or on any OpenAI-compatible HTTP server. Mix freely: e.g. local embedding + reasoning on a LAN box + vision on Ollama.

The three slots

Slot	Purpose	Runs on
Embedding	Vectorize text and captions for search.	Local fastembed/ONNX or /v1/embeddings server.
Reasoning	Chat, reorganization plans, folder descriptions, file-sense extraction.	Any /v1/chat/completions server.
Vision	Image captions and PDF OCR (opt-in).	Any multimodal /v1/chat/completions server.

Local engine vs. HTTP server

• Local engine (embedding only): fastembed runs an ONNX model in-process. No server, no network. ONNX Runtime is provisioned on first use; CUDA is used if use_gpu is set, with a graceful CPU fallback. Ideal default for embeddings.
• OpenAI-compatible server: point a base_url at Ollama, LM Studio, a home server on your LAN, or an external API. Required for reasoning/vision; optional for embedding.

Typical endpoints

Runner	base_url	Install a model
Ollama	http://localhost:11434/v1	ollama pull <name>
LM Studio	http://localhost:1234/v1	lms get <name> (enable the local server)
Home / LAN	http://<ip>:11434/v1	on that machine
External API	provider URL + api_key	n/a

The live model catalog

Instead of guessing model names, open the catalog for each slot. It pulls live leaderboards and lets you download and select with one click:

• Embedding — scores and specs from the official MTEB leaderboard API; you choose which boards matter (multilingual, retrieval, …) and models are ranked by their official leaderboard rank.
• Reasoning and Vision — live benchmarks from OpenCompass (academic reasoning board; OpenVLM for vision).

Catalog capabilities:

• One-click download with a progress bar, resolving the correct Ollama (hf.co/<repo>:<quant>) or LM Studio install name automatically.
• After download, the model appears in the slot's dropdown, and selecting it auto-switches the endpoint to wherever it was installed.
• Filters keep every model visible while letting you narrow to open-source and/or downloadable — no models are hidden by a hardcoded denylist.

Recommendations by hardware

These are starting points, not rules — the catalog lets you compare live.

8 GB VRAM (e.g. RTX 2080 Super)

• Embedding: a small multilingual model — multilingual-e5-small (built-in, CPU-friendly) or a 0.6B GGUF via Ollama. Small, fast, plenty for filename/document sense.
• Reasoning: a 7–8B instruct model (e.g. qwen2.5:7b, llama3.1:8b). Instruction-following matters most here (SenseTree asks for strict JSON), so favor solid IFEval over pure math/code specialists.
• Vision: a small multimodal that Ollama serves natively — moondream, qwen2.5vl:3b, or minicpm-v.

One GPU, both reasoning + vision on the same box

An 8 GB GPU cannot hold two 7–8B models at once. If you run reasoning and vision on the same GPU, the runner swaps models in and out on every alternation, which makes the first call after a swap slow (or time out). Options:

• Put reasoning and vision on different machines/GPUs, or
• Use a small vision model that co-resides with the reasoning model, or
• Accept the swap — SenseTree is resilient to transient vision failures (it retries, and only falls back to context as a last resort).

Notes & gotchas

• Vision in Ollama needs a real multimodal model (with its projector). Ollama's /api/show must list vision in capabilities. Pulling a plain text GGUF won't do vision.
• Only one Ollama instance. A stale ollama serve can report a freshly pulled model as "not found" — restart it. See Troubleshooting.
• External APIs send content off-device. Everything else in SenseTree is local; an external endpoint is the one exception, by your choice.