Architecture

Architecture

SenseTree is a Tauri v2 desktop app: a Rust core exposing typed IPC commands to a React 19 / TypeScript / Tailwind v4 frontend.

React 19 + TS + Tailwind v4
Explorer · Search · Meaning tree · Chat/Agent · Settings · Model catalog · Queue · Throughput
                    │  Tauri IPC (typed commands + events)
────────────────────┼──────────────────────────────────────────────────────────
Rust core (Tauri v2)│
                    │
  crawler ──┐                     ┌─ providers ────────────────────────────┐
  watchdog ─┼─▶ SQLite queue ─▶ worker ─▶│ embedding   local fastembed/ONNX │
  folders ──┘                     │      │             or HTTP /embeddings   │
  classifier                      │      │ reasoning   HTTP /chat/completions│
                                  │      │ vision      HTTP + image_url      │
  search ◀── vectordb (LanceDB) ◀─┘      │ transcription HTTP multipart      │
    │  dense + BM25 + RRF + rerank       │ video       HTTP + video_url      │
    │                                    │ reranker · CLIP  (local ONNX)     │
  actions (agent · Dry-Run · rollback)   └───────────────────────────────────┘
    └─ mcp (external tools, HTTP + stdio)

  ┌──────────────────────┐   ┌───────────────────────────────┐
  │ SQLite (r2d2, WAL)   │   │ LanceDB                        │
  │ catalog · queue ·    │   │ `chunks` (dim = model)         │
  │ semantics · folder   │   │ `images` (CLIP, 512)           │
  │ profiles · memory ·  │   │ + BM25 full-text index         │
  │ transaction log      │   └───────────────────────────────┘
  └──────────────────────┘

Data stores

SQLite (rusqlite + r2d2 pool; WAL, busy_timeout, synchronous=NORMAL, foreign keys on):

Table	Holds
file_catalog	Path, parent, name, size, mtime, content_hash.
indexing_queue	status, priority, retry_count, last_error.
indexed_files	Truth table for incremental sync (last_modified, last_seen).
file_semantics	summary (the sense), extract (the content), doc_type, pinned.
folder_profiles	recursive / block / pending, and its source (heuristic, llm, manual).
agent_memory	Durable notes the agent saved.
transaction_log	Dry-Run plans: draft → committed / discarded.

Migrations are additive and tolerant, with a versioned reconciliation that purges derived tables when a database predates the current pipeline.

LanceDB (embedded, serverless) — chunks (id, path, chunk_index, text, content_hash, mtime, vector) with a runtime-mutable vector dimension and a native BM25 index on text; images (path, vector) at CLIP's fixed 512 dimensions. Cosine search with optional path-prefix filtering.

Backend modules (src-tauri/src/)

Module	Role
lib.rs	App setup, shared state, background thread launch, and every Tauri command registration.
state.rs	AppState: DB, config, AI engine, vector store, pause flag, scan registry/epoch, gardener report, MCP discovery cache.
config.rs	Persistent settings.json, defaults, and the ten built-in prompts.
providers.rs	Model-agnostic AI: local embedder, OpenAI-compatible embedding/chat/vision clients, transcription and video clients, cross-encoder reranker, CLIP; AiEngine resolves and caches providers from config.
vectordb.rs	LanceDB: upsert, cosine search, BM25 full-text search, image table, delete/rename, mutable dimension.
db.rs	SQLite layer: pool, schema, migrations, queue, semantics, memory, folder profiles, transactions.
crawler.rs	Initial recursive scan (the "past"); enqueues, schedules re-scans, purges orphans, honours scan epochs.
watchdog.rs	Real-time FS events (the "present"), debounced 2 s; syncs deletions and renames.
worker.rs	The pipeline: route → extract → hash → chunk → embed → store. Vision, OCR, media and contextual paths; sequential and batch modes; retry policy.
parser.rs	File-type routing (text / document / image / media / AI-routing / ignored).
chunker.rs	Structure-aware chunking: paragraphs → sentences → windows, word-safe overlap.
folders.rs	Recursive-vs-block classifier: heuristics, block_bias, LLM decision, deferral.
classifier.rs	Background retry of folders whose classification was deferred.
search.rs	Hybrid retrieval, RRF fusion, reranking, dedup; and the semantic tree.
actions.rs	The agent (ReAct loop, tool schemas, execution), Dry-Run planning, transactional apply with rollback, directory analysis.
mcp.rs	MCP client: HTTP (Streamable HTTP / JSON-RPC, SSE-aware) and stdio transports, discovery cache.
gardener.rs	Periodic read-only structural audit; per-folder health badges.
explorer.rs	Live filesystem listing joined with index status; path details.
metrics.rs	Per-stage throughput counters (vision, media, reasoning, embedding).
benchmarks.rs	Live embedding benchmarks (official MTEB leaderboard API).
catalog.rs	Live vision and reasoning benchmarks (OpenCompass / OpenVLM).
ollama_catalog.rs	Live Ollama library and per-model tags (quantizations, sizes).
ollama_server.rs	Native Ollama observation and control (/api/ps, unload).
installs.rs	Resolves a model's Ollama / LM Studio install name via Hugging Face GGUF lookup.
ort_setup.rs	Provisions ONNX Runtime (dynamic loading, CPU or CUDA).
main.rs	Binary entry point.

Frontend (src/)

File	Role
App.tsx	Three-panel layout, state, indexing polling, pause/resume, update banner.
components/Sidebar.tsx	Indexed roots, gardener health badges, Settings.
components/Explorer.tsx	File list with semantic tags, index status, per-file actions.
components/TreeView.tsx	Meaning-tree view.
components/ChatPanel.tsx	Agent chat: live tool trace, clickable citations, scoped conversations.
components/ActionPlanCard.tsx	Before → After Dry-Run diff, per-operation checkboxes, Approve/Discard.
components/DetailDrawer.tsx	Per-file details: sense vs extract, on-demand qualification.
components/SettingsModal.tsx	All slots, indexing, retrieval, MCP, memory, prompts.
components/ModelCatalog.tsx	Live per-task catalog: benchmarks, Ollama library, quantizations, VRAM fit, download/delete.
components/IndexingQueueModal.tsx	Live queue: current file and its stages, pending, failed, retry.
components/ThroughputPanel.tsx	Per-stage model throughput.
components/ImageSearchModal.tsx	CLIP visual search.
components/GardenerModal.tsx	Directory audit.
components/UpdateBanner.tsx	Signed auto-update: available, download, install, relaunch.
lib/ipc.ts	Typed wrappers over every Tauri command.
lib/types.ts, lib/models.ts, lib/format.ts	Shared types and helpers.

IPC surface

Config & health — get_config, set_config, get_default_prompts, ai_health, test_chat_endpoint, test_embedding_endpoint, gpu_available, app_version.

Indexing — indexing_stats, indexing_queue, get_recent_activity, set_indexing_paused, indexing_paused, retry_indexing, ignore_indexing, retry_all_failed, reindex_path, reindex_all, indexing_throughput, reset_throughput.

Explore & search — list_directory, path_details, open_path, get_roots, add_indexed_folder, remove_indexed_folder, pick_folder, set_folder_mode, semantic_search, semantic_tree.

Sense — set_file_summary, qualify_file, qualify_folder.

Models — list_installed_models, list_local_models, download_local_model, pull_model, delete_model, resolve_installs, model_benchmarks, list_benchmark_boards, vision_benchmarks, list_vision_boards, reasoning_benchmarks, list_reasoning_boards, ollama_library, ollama_tags, ollama_loaded, ollama_unload.

Chat, actions, gardener — chat_with_assistant, plan_reorganization, apply_action_plan, discard_action_plan, analyze_directory, gardener_health, agent_memory_list, agent_memory_delete, agent_memory_clear.

Images — index_images, image_search, image_data_url.

Events (backend → UI) — agent-step (live tool trace), model-pull-progress (download progress).

Concurrency model

• Background threads: one crawler per root (guarded by a registry, with re-scan flagging and scan epochs), one watchdog thread per root, the deferred-folder classifier, and the gardener.
• Async tasks: the indexing worker and every IPC command, on Tauri's Tokio runtime.
• Blocking work is isolated: file reads, hashing, PDF extraction and rendering, fastembed inference and CLIP all run on spawn_blocking, never on the async runtime.
• Lazy loading and unloading: the embedder, reranker and CLIP models are built on demand, cached by configuration key, and released when paused or idle — ONNX Runtime's thread pool spins as long as a session exists.
• Clean shutdown: background work stops, local models are freed, remote models are unloaded from Ollama, and the SQLite WAL is checkpointed.

Key dependencies

Rust — tauri 2 (+ updater, dialog, opener, process plugins), lancedb, rusqlite + r2d2, fastembed + ort (dynamic loading), reqwest (rustls, multipart, stream), notify + notify-debouncer-mini, pdf-extract / lopdf / hayro (page rendering) / image, zip, flate2, sha2, walkdir, infer, arrow-*, tokio + tokio-util, tracing.

Frontend — react 19, @tauri-apps/api, lucide-react, tailwindcss v4, vite, typescript.