FrameScout — Offline AI Search: v3.1.0 — Codebase Refactor, GPU Inference & Full Source Release

[bobgsning]

---

FrameScout — Offline AI Search: v3.1.0 — Codebase Refactor, GPU Inference & Full Source Release

Release date: 2026-09-23

This is primarily a maintenance and consolidation release, but it also ships two real improvements: GPU-accelerated embeddings (via DirectML) and video frame indexing (previously broken). It does not change the AI models or search quality, but it substantially cleans up the codebase, fixes a long list of latent bugs, refactors the Tauri/Rust entry structure, and — importantly — publishes the complete project source tree to the repository.

✨ What's New

• ⚡ GPU-Accelerated Embeddings (DirectML) — The Python worker now runs on onnxruntime-directml. SigLIP 2 embedding inference is executed on the GPU via the DirectML execution provider on Windows, delivering a substantial speedup over the previous CPU path (most noticeable for video frame extraction and bulk image encoding). This works across NVIDIA / AMD / Intel GPUs with zero CUDA or driver-version juggling. (EasyOCR text recognition still runs on CPU — only the embedding step is GPU-accelerated.)
• 🎬 Video Frame Indexing (previously broken) — _handle_batch now detects video files and extracts frames via extract_video_frames; images still go through load_image_pil. Each path yields multiple timestamped FrameResult entries, so videos finally enter the index instead of being silently dropped.
• 🗄️ Multi-Frame Data Model — The frame_vectors table primary key changed from path to (path, timestamp). Old single-frame databases are migrated automatically inside a transactional block (rename → create → copy → drop, with rollback on any failure).
• 📦 Complete Source Tree Published — The full Rust, Vue, and Python source — including the model_code module, all build configs (Cargo.toml, ai_worker.spec), docs (BUILD.md, READMEs), and the bundled protoc.exe — is now committed. This is a true full-source Community Edition.
• 🔀 Smart Folder Result Consistency — applySmartFolder now reuses normalizeResults, so smart-folder results behave identically to manual search (videos get a 25s load timeout with "File Unavailable" fallback; low-confidence images auto-collapse). Applying a folder from "Show All" mode no longer leaves the pagination bar stuck.

🐛 Bug Fixes

• "Show All" mode exited on page turn — While in "Show All" (full-file preview) mode, turning pages previously triggered a background re-search/refresh that reset the view back to paginated browsing, kicking you out of Show All. The view mode is now preserved across page turns.
• Worker crash on startup with missing models — main.py now catches FileNotFoundError and prints a clean FATAL message then exits, instead of dumping a raw traceback.
• Latent UnboundLocalError in the worker loop — task_id is now initialized at the top of the request loop, fixing a crash when an exception occurred during recv()/ParseFromString() (where req was never assigned).
• OCR config never honored — The long-dormant single_file_ocr_config proto field (field 6) is now actually read; when set it drives per-file enable_ocr / languages, otherwise behavior is unchanged (always OCR, default en).
• Stale data & inconsistent dedup — Search dedup was verified to preserve each file's highest-scoring frame and its timestamp (used for instant seek); clarifying comments added. index_cmd.rs now refreshes vector_json on ON CONFLICT(path, timestamp) to avoid mixing a first-frame vector with a last-frame OCR text.
• HTML injection in results — escapeHtml now escapes &, ", ' in addition to </>, preventing path text from breaking the v-html rendering.
• HMR event-loss — Four Vue composables (useSearch, useEngineStatus, useLicense, useScanner) now reset lifecycleBound in onUnmounted, so engine-status / scan-progress / key listeners re-bind correctly after hot reload.
• Redundant license refresh — Removed a duplicate emit('refresh-license') + checkLicense() in LicenseModal and the corresponding binding in App.vue.

🔧 Under the Hood

• Tauri entry refactor — lib.rs now owns all 7 module declarations and the Tauri Builder (run() carries #[cfg_attr(mobile, tauri::mobile_entry_point)], Apache header preserved); main.rs is a 4-line shell calling framescout_ui_lib::run().
• Dependency & metadata cleanup — Removed unused tauri-plugin-opener (no frontend calls, no capability); Cargo.toml version 3.0.0 → 3.1.0 (aligned with tauri.conf.json); keywords trimmed to the crates.io 5-item limit.
• Zero-warning builds — Fixed several cfg-gated dead-code/unused-import warnings (guard.rs, verifier.rs, index_cmd.rs, storage/mod.rs); merged the duplicate PaginatedResult struct into PagedResponse; removed a deprecated _state param from ping_engine.
• Python worker hygiene — Replaced __import__("PIL") hacks with a proper from PIL import Image; removed an unused EMBEDDING_DIM import; cleaned leftover comments and a duplicate docstring in video_extractor.py; added a cap.isOpened() guard.
• Packaging robustness — ai_worker.spec no longer hardcodes an absolute .venv path (now resolved relative to SPECPATH), so the build works after moving the venv or the repo. requirements.txt is now pinned to a single ONNX Runtime package (onnxruntime-directml) — the CPU onnxruntime and DirectML packages both provide the onnxruntime module and overwrite each other, so they must never be co-installed. The DirectML build also bundles DirectML.dll for the GPU path.
• Type safety / IDE — Added pyrightconfig.json (correct filename for basedpyright) plus a hand-written search_pb2.pyi stub and type annotations across server.py / config.py / engines / video_extractor.py, bringing the whole inference-worker to 0 errors / 0 warnings under basedpyright.
• Frame-extraction guards — Added MAX_FRAMES_PER_VIDEO and MAX_FRAMES_PER_ENCODING_BATCH (config + constants) so long videos are bounded in time/memory and large batches don't overflow GPU memory; batching also isolates a single-file failure from the rest of the batch.
• Extension whitelist single-source — IMAGE_EXTENSIONS / VIDEO_EXTENSIONS now live in Rust constants.rs and Python video_extractor.py with a cross-language "keep in sync" note; scan_folder uses the constant instead of inline literals.

📋 Upgrade Notes

• No model re-download required — SigLIP 2 ONNX models are unchanged since v3.0.1; existing models/ remain valid.
• Existing databases auto-migrate — Single-frame frame_vectors tables are upgraded to the composite (path, timestamp) key on first launch; your previously indexed images are preserved. (Videos were never stored before, so there is no legacy video data to migrate.)
• If you build from source: re-run pyinstaller per BUILD.md with onnxruntime-directml active (so DirectML.dll is bundled), then copy the complete dist/ai_worker (including its _internal/ folder) into src-tauri/bin/ai_worker — the onedir layout requires _internal/python312.dll to be present.

🙏 Tech Stack

Google SigLIP 2 · ONNX Runtime (DirectML / GPU) · EasyOCR (CPU) · Tauri · ZeroMQ · Protobuf · Rust + Vue 3

Full Changelog: v3.0.3...v3.1.0

The main Tauri application executable stays lightweight (~5 MB); the Python AI worker is bundled as a separate sidecar. Embedding inference now runs on your GPU via DirectML — noticeably faster than before — while OCR remains on CPU.

---

---

This discussion was created from the release FrameScout — Offline AI Search: v3.1.0 — Codebase Refactor, GPU Inference & Full Source Release.