FrameScout — Offline AI Search: v3.1.0 — Codebase Refactor, GPU Inference & Full Source Release

---

FrameScout — Offline AI Search: v3.1.0 — Codebase Refactor, GPU Inference & Full Source Release

Release date: 2026-09-23

This is primarily a maintenance and consolidation release, but it also ships two real improvements: GPU-accelerated embeddings (via DirectML) and video frame indexing (previously broken). It does not change the AI models or search quality, but it substantially cleans up the codebase, fixes a long list of latent bugs, refactors the Tauri/Rust entry structure, and — importantly — publishes the complete project source tree to the repository.

✨ What's New

• ⚡ GPU-Accelerated Embeddings (DirectML) — The Python worker now runs on onnxruntime-directml. SigLIP 2 embedding inference is executed on the GPU via the DirectML execution provider on Windows, delivering a substantial speedup over the previous CPU path (most noticeable for video frame extraction and bulk image encoding). This works across NVIDIA / AMD / Intel GPUs with zero CUDA or driver-version juggling. (EasyOCR text recognition still runs on CPU — only the embedding step is GPU-accelerated.)
• 🎬 Video Frame Indexing (previously broken) — _handle_batch now detects video files and extracts frames via extract_video_frames; images still go through load_image_pil. Each path yields multiple timestamped FrameResult entries, so videos finally enter the index instead of being silently dropped.
• 🗄️ Multi-Frame Data Model — The frame_vectors table primary key changed from path to (path, timestamp). Old single-frame databases are migrated automatically inside a transactional block (rename → create → copy → drop, with rollback on any failure).
• 📦 Complete Source Tree Published — The full Rust, Vue, and Python source — including the model_code module, all build configs (Cargo.toml, ai_worker.spec), docs (BUILD.md, READMEs), and the bundled protoc.exe — is now committed. This is a true full-source Community Edition.
• 🔀 Smart Folder Result Consistency — applySmartFolder now reuses normalizeResults, so smart-folder results behave identically to manual search (videos get a 25s load timeout with "File Unavailable" fallback; low-confidence images auto-collapse). Applying a folder from "Show All" mode no longer leaves the pagination bar stuck.

🐛 Bug Fixes

• "Show All" mode exited on page turn — While in "Show All" (full-file preview) mode, turning pages previously triggered a background re-search/refresh that reset the view back to paginated browsing, kicking you out of Show All. The view mode is now preserved across page turns.
• Worker crash on startup with missing models — main.py now catches FileNotFoundError and prints a clean FATAL message then exits, instead of dumping a raw traceback.
• Latent UnboundLocalError in the worker loop — task_id is now initialized at the top of the request loop, fixing a crash when an exception occurred during recv()/ParseFromString() (where req was never assigned).
• OCR config never honored — The long-dormant single_file_ocr_config proto field (field 6) is now actually read; when set it drives per-file enable_ocr / languages, otherwise behavior is unchanged (always OCR, default en).
• Stale data & inconsistent dedup — Search dedup was verified to preserve each file's highest-scoring frame and its timestamp (used for instant seek); clarifying comments added. index_cmd.rs now refreshes vector_json on ON CONFLICT(path, timestamp) to avoid mixing a first-frame vector with a last-frame OCR text.
• HTML injection in results — escapeHtml now escapes &, ", ' in addition to </>, preventing path text from breaking the v-html rendering.
• HMR event-loss — Four Vue composables (useSearch, useEngineStatus, useLicense, useScanner) now reset lifecycleBound in onUnmounted, so engine-status / scan-progress / key listeners re-bind correctly after hot reload.
• Redundant license refresh — Removed a duplicate emit('refresh-license') + checkLicense() in LicenseModal and the corresponding binding in App.vue.

🔧 Under the Hood

• Tauri entry refactor — lib.rs now owns all 7 module declarations and the Tauri Builder (run() carries #[cfg_attr(mobile, tauri::mobile_entry_point)], Apache header preserved); main.rs is a 4-line shell calling framescout_ui_lib::run().
• Dependency & metadata cleanup — Removed unused tauri-plugin-opener (no frontend calls, no capability); Cargo.toml version 3.0.0 → 3.1.0 (aligned with tauri.conf.json); keywords trimmed to the crates.io 5-item limit.
• Zero-warning builds — Fixed several cfg-gated dead-code/unused-import warnings (guard.rs, verifier.rs, index_cmd.rs, storage/mod.rs); merged the duplicate PaginatedResult struct into PagedResponse; removed a deprecated _state param from ping_engine.
• Python worker hygiene — Replaced __import__("PIL") hacks with a proper from PIL import Image; removed an unused EMBEDDING_DIM import; cleaned leftover comments and a duplicate docstring in video_extractor.py; added a cap.isOpened() guard.
• Packaging robustness — ai_worker.spec no longer hardcodes an absolute .venv path (now resolved relative to SPECPATH), so the build works after moving the venv or the repo. requirements.txt is now pinned to a single ONNX Runtime package (onnxruntime-directml) — the CPU onnxruntime and DirectML packages both provide the onnxruntime module and overwrite each other, so they must never be co-installed. The DirectML build also bundles DirectML.dll for the GPU path.
• Type safety / IDE — Added pyrightconfig.json (correct filename for basedpyright) plus a hand-written search_pb2.pyi stub and type annotations across server.py / config.py / engines / video_extractor.py, bringing the whole inference-worker to 0 errors / 0 warnings under basedpyright.
• Frame-extraction guards — Added MAX_FRAMES_PER_VIDEO and MAX_FRAMES_PER_ENCODING_BATCH (config + constants) so long videos are bounded in time/memory and large batches don't overflow GPU memory; batching also isolates a single-file failure from the rest of the batch.
• Extension whitelist single-source — IMAGE_EXTENSIONS / VIDEO_EXTENSIONS now live in Rust constants.rs and Python video_extractor.py with a cross-language "keep in sync" note; scan_folder uses the constant instead of inline literals.

📋 Upgrade Notes

• No model re-download required — SigLIP 2 ONNX models are unchanged since v3.0.1; existing models/ remain valid.
• Existing databases auto-migrate — Single-frame frame_vectors tables are upgraded to the composite (path, timestamp) key on first launch; your previously indexed images are preserved. (Videos were never stored before, so there is no legacy video data to migrate.)
• If you build from source: re-run pyinstaller per BUILD.md with onnxruntime-directml active (so DirectML.dll is bundled), then copy the complete dist/ai_worker (including its _internal/ folder) into src-tauri/bin/ai_worker — the onedir layout requires _internal/python312.dll to be present.

🙏 Tech Stack

Google SigLIP 2 · ONNX Runtime (DirectML / GPU) · EasyOCR (CPU) · Tauri · ZeroMQ · Protobuf · Rust + Vue 3

Full Changelog: v3.0.3...v3.1.0

The main Tauri application executable stays lightweight (~5 MB); the Python AI worker is bundled as a separate sidecar. Embedding inference now runs on your GPU via DirectML — noticeably faster than before — while OCR remains on CPU.

---

FrameScout — Offline AI Search: v3.0.3 — Backend-Persistent Smart Folders, On-Demand OCR & Storage Cleanup

FrameScout — Offline AI Search: v3.0.3 — Backend-Persistent Smart Folders, On-Demand OCR & Storage Cleanup

Release date: 2026-08-10

This release brings three major improvements: fully backend-driven Smart Folders, on-demand OCR for selected files with custom languages, and complete removal of browser-local storage dependencies. Smart Folders are now truly persistent — surviving database backups, app reinstalls, and cross-machine transfers. OCR can be triggered independently on any indexed file with language-specific settings. Plus, we've fixed the misleading (0) badge on vector-only folders and cleaned up legacy localStorage remnants.

✨ What's New

📁 Backend-Driven Smart Folders

• Smart Folder definitions are stored exclusively in SQLite (smart_folders table).
• New backend command execute_smart_folder performs the search natively using the same engine as manual queries.
• Frontend no longer caches any folder rules — no more lost folders after clearing browser data.
• Each folder displays a live match_count badge. Pure vector folders show ? instead of 0 for honest indication.

🔍 On-Demand OCR for Selected Files

• Run OCR on specific files with custom language codes (e.g., en, ch_sim, ja).
• The run_ocr_for_selected_files command updates both SQLite and in-memory metadata instantly.
• A dedicated OCR button appears on each search result card; click it to trigger OCR with your preferred language.
• Language configuration is now fully separated from the scan/index process — you can index first, OCR later.

🧹 Complete Removal of Browser Storage

• The savePath function and its associated localStorage.setItem calls have been deleted.
• On startup, any remaining framescout_smart_folders or framescout_folder_path keys are cleared.
• The folder path input no longer persists across sessions (future versions may add backend-based preference storage).

🐛 Bug Fixes

• Smart Folder count showing 0 even when matching files exist (vector-only folders now display ?).
• Runtime error caused by calling a commented-out savePath function.
• Potential duplication of Smart Folders due to old localStorage data merging with backend data.
• OCR results not reflected in in-memory metadata after on-demand update.

🔧 Under the Hood

• get_smart_folders now computes match_count by scanning in-memory metadata for text matches.
• SmartFolder struct extended with match_count: usize.
• Frontend applySmartFolder delegates entirely to execute_smart_folder.
• Template updated to conditionally render ? for pure vector folders.
• run_ocr_for_selected_files now properly updates both SQLite and FlatVectorMatrix metadata.
• Scan/index commands accept explicit enable_ocr and ocr_languages parameters.

📋 Upgrade Notes

• No model re-download required — SigLIP 2 models from v3.0.1 remain unchanged.
• Existing Smart Folders in the database are automatically compatible; no migration needed.
• If you previously relied on localStorage for folder path memory, you'll need to re-enter the path after upgrading (we recommend using the Browse button).
• On-demand OCR uses the same EasyOCR models already present in models/easyocr/.

🙏 Tech Stack

Google SigLIP 2 · ONNX Runtime · EasyOCR · Tauri · ZeroMQ · Protobuf · Rust + Vue 3

Full Changelog: v3.0.2...v3.0.3

---

The packaged exe is only about 5 MB.

FrameScout — Offline AI Search: v3.0.2 — Stable Browsing, Index-Time Ordering & “Show All” Mode

FrameScout — Offline AI Search: v3.0.2 — Stable Browsing, Index-Time Ordering & “Show All” Mode

This release focuses on predictability and stability. It introduces index-time ordering so newly scanned files always appear first, a “Show All” mode for viewing every file at once, and numerous defensive fixes against unexpected UI changes.

✨ What’s New

• 📋 “Show All” Mode
  Click the “📋 Show All” button in the pagination bar (or the search toolbar) to load every indexed file in one flat list. A hint bar shows the total count; click “📄 Paginated View” to return to paginated browsing. Works seamlessly with the incoming-file banner and ghost cleaning.

• ⏱️ Index-Time Sorting
  Each indexed file now carries a precise index_time (Unix timestamp). The browse-mode listing (list_all_files) sorts by index_time DESC, so newly added files always appear at the top. No more confusing order after scanning.

🛡️ Stability Improvements

• Request debouncing in performSearch: only the latest search/ page-turn takes effect – no more flickering from stale responses.
• clean_ghosts returns detailed info: frontend accurately removes ghost paths from the current view and reloads appropriately.
• acceptIncomingFiles behaves intuitively: clicking the banner now enters browse mode (first page) or refreshes “Show All” mode – never disrupts your current context.
• Page boundary protection for changePage and doJump.
• clearImageSearch safety guard: does nothing unless actually in image-search mode.
• toggleClustering fixed: exiting clustering now correctly restores the previous result set.

🔧 Under the Hood

• AppState.memory_db upgraded to RwLock – concurrent searches don’t block each other.
• request_vector creates a fresh ZMQ socket per call – eliminates shared-socket race conditions.
• New backend command get_all_files – returns all records in one go, used by “Show All”.
• Database migration: frame_vectors gains index_time REAL column (auto‑migrated on startup).

📋 Upgrade Notes

• No model re‑download required – SigLIP 2 models from v3.0.1 remain unchanged.
• Existing records get index_time = 0.0 after migration. To give them proper timestamps, simply re‑scan your folders. They will then sort correctly.
• Recommended: run a full scan after upgrading to enjoy the best browsing experience.

🙏 Tech Stack

Google SigLIP 2 · ONNX Runtime · EasyOCR · Tauri · ZeroMQ · Protobuf · Rust + Vue 3

Full Changelog: v3.0.1...v3.0.2

The packaged exe is only about 5 MB.

---

FrameScout — Offline AI Search: v3.0.1 & ONNX + SigLIP 2 Upgrade

FrameScout v3.0.1 — ONNX + SigLIP 2 Upgrade

Release date: 2026-08-07

This release brings a major AI inference engine overhaul and several UX improvements. FrameScout now runs on ONNX Runtime with Google SigLIP 2 (768‑dimension embeddings), delivering better semantic understanding, broader GPU support, and faster inference.

✨ What's New

• 🧠 SigLIP 2 Embeddings — Upgraded from OpenAI CLIP (512D) to Google SigLIP 2 (768D). More accurate semantic matching, especially for non-English queries and fine-grained concepts.
• ⚡ ONNX Runtime — Replaced PyTorch with ONNX Runtime for inference. Automatic hardware acceleration detection:
  • DirectML (AMD / NVIDIA / Intel GPUs on Windows)
  • CUDA (NVIDIA GPU)
  • CPU (fallback)
  • Expect 15–30% faster batch inference compared to PyTorch.
• 🔽 Low-Score Collapsible Cards — Low-confidence results are now collapsed by default. Click the red prompt bar to expand details, and use the "▲ Collapse" button to hide them again. Cleaner search results.
• 📦 Fully Offline OCR — EasyOCR model storage is now fixed to models/easyocr/. No unexpected network requests after initial download.

🔧 Improvements

• Model download script updated to fetch SigLIP 2 ONNX models (run python scripts/download_models.py again).
• Logging redirected to %TEMP%\FrameScout_Global\ for easier debugging.
• Improved error handling for video frame extraction with NaN FPS values.
• Better tag display logic for low-confidence results.

🐛 Bug Fixes

• Fixed crash when processing video files with corrupted or unknown FPS metadata.
• Fixed conflicting display between low-score tags and semantic tags.
• Implemented polling loop with PING_ENGINE until engine responds.
• Fixed ZMQ REQ state machine by creating fresh socket per attempt.
• Moved timeout panic out of app.run to ensure early abort.
• Removed unused outer socket variable.
• Main socket now created only after engine is confirmed ready.

🗑️ Removed

• Runtime dependency on PyTorch (still available in requirements.txt for development).

📋 Upgrade Notes

⚠️ Model re-download required.
Due to the switch from CLIP to SigLIP 2, you must re-run python scripts/download_models.py to obtain the new ONNX models. Old CLIP models in models/openai-clip/ are no longer used and can be deleted.

🚀 Quick Start (for new users)

git clone https://github.com/bobgsning/FrameScout.git
cd FrameScout
# Follow instructions in README.md

🙏 Acknowledgments

Built with Google SigLIP 2, ONNX Runtime, EasyOCR, Tauri, ZeroMQ, and the Rust ecosystem.

---

Full Changelog: v3.0.0...v3.0.1

Did you notice? The packaged exe is only about 4 MB.

FrameScout v3.0.0 Community Edition

FrameScout v3.0.0 Community Edition

The first public release of FrameScout — a fully-offline, privacy-first, multi-modal desktop search engine.

✨ Features

• 🔒 100% Offline — No internet required. Your data never leaves your computer.
• 💡 Semantic Search (CLIP) — Search by meaning, not just filenames.
• 🖼️ Image-to-Image Search — Find visually similar images.
• 🔍 OCR Text Search — Extracts and indexes text from images.
• 🎬 Video Frame Indexing — Automatically extracts key frames from videos.
• 🧩 Visual Clustering — Discover groups of similar images.
• 📁 Smart Folders — Save any search as a dynamic folder.
• 📝 Personal Notes — Attach markdown notes to any image.

🔧 Technical Highlights

• Three-process architecture: Vue 3 frontend + Rust/Tauri core + Python inference worker
• Hybrid scoring: vector similarity + OCR + note + filename
• FlatVector matrix: Fast brute-force search for up to 50K images
• ZeroMQ + Protobuf: Microsecond-latency IPC

📋 System Requirements

• Windows 10/11
• Rust 1.75+, Python 3.10+, Node.js 18+

🚀 Quick Start

git clone https://github.com/bobgsning/FrameScout.git
cd FrameScout
# Follow the instructions in README.md

📝 Notes

This is the Community Edition with full source code. See README.md for build instructions.

🙏 Acknowledgments

Built with OpenAI CLIP, EasyOCR, Tauri, ZeroMQ, and the Rust ecosystem.

Full Changelog: https://github.com/bobgsning/FrameScout/commits/v3.0.0