WABS - v1.2.0

Full Changelog: v1.1.0...v1.2.0

WABS v1.2.0 Release Notes

I am thrilled to announce the release of WABS v1.2.0! This major update brings a Multi-Layer Defense-in-Depth Security Architecture, an expansive Social Taxonomy & Inter-Person Relationship Graph with GEDCOM/PDF Genealogy Exports, a complete Search Engine & Query Builder overhaul, and seamless Self-Healing Configuration & Sidecar Database Management.

---

🌟 Highlights & Key Features

🛡️ 1. Multi-Layer Defense-in-Depth Security Architecture

• Localhost Network Isolation by Default (127.0.0.1): The backend now binds strictly to 127.0.0.1 by default, blocking unauthorized LAN/Wi-Fi devices from accessing your local archive. An explicit toggle in Settings (Allow LAN Access) enables remote phone/tablet access only when explicitly requested.
• CORS Lockdown & Regex Origin Filtering: Removed wildcard CORS origins (*), enforcing strict localhost regex filtering (^https?://(localhost|127\.0\.0\.1)(:[0-9]+)?$) to prevent background web tab data exfiltration.
• Master PIN Authentication (4–12 Digits): Numerical PIN protection hashed via PBKDF2-HMAC-SHA256 (100,000 iterations + 16-byte random salt). Non-numeric input is sanitized automatically across both frontend and backend.
• Cryptographically Secure Session Tokens & Strict Lock Revocation: Issues 32-byte URL-safe session tokens (X-Session-Token). Locking the application immediately revokes the session on the backend and wipes tokens from localStorage & sessionStorage.
• Protected Background Task Execution: Background workers (file indexing, face recognition, object classification, OCR) continue safely in the background on the local machine when the UI is locked, while unauthorized network/device access remains blocked until unlocked.
• Anti-Brute Force Rate Limiting: Enforces a 5-minute cooldown lockout after 5 consecutive failed PIN attempts with a real-time countdown timer.
• Responsive Lock Screen Overlay (LockScreen.jsx): Modal PIN overlay featuring an on-screen touch numpad, lockout timer countdown, setup wizard, and instant unlock.
• Inactivity Auto-Lock & One-Click Topbar Lock: Automatically locks the session after configurable inactivity (5m, 15m, 30m, 1h, Never) or instantly with the new lock icon in the Topbar.
• Dedicated "Security & Privacy" Settings Tab: Centralized panel to configure/change Master PIN, toggle LAN access, set auto-lock timers, toggle AI PII redaction, and manage cache purges.
• AI Privacy & PII Redaction: Automated regex filter that masks sensitive personal identifiers (phone numbers, emails, IP addresses) before prompts or file metadata are sent to external AI providers.
• Thumbnail Cache Management & DAR Guidance: Options to clear thumbnail caches on demand or auto-purge on exit. Integrated Data at Rest (DAR) notices clarifying database encryption recommendations (BitLocker / LUKS / FileVault).
• Security Audit Logging & Test Suite: Full audit logging (wabs.security / wabs.auth) in wabs.log and a dedicated test suite (tests/test_security.py) verifying PBKDF2 hashing, session revocation, rate limiting, and PII masking.

---

👥 2. Social Taxonomy, Inter-Person Graph & Genealogy Exports

• Dedicated Sidecar Database (relationships.db): Isolates user-curated social classifications and family connections from core file indexing (archive.db) and disposable AI face clusters (ai_metadata.db).
• Inter-Person Relationships Graph (person_connections): Connect spouses, partners, parents, children, and siblings directly from any profile. Reciprocal connections (spouse ↔ spouse, parent ↔ child, sibling ↔ sibling) are synchronized automatically.
• GEDCOM 5.5.1 Genealogy Export: Full export compatibility with standard genealogy software (Gramps, Ancestry, FamilySearch). Export the entire tree or family subtrees with individual records (INDI), family units (FAM), gender inference, and WABS lineage metadata.
• PDF & Printable Tree View: Instant vector PDF export and print formatting for the full relationship graph or isolated family sub-branches.
• Rescan & Wipe Protection (Stable Person Registry): Relationship links and social structures remain 100% intact even if AI face metadata is wiped or re-indexed, auto-relinking when profiles are rescanned.
• "Who Am I?" Identity & Badging: Set your own profile (me_name) in Settings ➔ General to anchor relative kinship branches (Spouse, Parents, Children, Siblings, Grandparents, Cousins, In-laws). Viewing your own card features a distinct blue border and dedicated "Me" badge.
• Interactive Multi-Column Tree View (RelationshipTree.jsx): Multi-column responsive card hierarchy displaying Family, Friends, and Others side-by-side with avatar thumbnails, photo counters, branch badges, subtree exports, instant search, and expand/collapse toggles.
• Dedicated Tabbed Navigation for People:
  • People Tab (Default): Complete profile grid with Category Filters (All, Family, Friends, Others, Uncategorized), sorting, and face clustering tools.
  • Tree View Tab: Dedicated view for the full Multi-Column Relationship Tree.
• Person Gallery Photo Cache & Scroll Retention: Seamless memory caching (personGalleryCache) and scroll restoration (savePersonScroll) so navigating back and forth between People and photo galleries preserves loaded pages and exact scroll positions.
• Explicit Relationship Confirmation: Inline editor in Person view with buffered changes and explicit Save (✓) and Cancel (✕) buttons (with Enter and Escape shortcuts) to prevent accidental database edits.
• Extended Kinship Support: Built-in "In-law" classification (Father-in-law, Mother-in-law, Brother-in-law, Sister-in-law) integrated directly into the hierarchy tree.

---

🔍 3. Comprehensive Search Engine & Query Builder Overhaul

The search engine (search.py) has been refactored with a robust tokenization parser supporting advanced search patterns:

• Compound Size & Range Filters: Supports compound comparisons (size:>100MB, <5GB, size:>100MB <5GB), ranges (size:100MB-5GB), comparisons (size:>100MB, size:<=2GB), and standard/shorthand units (B, KB, MB, GB, TB, PB, k, m, g, t, p).
• Duration & Media Length Filters: Filter audio/video by ranges (length:5m-1h), compound comparisons (length:>5m, <1h), timestamps (length:>01:30, length:>01:30:00), and abbreviations (s, m, h).
• Advanced Date & Year Range Filters: Multi-date lists (date:2020-2022, 2023-10-25), year ranges (2020-2022), comparisons (>2020, <=2023-01-01), exact dates (YYYY-MM-DD, MM-DD-YYYY, DD-MM-YYYY), and specific years/months (2023, 2023-10).
• Category & Extension Filters (type:): Filter by category (type:photo, type:video, type:audio, type:document), multi-type lists (type:audio,video), or exact file extensions (type:mp3, type:pdf).
• Case-Insensitive Tag, Object & Person Matching: Normalized searches for namespaced (person:John Doe, object:car), plain (car), quoted (person:"john doe"), or partial (person:john) queries.
• Wildcards & Boolean Search Semantics: Full wildcard matching with * and ? (*.mp3, *vacation*). Terms default to OR, + enforces required matches across all attributes, and - excludes terms.
• Dynamic Autocomplete & Prefix Suggestions: Real-time suggestion drawer provides autocompletion pills for object:, person:, tag:, and type: prefixes along with keyword autocompletion and did-you-mean spell checks.
• Instant Search on Enter: Pressing Enter in the search bar executes immediately, bypassing debounce timers.

---

💾 4. Data Management & Configuration Upgrades

• Self-Healing Configuration Schema: load_config() automatically backfills missing configuration keys on startup into config.yaml, while save_config() safely merges partial frontend updates without dropping untouched fields.
• Data Management Exports: Added standalone Export / Import JSON for Relationships & People Categories in Settings, as well as GEDCOM (.ged) and PDF exports directly from the Relationship Tree.
• Unified Backup & Cleanup: Combined backups and Full Database Backups now include relationships.db alongside archive.db, ai_metadata.db, and config.yaml. Database Cleanup now purges dead relationship links and vacuums relationships.db.

---

🐞 Bug Fixes & Refinements

• Tree Subgroup Collision Fix: Resolved a UI crash (TypeError: familySubgroups.children.push is not a function) when assigning kinship to Children/Sons by renaming internal subgroup keys to avoid property collisions.
• Sidecar Database Directory Auto-Creation: Fixed startup failure (unable to open database file) when starting WABS with clean or custom database folders by ensuring directories exist prior to database initialization.
• Python Inner Import Shadowing Fix: Removed shadowing import sqlite3 inside _process_unified_scanners in indexer.py that caused cannot access local variable 'sqlite3' runtime exceptions.
• Frontend Comma Stripping Fix: Removed destructive comma stripping in useExplorer.jsx and App.jsx that interfered with compound search queries (e.g. size:>100MB, <5GB).
• Search Pagination Reference Error Fix: Fixed an undefined safeQuery variable in loadPrevious in useExplorer.jsx, restoring smooth reverse-scrolling on search result views.
• Automated Search Pattern Test Suite: Integrated a 22-test automated suite ([tests/test_search_patterns.py](...

WABS - v1.1.0

Full Changelog: v1.0.1...v1.1.0

I am excited to release WABS v1.1.0! This release introduces nested virtual folders, physical and virtual directory actions, audio cover art extraction, export progress tracking, and critical performance and stability hotfixes for large archives.

---

🚀 Major New Features & Enhancements

📁 Hierarchical Virtual Folders & Export

• Indefinite Nesting: Create, rename, and organize virtual folders recursively into a parent-child tree navigation structure, complete with breadcrumbs navigation.
• Direct Disk Export: Export virtual folders and all nested contents recursively to external drives using a native OS-level folder picker (Windows/Linux).
• Progress Tracking & Control: Real-time progress bars and file copying states for exports, along with instant cancellation support in the Explorer and Virtual Folders views.
• Dynamic Queries & Hybrid Folders: Apply dynamic filter queries (e.g., tags or person:Name) to virtual folders. The matches are automatically combined with manually linked files.
• Folder Customization: Personalize your workspace with a curated palette of 10 modern colors and 11 descriptive icons per virtual folder.
• Recursive Operations: Extended move, copy, and delete endpoints to handle directories and virtual folder paths (virtual_folder:{id}) recursively on disk and in the database.

🎵 Audio Metadata Extraction & Placeholders

• Cover Art Extraction: Automated extraction of embedded cover/album art from MP3 (ID3 APIC), MP4/M4A (covr), FLAC, OGG, and WMA/ASF audio formats.
• Cached Thumbnails: Cached cover art is scaled and saved to .wabs_cache/audio/ for fast preview rendering.
• Indigo Fallbacks: Designed modern indigo-themed SVG placeholders for audio files without cover art to match the active UI theme.

⚙️ UI, Settings & Navigation Improvements

• Locate Dropdown: Upgraded the file locator action to select between locating files in the Folder Tree, Flat List, or within any of their containing Virtual Folders.
• Offset Pagination Cache: Integrated scroll and page-offset caching when locating files outside the first chunk of 50 files.
• Backup Nudges: Displays a notification nudge if settings are saved without any configured backup locations, pointing directly to the Backup settings tab.
• Layout Persistence: Width preferences for the sidebar, timeline, and details panel are now saved to config.yaml to persist across restarts.

---

🐞 Bug Fixes & Refinements

• SQLite Variable Limit Resolution (Hotfix): Resolved the sqlite3.OperationalError: too many SQL variables crash when loading or searching folders matching tens of thousands of files by moving queries to temporary tables and chunking parameter lists in batches of 900.
• Clean Export Progress UI (Hotfix): Redesigned the export progress bar panel and cancellation button to prevent crowding and text overlapping in the sidebar and folder cards.
• Graceful Export Shutdown (Hotfix): Updated the application's shutdown hook (graceful_os_shutdown) to register the export background thread, allowing active operations to terminate and clean up before exit.
• Root-Only Dashboard Count: Fixed the dashboard stats to only display root-level virtual folder counts.
• Tray Icon & Server Shutdown: Resolved Windows system tray hangs and uvicorn keep-alive socket leaks by introducing force-exit triggers.

WABS - v1.0.1

Full Changelog: v1.0.0...v1.0.1

WABS v1.0.1 Release Notes

We are excited to announce the release of WABS v1.0.1! This release focuses on massive performance optimizations (specifically memory management and lazy loading), critical bug fixes, improved scanning accuracy, robust temporary folder handling on Windows/Linux, and database cleanup routines.

---

⚡ Performance & Caching

• Compiled ImageNet Mapping: Converted ImageNet class mapping from external JSON to a compiled Python static dictionary (imagenet_mapping_data.py), loading instantly via Python bytecode (.pyc) to bypass disk reading and JSON string parsing at runtime.
• On-Demand Lazy Loading & Module Release: Machine learning models (ONNX models for face, object, and OCR detection) and heavy libraries (like onnxruntime and rapidocr_onnxruntime) are now loaded strictly on-demand. When the application goes idle, a memory release routine automatically unloads them and triggers garbage collection (gc.collect()) and dynamic heap trimming (malloc_trim) to free system RAM.
• OpenCL DNN Backend Fallback Disabled: Removed OpenCV OpenCL DNN backend fallback due to lockups/freezes caused by dynamic OpenCL convolution kernel compilations, defaulting back to CPU (CUDA remains active for NVIDIA GPUs).
• Face Scanner Optimization: Skipped face scans on very small images and icons (dimensions < 100px) to prevent false-positives and accelerate scan speeds.
• Unified Pipeline Reuse: Redundant database/YuNet scans are avoided by reusing cached face records, and each file is decoded exactly once per scan step across face, object, and OCR pipelines.
• Extended Memory Controls: Added 15 minutes and 2 hours idle memory release timeouts in Settings.
• Added lightning-fast in-memory face exemplar curation: Optimized scanner loops to eliminate CPU bottlenecks, and refined default profile thumbnails using centroid face matching.

🐞 Bug Fixes & Refinements

• Name-Based Preference Persistence: Favorite (pinned) and hidden profiles are now persisted by name (or ID for unknown profiles) in user preferences, resolving issues where pinned profiles were lost or mismatched after re-indexing or importing Known/Names people.
• Face-Aware Animal Thresholding: Elevated classification thresholds for animal/pet categories (e.g., dog, cat) to a strict minimum of 0.45 if a human face is present in the image, eliminating false-positive animal tags on family photos.
• Improved Object Classification Accuracy:
  • Implemented probability aggregation (e.g., summing subclasses like dog breeds to dog, animal, and pet).
  • Added a 1.5% subclass noise floor threshold to suppress false-positives on screenshots/documents.
  • Retained specific tags (e.g., object:golden_retriever) alongside high-level tags when confidence exceeds sensitivity thresholds.
• Transparent Image Compositing: Added transparent-to-white background compositing for PNG/GIF/WebP files, preventing transparent icons from being misclassified by the object scanner (e.g., classifying a transparent hexagon as a "spatula").
• Aspect Ratio-Preserving Letterboxing: Resizes objects to a white 224x224 canvas to improve classification accuracy on non-square photos.
• System Tray & Shutdown Fixes:
  • Silenced cosmetic Uvicorn shutdown exceptions (KeyboardInterrupt and asyncio.CancelledError).
  • Suppressed pystray icon docking tracebacks on Linux/Ubuntu during shutdown.
  • Tied system tray icon termination to the Web UI shutdown endpoint (/shutdown) to prevent terminal hangs.
• Other Fixes: Resolved an UnboundLocalError for OCR during transparent image metadata passes, added timestamps to verbose console logs, and implemented uncropped source photo fallback for person cover profiles if models are missing.

🛡️ Safety & System Cleanliness

• PyInstaller Temp Folder Locking (Windows): Locks static assets in the temporary extraction folder (_MEIPASS) by keeping read-only file descriptors open during process execution. This prevents Windows from allowing manual deletions or system disk cleanups while the application is running, while still allowing clean folder deletion on process exit.
• Dynamic Model Health Checks: Checks for required AI models during status polls and displays a warning banner at the top of the React UI if any are missing. Checks are also run prior to starting scans, returning a clean HTTP 400 instead of failing silently in background threads.
• Atomic Directory Renaming on Exit: Replaced legacy lockfiles with an atomic folder-renaming strategy on shutdown (renaming _MEIxxxxx to _MEIxxxxx_to_delete first to ensure DLL locks are checked before deletion).
• Orphaned DLL Temp Folder Purger: Added a startup routine to scan and purge legacy _MEI temp folders containing only C-runtime DLLs (msvcp140.dll, etc.), safely reclaiming disk space without affecting other active applications.
• Strict Deletion Paths: Reinforced deletion constraints to ensure they are strictly subdirectories of the system's root temp directory.

🧹 Database Cleanup & Optimization

• Orphaned Records Purging: Enhanced the database cleanup routine to delete orphaned entries in faces, processed_files, processed_objects, processed_text, and file_text_fts tables that reference missing/deleted files.
• Empty Profile & Cache Cleanup: Automatically deletes empty people profiles (zero face detections) and removes their cached thumbnail files (person_{person_id}.jpg) from disk.
• SQLite Database Vacuuming: Runs a SQLite VACUUM on both the main database (archive.db) and the AI database (ai_metadata.db) at the end of the cleanup routine to reclaim physical disk space and optimize queries.

WABS - v1.0.0

WABS v1.0.0

We are thrilled to announce the official v1.0.0 stable release of WABS! This milestone release represents months of optimization, architectural modularity, performance enhancements, and the addition of major features like offline OCR, system tray execution, automatic startup launch, and advanced hardware acceleration.

---

🚀 Key Highlights & Major Features

🔍 Offline OCR (Optical Character Recognition) Integration

WABS now features fully local, offline RapidOCR text extraction powered by PaddleOCR models (paddleOCR_det.onnx, paddleOCR_rec.onnx).

• Full-Text Search (FTS5): Text from images, photos, and scanned PDFs is automatically extracted and indexed into SQLite's FTS5 engine, making receipts, invoices, screenshots, and documents instantly searchable.
• Smart Photo Filtering (ocr_only_no_ai_tags): Prevents scanning family or scenic photos by automatically skipping OCR on files with detected faces/objects (enabled by default), drastically optimizing database size and scan time.
• Scale-On-Decode (OCR): Large images (>3000px) are decoded at 1/2 size using OpenCV's scale-on-decode capabilities to save substantial memory and CPU/GPU cycles.
• Image Scaling Safety: Automatically skips OCR on extremely small images (<20px) and pads tiny photos (<150px) to prevent PaddleOCR engine upscaling crashes.

⚙️ Hardware & GPU Acceleration

Unleash the full speed of your hardware:

• ONNX DirectML & CUDA support: Automatic detection and configuration of GPU acceleration. On Windows, execution is routed to DirectML (DmlExecutionProvider) and CUDA if present, falling back gracefully to CPU.
• OpenCV DNN OpenCL target: Activates OpenCL (cv2.ocl.haveOpenCL()) for DNN model operations when no CUDA-compatible card is found.
• GPU Stability Fallback: WABS performs self-check test runs for the Face Detector, Face Recognizer, and Object Classifier on startup. If a GPU driver error occurs, it dynamically falls back to CPU to ensure uninterrupted service.

📦 Optimized Memory Management & Startup

• Lazy Deferred Imports: Startup time has been slashed from ~8 seconds to under 2 seconds by deferring heavy imports (OpenCV, fitz, docx, mutagen, etc.) to runtime execution.
• Idle Memory Monitor: A background thread monitors application idle states. It automatically unloads unused libraries from the Python module cache and vacuums SQLite caches when inactive, releasing memory on a configurable timer (5m, 10m, 30m, 1h).
• Single-Decode Pipeline: Loads and decodes image files exactly once, reusing the same memory-mapped representation for Face, Object, and OCR detectors.

🖥️ Desktop Integration

• System Tray (Taskbar) Support: WABS now runs as a system tray app using pystray. Right-clicking the tray icon lets you open the dashboard, open Settings, or gracefully shutdown the backend.
• Auto Run on Startup: Configurable toggle to automatically launch WABS on user login in background mode (hidden terminal console and --no-browser) for Windows (Registry startup keys) and Linux (XDG autostart entry).

---

🛠 Bug Fixes, Refinements & Build Assets

• EXIF Portrait Photo Rotation: Portrait photos are rotated natively inside OpenCV based on their EXIF orientation headers, improving detection rates for YuNet face models and PaddleOCR.
• Vectorized Face Similarity: Stacked face embeddings into a 2D NumPy array to perform matrix-vector dot products via np.dot in matching loops, accelerating face-search routines.
• In-Memory Batch Buffering: Newly found face clusters are buffered in-memory (new_embs / new_ids) during scanner runs, allowing subsequent photos in the same batch to be matched against them immediately without hitting the database.
• Custom PyInstaller Build Spec: Introduced WABS-Windows.exe.spec to slim down the compiled Windows installer by excluding unused large libraries (like pyarrow and tzdata) while bundling resources properly.
• Cleaner Diagnostics: Added user-friendly log reports when scanning or indexing backup paths and when invalid roots are encountered.

---

Full Changelog against v1.0.0-beta.8: v1.0.0-beta.8...v1.0.0

WABS - v1.0.0-beta.8

WABS v1.0.0-beta.8 Release Notes

WABS v1.0.0-beta.8 introduces major architectural improvements, randomized auto-pick cover selection, instant indexer start/stop performance optimizations, in-memory face exemplar curation with chronological and scale diversity, standardized comma-separated tag delimiters, DNG RAW photo support, graceful background task shutdown, fast JPEG scale-on-decode, multi-scale face detection fusion, smart offline drive cleanup protection, and modular frontend/backend routing.

What's New in v1.0.0-beta.8

🚀 Major Features & Architectural Enhancements

• Randomized Auto-Pick Cover: Refactored the "Auto-Pick Cover" feature to dynamically select random photos from the top candidate covers (within 50% of the best score, up to 5 candidates). It excludes the current cover photo (if others exist) to rotate covers and prevent getting stuck on the same image.
• Indexer Start/Stop & Lazy Preloading: Added stop/shutdown checks inside directory walking (os.walk) loops in both the main indexer and unified scanners. Replaced full table metadata preloading on startup with dynamic chunked lazy preloading (1,000 files at a time) during runs. Bypasses completion delays on manual stop, making start/stop instant even on massive databases.
• Smart Cleanup Safety Filter: Protects configured backup paths that are currently offline (e.g. unplugged external USB drives or network shares) from being accidentally purged during database cleanup operations. Compares directories using a drive-letter-invariant matching method.
• Standardized Tag Delimiter: Standardized all database file tags to be comma-separated , instead of space-separated, enabling correct indexing, display, and search for multi-word tags (e.g., person:John Doe or object:cell phone). Implemented robust synchronization and backward compatibility in parsing.
• DNG RAW Photo Support: Added full raw DNG photo indexing, face/object scanning, and browser-compatible JPEG preview rendering fallbacks.
• Modular Codebase: Refactored the monolithic backend (main.py) and frontend (App.jsx) into clean, modular routes, pages, hooks, and helpers.
• Unified Document Scanning Depth: User-configurable scanning depth (low, medium, high) dynamically scales limits across PDF, Word, Excel, PowerPoint, and plain-text files.
• Memory-Optimized Hybrid Document Scanning: Tokenizes text within configured limits and switches to regex matching for strong identifiers (emails, URLs, hashtags) beyond limits using 128KB chunks and a 1,000-identifier early breakout.
• Global Background Task Indicator: Added a persistent, globally visible pulsing LED indicator to the bottom-left of the application. It dynamically changes colors and text to track active background tasks (face scanning, clustering, file indexing, purging) across all pages.
• Context-Preserving Capping: Limits unique boosted identifiers (top 50 strong entities, top 100 proper nouns) to prevent log files or link directories from crowding out core context keywords.
• High-Efficiency Face Export & Import: Serializes 128D embeddings as compact Base64 float32 byte arrays, reducing backup file size by ~72%.
• Memory-Safe Frontend Downloads: Replaced browser data-URIs with native Blob and object URL streams to prevent browser freezes when exporting large databases.

⚡ Performance & Caching

• Fast JPEG Scale-on-Decode: Integrates header-only dimension reads (via Pillow) and leverages OpenCV's native scale-on-decode capabilities (cv2.IMREAD_REDUCED_COLOR_*) to decode large images at 1/2 or 1/4 size directly, reducing image load times by 5x-10x.
• Multi-Scale Face Detection Fusion: Runs two concurrent detection passes when face_sensitivity is set to high (1024px for small/distant background faces, and 320px for large foreground close-ups) and fuses overlapping boxes using Non-Maximum Suppression (NMS) to capture both background and foreground faces accurately.
• Dynamic Chronological & Size Curation: Implemented a central curation helper (get_or_create_exemplars) that curates up to 15 exemplars. For profiles > 15 faces, it samples 50 timeline-distributed files, filters out the bottom 25% blurriest using Laplacian variance, and selects the oldest, newest, middle, smallest, and largest faces before backfilling. Includes a 0ms SQL bypass for small profiles (<= 15 faces).
• In-Memory Caching on Import: Caches full SQLAlchemy FileIndex model objects by path during import, eliminating redundant database queries.

🐞 Bug Fixes & Refinements

• Scanner Graceful Shutdown: Spawns scanner threads as daemon threads and integrates APP_SHUTTING_DOWN state monitoring to instantly terminate loops on application exit.
• Background AI Race Condition Fix: Prevented a KeyError crash in EXEMPLAR_CACHE invalidation and added gracefully handled 404 warnings when the UI attempts to interact with profiles actively being merged or deleted by background clustering.
• Persistent Cover Photo Cache Busting: Fixed aggressive browser caching preventing UI updates after changing a profile's cover photo by injecting dynamic ?v= fingerprints into thumbnail URLs.
• UI State Preservation: Prevented AI bulk operations menus from collapsing when navigating away from and returning to the People page while tasks are running.
• Manual Tagging Logs: Added logging coverage for manual tagging/untagging operations under the backend enable_logging preference.
• Untagged Media Count Fix: Solved a SQL query bug where NULL evaluations returned near-zero counts for untagged photos.
• Tags Export NameError: Fixed a NameError crash inside /system/export-tags by adding the missing load_config import.
• Transaction Safe Imports: Wrapped the import routines in database transactions to roll back sessions on failure.

---

Full Changelog: v1.0.0-beta.7...v1.0.0-beta.8

WABS - v1.0.0-beta.7

What's Changed in v1.0.0-beta.7

Full Changelog: https://github.com/wizwin/WABS/commits/v1.0.0-beta.7

🚀 Major New Features

• Document Text Extraction & Search: Introduced a powerful new text extraction pipeline for documents (PDF, DOCX, PPTX, XLSX, and text/code files). Extracted keywords are indexed into a new FTS5 virtual table (file_text_fts), allowing you to instantly search for files based on their contents. Added dedicated start/stop/reset controls in the UI for the document scanner.
• AI Search Assistant: Added experimental support for LLM-powered natural language searches. WABS can now translate natural language prompts into valid WABS search queries (e.g., "Find photos of cars from 2022"). Configurable in Settings with connection testing.
• Favorite (Pinned) People: You can now pin or favorite specific people on the People page. Pinned individuals will always remain sorted at the top of the grid for quick access.
• Hardware Acceleration: Added hardware acceleration support for OpenCV video frame extraction and CUDA support for DNN (Deep Neural Network) modules, significantly improving media processing speeds.

✨ UI/UX Enhancements

• Multi-Person Manual Tagging: Replaced the single-select auto-assign dropdown with a robust multi-select component (Select + Add/Remove buttons) for manual person tagging.
• Intelligent Undo & Refresh: Vastly improved the undo action for tagging. Reverting a tag now flawlessly refreshes the UI, dashboard statistics, and maintains your selected file states without manual page reloads.
• Bi-Directional Pagination & Lazy Rendering: The frontend now supports bi-directional pagination and utilizes IntersectionObserver for lazy rendering of Date Groups, providing buttery-smooth scrolling on massive archives. Chunk limits are now configurable in Settings.
• Streaming API Responses: Converted file, search, and people-photo endpoints to use streaming JSON responses, dramatically reducing frontend wait times when loading large sets of results.
• Advanced Date Normalization: Enhanced EXIF date parsing and formatting (json_extract caching). The timeline and date-range queries now correctly handle diverse formatting, including colon replacement.
• Sort Presets & Propagation: File sorting preferences are now persisted to localStorage and correctly propagated to backend file and search requests.
• Merge People Improvements: The flow for merging people now proactively surfaces potential name conflicts to prevent accidental overwrites.
• Image Loading & Retry: Implemented automatic image retry with exponential backoff and error handling for robust thumbnail loading.

🐞 Bug Fixes & Performance

• Path Mapping & Safe Deletion: Added configurable path_mappings during indexing to normalize paths across different OS environments. Removed the risk of accidental deletions by gating missing-file removal behind allow_delete_missing or explicit force_reindex flags.
• Binary Newline Counting: Switched text/code line counts to highly optimized binary newline counts, greatly improving performance on large code repositories.
• LLM Provider Enhancements: Hardened the AI tagging integration (llm_classify) by normalizing provider URLs, intelligently appending /chat/completions, including file metadata in prompts, and sanitizing LLM responses.
• Database Startup Optimizations: The backend now creates necessary DB expression indexes automatically at startup and safely suppresses harmless Windows asyncio ConnectionResetErrors.
• Exact Match Operator: Added support for the = operator in size and length filters (e.g., size:=1MB).
• Keyword Limit Configuration: Introduced a new text_extraction_limit (default 300 words) setting in the backend, fully exposed to the frontend Settings UI, giving you control over document scanning depth.
• Memory & Resource Safety: Improved OpenCV backend selection, added safety limits to prevent crashing on exceptionally large EXIF blobs, and improved text-to-SVG generation to prevent memory spikes.

WABS - v1.0.0-beta.6

Full Changelog: https://github.com/wizwin/WABS/commits/v1.0.0-beta.6

WABS v1.0.0-beta.6: Smart Covers, Cluster Unknowns, Light Theme & Massive AI Optimizations! 🚀

This release introduces major quality-of-life improvements to Face Recognition, including the highly requested ability to automatically pick the sharpest cover photos, safely cluster similar unknown faces to reduce clutter, a beautiful new Light Theme, and a highly requested Undo button for photo tagging!

Under the hood, we've completely eliminated UI freezing on massive databases, implemented a thread-safe LRU Exemplar Matrix Cache, and vectorized the AI similarity engine. Face clustering and searches are now lighting-fast, heavily optimized, and highly accurate. ⚡

🚀 Major New Features

Cluster Unknowns: Added a powerful new "Cluster Unknowns" feature on the People page. It allows you to instantly compare Unknown Persons against other Unknown Persons and automatically merges them if they meet your configured Similarity Threshold, significantly reducing clutter before you assign names.

Reclassify Unknowns: Added a powerful new "Reclassify" feature that breaks apart wrongly-clustered Unknown profiles and re-evaluates every single face against all Named profiles and other Unknowns using your current Similarity Threshold.
Light & Dark Theme Support: Full application theme toggling added to Settings, utilizing a highly-efficient CSS inversion approach that perfectly preserves media and icon colors.

Auto-Pick Cover Photo: A new smart UI button that automatically evaluates bounding box sizes and Laplacian variance (sharpness) across a person's photos to pick the best possible thumbnail.

Purge Small Unknowns: A dedicated database management routine in Settings allowing you to instantly delete noisy "Unknown Person" profiles that contain fewer than a specified number of photos.

Direct Move to Person: You can now select specific photos from any profile (or Unknown group) and instantly reassign them to another named person via a convenient UI dropdown.

Hidden People: Added the ability to explicitly "Hide" specific Known or Unknown people from the UI and Search Auto-Suggest. Hidden profiles are kept in the database to prevent the AI from repeatedly rescanning them.

Video Extraction Features: Added native video container date parsing and metadata tag parsing via mutagen. Video files now map perfectly into the chronological timeline alongside your photos.

✨ UI/UX Enhancements

Live Chunked Progress Bars: Re-engineered the frontend to send bulk AI operations in safe chunks of 250 profiles. This prevents HTTP connection timeouts on massive 30,000+ face databases and provides a smooth, real-time progress bar.

Long-Running Task Cancellations: Added the ability to gracefully cancel massive AI clustering or reclassification tasks midway. The UI dynamically switches the action buttons to "Cancel" during processing.

Undo Action for Tagging: Added an intuitive "Undo" button to the Toast notifications with a comfortable 5-second timeout, allowing you to instantly reverse accidental photo assignments, removals, or profile moves!

AI Actions Menu: Grouped all advanced similarity sliders, clustering, and purge operations into a clean, collapsible "AI Actions" panel to keep the main People UI uncluttered.

Similar Faces Pagination & Sorting: The "Find Similar Unknowns" panel now supports pagination (500 profiles per page) and intelligently sorts ties by matching photo counts and directory/time context.

Lightning-Fast Thumbnails: Replaced artificial JavaScript load delays with native HTML lazy-loading and aggressive backend Cache-Control headers, instantly snapping thumbnails into view from browser memory.

Refined Similarity Threshold: Adjusted the default Cosine Similarity threshold to 55% (and lowered the slider minimum to 35%) to drastically improve matching on side-profiles and diverse lighting.

Intelligent Scroll Memory: Navigating back to the People page from a specific person's photo grid now automatically snaps you back to your exact scroll position and active page.

Thumbnail Debouncing & Fade-ins: Added a 250ms Javascript debounce wrapper to face thumbnails and smooth CSS opacity fade-ins to prevent backend spam and eliminate harsh image popping.

🐞 Bug Fixes & Performance

React O(N²) Render Bottleneck: Fixed a catastrophic browser freeze when selecting thousands of unknown profiles by replacing inline array .find() and .sort() operations with highly optimized O(1) Hash Maps and strict memoization.

LRU Exemplar Matrix Cache: Implemented a highly optimized, thread-safe memory cache that builds a curated 25-photo baseline for each person. It dramatically improves AI accuracy by dropping blurry outliers and speeding up matrix multiplications.

Vectorized AI Similarity: Replaced slow native Python math loops with highly optimized numpy vector operations for cosine similarity, slashing face clustering times.

Comprehensive Backend Logging: The backend now fully logs summary statistics for all Bulk AI operations (Auto-Pick Cover, Merges, Clusters, Reclassifications) to the wabs.log file when Background Logging is enabled.

Dynamic Cache Invalidation: The similarity caches are now explicitly invalidated and recalculated instantly whenever you rename, merge, delete, or manually tag a person.

Database Synchronization & Ghost Faces: Fixed a major bug where the ai_metadata.db retained "ghost" faces for files that were moved or deleted. The System Cleanup routine now flawlessly cross-references the main database and completely purges all orphaned AI records.

Theme-Aware SVG Placeholders: The backend now dynamically generates offline text and document preview SVGs based on your active UI theme, fixing invisible black text in Light Mode.

Group Photo Speedup: Added a 98% match early-exit optimization for face thumbnail extraction. The backend no longer wastes computationally expensive AI cycles checking every single face in massive group photos once the target person is found.

Pillow (PIL) Media Fallbacks: Added robust Pillow fallback logic to both the large photo caching routine and the face cropper, completely fixing missing thumbnails for modern formats (like .webp) that OpenCV silently fails to decode.

Start Scan In-Memory Optimization: The Indexer now pre-fetches all existing file paths into a Python set when starting a scan. O(N) database queries have been replaced with O(1) in-memory lookups, drastically speeding up indexing.
(Includes many more crucial fixes for directory exclusions, orphaned thumbnail cleanup, lazy hashing logic, and SQLite optimizations!)

🛠 Build & CI Updates

Windows Executable Polish: The GitHub Actions Windows build now automatically uses the --noconsole flag (hiding the background command prompt) and utilizes per-matrix icon embedding for a custom application icon (.ico).

ARM Compatibility: Switched the GitHub Actions ARM build runner to ubuntu-22.04 for enhanced build stability and compatibility.

📦 Installation

Download the appropriate executable for your Operating System from the Assets section below. (Note: As always, we strongly recommend running a Full Database Backup from the Settings page before migrating to a new major Beta version!)

WABS - v1.0.0-beta.5

Full Changelog: https://github.com/wizwin/WABS/commits/v1.0.0-beta.5

WABS v1.0.0-beta.5 - Massive Performance & AI Optimizations

Welcome to WABS v1.0.0-beta.5! This release focuses heavily on taking the app's performance to the next level. We've completely vectorized the offline AI engine, bypassed heavy ORM overheads for lightning-fast database writes, and integrated SQLite's JSON1 extension to unlock incredibly fast, native metadata searches.

Here is everything new, improved, and fixed in this update:

🚀 Major New Features

• Combined AI Scanner: Added a new "Combined Scan" mode to optimize performance by scanning for files, faces, and objects simultaneously in a single pass.
• Advanced JSON Search: Added high-speed JSON prefix searches (camera:, resolution:, fps:, artist:, album:, genre:, and the catch-all meta:) utilizing SQLite's JSON1 extension for native metadata querying.
• Portable Data Export: Added JSON Export/Import tools for Known People and Object/Custom Tags. This includes a Smart Path Fallback Matcher so your tags and AI profiles survive drive letter changes and migrations!
• People Timeline: Added a chronological Timeline View and "Full Archive Timeline" jump integration directly into the People Photos page.
• Document Thumbnails: Added rich thumbnail generation support for .doc and .docx files.
• Smart Search UI & People Management: Improved the user interface for Smart Search, renaming people, and resolving person mismatches.

✨ UI/UX Enhancements

• Data Management Redesign: Completely redesigned the Data Management section in Settings with clean, descriptive UI cards.
• Action Safety Locks: Added dynamic UI safety locks to maintenance tasks (like clearing cache, backups, and exports) to prevent data corruption while background scanners are running.

🛠 Build & CI

• Raspberry Pi Build: Added a dedicated Raspberry Pi build target in GitHub Actions.
• Build Selection: Added UI to pick the specific OS build target when running workflows manually.

⚡ Massive Performance Improvements

• Vectorized AI Engine: Completely replaced slow Python math loops with optimized numpy vector and matrix multiplications for face clustering and similarity searches, dropping computation times from minutes to milliseconds.
• Database Speedups: Bypassed heavy SQLAlchemy ORM instantiation overhead in the indexer, search, and duplicate hasher routines. Integrated bulk_update_mappings and raised batch commit thresholds from 50 to 500 for lightning-fast SQLite writes.
• FTS5 Search & SQLite Tuning: Enabled WAL mode for improved concurrency, heavily optimized FTS5 trigger scoping to slash unnecessary disk I/O, and added Porter stemming and prefix tokenization for highly accurate partial filename matches.
• JSON1 Hasher Optimization: Upgraded the background lazy hasher to use SQLite's native json_extract filtering, completely preventing already-verified duplicate files from being unnecessarily loaded into memory.
• OOM Memory Optimizations: Resolved severe backend Out-Of-Memory crashes and SQLite lock contentions when scanning massive archives (>90,000 files) by implementing batched .yield_per() queries and ID-level tracking.

🐞 Bug Fixes

• Database Cleanup & Optimization: Added a dedicated routine in Settings to automatically scan for missing files, remove dead links, purge orphaned AI profiles, and vacuum the SQLite databases to reclaim disk space.
• CPU Drain Fix: Fixed a major bug where stopping the standalone face scan would fail to terminate the background loop, causing indefinite high CPU usage.
• Connection Stability: Fixed backend HTTP connection drops ([WinError 10054]) during rapid frontend scrolling using AbortController network cancellation.
• Database Limits: Fixed fatal SQLite OperationalError crashes during bulk tagging by chunking arrays to circumvent the hard 999 IN(...) variable limit.
• Pagination & Navigation: Added robust pagination to the People Photos API to prevent browser freezing on profiles with thousands of matched faces. Fixed UI bugs when navigating away from the Duplicates page.
• UI Polish: Fixed issues with re-applying batch processing, UI scaling when resizing the details pane, and missing selection highlights.

---

📥 Installation Instructions

1. Download the standalone executable for your operating system below (WABS-Windows.exe, WABS-Linux, or WABS-RaspberryPi).
2. Run the executable anywhere on your computer (no installation required!).
3. Open your browser to http://127.0.0.1:8000
   (Note: If updating from an older beta, it is recommended to run the new Database Cleanup routine in Settings!)

WABS - v1.0.0-beta.4

WABS v1.0.0-beta.4 🚀

We are excited to announce WABS v1.0.0-beta.4! This update brings powerful new boolean search capabilities, major improvements to Face Recognition workflows (including profile merging), new database export tools, and a refreshed Settings interface.

🚀 Major New Features
Advanced Search Operators: Added support for explicit boolean search operators + (Require/Match All) and - (Exclude) to allow fine-grained filtering (e.g., +object:car -tag:blur).
Similar Face Detection: Introduced the ability to easily find and merge similar unknown faces directly into an existing person's profile.
Face Profile Merging: Added the ability to manually merge multiple face profiles into one for improved face detection organization.
Data Export & Backup: Added a new "Export / Backup Data" feature in Settings to safely create a portable copy of your databases and configuration without needing to stop the application.
Full Archive Timeline: Added an option to display the complete chronological timeline of your entire archive directly in the Explorer and Search views.
Logging Support: Add logs for critical failures to help debug issues. This can be enabled in Settings UI.

✨ UI/UX Enhancements
Settings UI Reorganization: Completely overhauled the Settings page into a clean, tabbed interface (General, UI Preferences, AI & Vision, Backups, Smart Searches) for much easier navigation.

🐞 Bug Fixes & Performance
Dashboard Statistics: Corrected the "Known People" and "Unknown People" counts on the dashboard to accurately reflect unique individuals instead of total face appearances.

📦 Installation / Updating
If you are updating from a previous beta, simply download the new executable for your OS below and replace your old version. Your archive.db, ai_metadata.db, and config.yaml files are fully forward-compatible!

Version of WABS - v1.0.0-beta.3

Release v1.0.0-beta.3

Full Changelog: https://github.com/wizwin/WABS/commits/v1.0.0-beta.3

🚀 Major New Features

• People (Face Recognition): An entirely new feature utilizing local ML/AI models to detect faces, automatically group them by person, and allow you to browse all photos of a specific individual.
• Tags (Object Classification): An entirely new feature utilizing local ML/AI models to automatically classify objects and scenes in photos, enabling powerful keyword searches.
• Smart Searches: You can now save your most-used complex search queries (e.g., type:video length:>1h) as one-click shortcuts. These are configurable in Settings and appear in the Search view for quick access.
• Advanced Search Operators: The search bar is now significantly more powerful, supporting relational operators (size:>1GB, length:<30s) and date ranges (date:2020-2022).
• Global Tag Management: The new dedicated "Tags" page now includes tools to manage your metadata globally. You can delete a specific tag from every file in the archive or clear all AI-generated object: tags at once.
• AI Detection Sensitivity: New dropdowns in the Settings page allow you to fine-tune the accuracy of the AI models for Face Detection, Face Clustering, and Object/Scene Classification independently.

✨ UI/UX Enhancements

• Dashboard & Settings Redesign: Updated the Dashboard with new feature tiles and modernized the Settings UI with a clean, card-based layout.
• Dedicated Tags Page: The "Detected Objects & Scenes" section has been migrated from the dashboard to its own dedicated, searchable, and paginated page, dramatically improving dashboard load times for large archives.
• People Page Pagination: The "Named Persons" and "Unknown Persons" sections are now paginated, preventing browser freezes when viewing thousands of profiles. Pagination controls are available at both the top and bottom of each section.
• "Locate in Explorer" Feature: A new button in the person photos view allows you to instantly find a specific photo within the main Explorer timeline, showing its context with other files from the same day.
• Real-time Progress Indicators: All background tasks (Hashing, Face Scanning, Object Scanning) now show detailed progress bars and display the name of the file currently being processed.
• Intelligent Resuming: The Object & Scene scanner now intelligently resumes where it left off, skipping previously tagged photos.
• Manual Tagging Auto-Complete: The "Manage Tags" input now provides auto-complete suggestions based on your existing tags.
• Quality of Life: Renaming a person now automatically selects the full name for faster editing.

🐞 Bug Fixes & Performance

• Major Performance Boost: Drastically improved application startup time and background polling responsiveness by removing a heavy, unpaginated network request from the main dashboard loop.
• UI Stability & Responsiveness: Fixed unresponsive "Stop" buttons, blank Settings/Tags pages, and browser freezes on the People page.
• Robust Polling & Caching: Implemented exponential backoff for API polling and fixed browser caching issues that prevented real-time dashboard updates.
• Search Engine Fixes: Corrected the date: search filter and improved the size: filter logic.
• Duplicate Sorting: Fixed bugs in sorting updated duplicate files.
• Configuration Resilience: The backend now generates a complete default config.yaml on first run and gracefully handles legacy database file paths to prevent startup errors.