Skip to content

RapidReady v0.5.0

Choose a tag to compare

@on370 on370 released this 14 Oct 16:00
· 39 commits to main since this release

RapidReady v0.5.0 — Persistent Chunk Cache, High-Res RAW Engine & Smooth NAS Workflows

RapidReady 0.5.0 is a major milestone release introducing an offline-first Persistent Local Disk Cache inspired by Google Picasa and Adobe Lightroom (packaged binary chunk container architecture indexed by SQLite for 60 FPS grid browsing across 100,000+ images), network-optimized in-process header extraction for NAS/SMB shares, high-resolution Canon CR2 preview extraction with differentiated in-memory downscaling, zero-latency Loupe View viewport debouncing, and high-throughput Phase 2 I/O smoothing for fast NVMe SSDs.


🚀 Highlights & Key Changes

🗄️ Persistent Local Disk Cache (Picasa & Lightroom Model)

  • Binary Chunk Container Architecture (chunk_XXXX.bin): Thumbnails are sequentially packed into 64 MB rolling binary blobs on local NVMe storage (~/Library/Caches/de.rapidready.app/ on macOS, %LOCALAPPDATA%\RapidReady\Cache on Windows). Completely eliminates inode exhaustion, APFS/NTFS filesystem fragmentation, and random seek penalties across 100,000+ photos.
  • Single-Seek $O(1)$ Byte-Range Reads: Retrieving any cached thumbnail requires exactly one lseek + read system call, delivering sub-millisecond thumbnail loading.
  • SQLite Catalog Index (archive_catalog.sqlite): Maps file paths, mtimes, sizes, orientations, and scales to chunk byte offsets in high-concurrency WAL mode.
  • 100% Offline-First Browsing: Once an archive is indexed, photo grid and filmstrip are served 100% locally from NVMe at 60 FPS without waking sleeping NAS disks.
  • Automatic Incremental Delta Sync: Background scans detect added, modified, or deleted files on archive reconnection and update entries incrementally without full rescans.

📷 Canon CR2 High-Res Preview & Differentiated In-Memory Downscaling

  • Hardened TIFF Strip Parser: Validates Compression == 6 || 7 tags, safely discarding uncompressed raw Bayer sensor data (e.g. 860 KB IFD2 slice) and giant sensor streams (>8 MB).
  • Extended 8 MB Preview Ceiling: Successfully captures Canon CR2's full-sensor IFD0 preview (5472×3648, 2.5–5.5 MB) instead of falling back to the 160×120 micro-thumbnail.
  • Differentiated In-Memory Downscaling:
    • Medium camera previews $\le 1.5,\text{MB}$ (Sony ARW 504 KB, Canon CR3 400 KB, Ricoh DNG 54 KB) are direct-stored with 0 ms CPU overhead.
    • Large full-sensor previews $> 1.5,\text{MB}$ (Canon CR2) are downscaled in-memory via Apple ImageIO in ~20 ms to 720px thumbnails (~80–120 KB), keeping the local chunk cache compact (1 GB instead of 35 GB for 10,000 images) while keeping the grid razor-sharp.
    • Full sensor resolution is strictly preserved for Loupe View and 100% Zoom (scale >= 100).

⚡ Loupe View Viewport Debouncing & Network Cancellation

  • 100 ms Viewport Debouncing: Rapid arrow navigation (<-- / -->) or fast double-clicking immediately jumps the filmstrip selection and bottom counter (0 ms feedback) while debouncing the heavy central viewport, skipping intermediate frames over slow NAS shares.
  • Proactive Network Abort: Unmounting images instantly cuts in-flight network connections (img.onload = null; img.src = ''; canceled = true;), preventing queue congestion on network mounts.

🏎️ Phase 2 Throughput Smoothing & SSD Responsiveness

  • Deferred Disk Flushes: Removed synchronous fsync on every individual 50 KB thumbnail write in chunk_cache.rs, deferring flushes to 64 MB chunk boundary rollover and leveraging the OS page cache for butter-smooth NVMe write performance.
  • Paced IPC Event Delivery: Replaced unthrottled burst reporting (done % 10 == 0) with a paced 150 ms (~6.6 Hz) reporting interval, completely eliminating React main-thread starvation and UI freezing on fast NVMe SSDs.
  • Cooperative CPU Scheduling: Worker threads yield execution (std::thread::yield_now()) to ensure interactive UI responsiveness during heavy background indexing.
  • Accurate Live Transfer Rates: Fixed speed calculation by transmitting true accumulated byte counters (processed_bytes), accurately displaying real MB/s throughput in the scan progress card.

🎨 UI Polish & Contrast Accessibility

  • Format Badges in Grid View: Refined background translucency and typography for format pills (JPG, RAW, VID), ensuring crisp, high-contrast readability against both light and dark photo backgrounds.

⚠️ Known Issues

  • Fujifilm RAW (.raf) Grid Thumbnails: In Grid View, .raf files currently do not render micro-thumbnails. Full-resolution preview and metadata inspection in Loupe View and the Inspector work correctly. A dedicated RAF header parser is scheduled for the upcoming point release (tracked in ROADMAP.md).

📦 Compatibility & System Requirements

  • macOS: Universal Binary (Apple Silicon M1/M2/M3/M4 & Intel x64), macOS 11+
  • Windows: 64-bit (x64), Windows 10/11