Skip to content

FuseClip 0.71.0

Choose a tag to compare

@ammarrik ammarrik released this 26 Aug 15:18
6c7cd28

FuseClip v0.71.0

FuseClip v0.71.0 brings intelligent active speaker face tracking with mouth-motion detection, customizable text stroke and outline styling for captions, hardware-aware transcription model recommendations, a redesigned onboarding experience, native disk-backed project persistence, and major performance optimizations across the local Whisper engine and video pipeline.

What's New

Active Speaker Detection & Stable Face Tracking

  • Mouth Motion & Lip Tracking: Integrated MediaPipe Face Landmarker with a dedicated MouthMotionTracker that analyzes jaw openness and speech dynamics in real time, automatically focusing the camera crop on whoever is actively speaking in multi-person scenes.
  • Temporal Face Locking (PrimaryFaceTracker): Eliminated camera jitter and sudden jumping between faces using IoU tracking, candidate hysteresis, and switch margins.
  • Adaptive Frame Sampling: Dynamically scales frame sampling rates for long videos (up to 1+ hour files) to keep face tracking fast and responsive on any CPU or GPU.

Caption Stroke & Outline Customization

  • Custom Stroke Controls: Added configurable stroke width (0–30%) and stroke color pickers for single-word / karaoke highlights, preset styles, and sticker captions.
  • High-Contrast Readability: Ensures burned and previewed captions remain crisp and legible against bright, high-contrast, or fast-moving video backgrounds.

Smart Model Recommendations & Guided Tooltips

  • Hardware-Aware Suggestions: Evaluates system specifications (Apple Silicon Metal GPU, NVIDIA CUDA, CPU cores, RAM) to recommend optimal Whisper models (such as Large v3 Turbo) for maximum transcription accuracy without slowdowns.
  • Language-Specific Deepgram Recommendations: Contextual tooltips detect selected languages and recommend optimal cloud models (e.g. Deepgram Nova-3 for multilingual code-switching, Urdu, Arabic, and Hinglish) with one-click model switching.
  • Forced Language Selection: Requires explicit language selection in the Transcribe panel to ensure accurate transcription and transliteration settings before generation begins.

Redesigned Onboarding Experience

  • Interactive Hardware Check: Updated Get Started screen with automated system diagnostics and clear side-by-side guidance between local Whisper and Deepgram cloud transcription.
  • Inline API Key Validation: Added real-time Deepgram credential testing directly within the onboarding setup.

Fixes & Improvements

  • Modular Whisper.cpp Engine: Refactored the local transcription service into dedicated, maintainable modules (runner, token-parser, downloader, binary-resolver, backend).
  • Precision DTW Timestamps: Implemented Dynamic Time Warping (--dtw) token parsing with fallback heuristics to deliver precise, non-overlapping word-level timing and natural punctuation placement without artificial pauses.
  • Whisper Performance Optimization: Resolved sluggish local model inference speeds by tuning execution parameters, thread allocation, and library loading paths.
  • Native Disk-Backed Project Store: Migrated project persistence from localStorage to atomic file writes on disk (<userData>/projects/) via Electron IPC with automatic migration for existing projects.
  • Extended Audio Extraction Timeout: Increased FFmpeg audio extraction timeout from 30 seconds to 10 minutes with an expanded buffer to handle long, high-resolution source media without timeouts.
  • Pipeline Cancellation State Cleanup: Fixed an issue where cancelling a generation run and restarting with fewer clips would re-process stale clips from the previous canceled job.
  • Interactive Generation Pipeline Progress: Added a dedicated preparation step to the generation ladder to clearly indicate disk read and model initialization progress.
  • Tooltip Persistence: Fixed an issue where recommendation tooltips would not reappear after switching between local models and cloud providers.