Optimize editor performance for large videos

[richiemcilroy]

Implement virtualization and rendering optimizations to improve editor performance for long videos.

The editor was experiencing significant lag with videos 10 minutes or longer due to rendering all timeline elements and frames regardless of visibility. This PR introduces virtualization for timeline segments (clips and zoom) to only render what's currently in the viewport. Additionally, frame rendering is optimized with a reduced throttle rate, requestAnimationFrame batching, and debounced project updates. The video decoder cache size has also been increased to reduce expensive seeking operations during scrubbing. These changes ensure that the computational cost remains constant, providing a smooth experience irrespective of video length.

---

 

---

Note

Virtualizes timeline clip/zoom tracks, batches/throttles frame rendering, and adds parallel frame prefetching with a larger decoder cache to improve responsiveness on long videos.

• Editor (UI/Rendering)
  • Virtualizes timeline rendering: add visibleTimeRange and isSegmentVisible in Timeline/context.ts; render only visible segments in ClipTrack.tsx and ZoomTrack.tsx using Index and filtered indices.
  • Frame rendering optimizations in Editor.tsx: throttle (~30fps), debounce project-driven updates, requestAnimationFrame batching, and proper cleanup.
• Playback (Rust)
  • Add parallel frame prefetching and buffering in crates/editor/src/playback.rs with bounded in-flight decodes, frame skip capping, and coordination via channels.
  • Use prefetched DecodedSegmentFrames when available; fall back to on-demand decode.
  • Increase video decoder FRAME_CACHE_SIZE to 500 in crates/rendering/src/decoder/mod.rs.

^{Written by Cursor Bugbot for commit d654add. This will update automatically on new commits. Configure here.}

Summary by CodeRabbit

• Performance Improvements
  • Timeline editor now renders only visible segments, improving responsiveness with large projects
  • Enhanced frame rendering with optimized throttling for smoother playback experience
  • Audio frame prefetching improves playback performance and responsiveness
  • Increased frame cache capacity for faster frame processing

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[cursor]

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
_{Learn more about Cursor Agents}

[coderabbitai]

Caution

Review failed

The pull request is closed.

Walkthrough

The PR refactors frame rendering and timeline segment visibility to reduce redundant renders, adds async audio frame prefetching with parallel decoding, and increases the frame cache size. Changes span editor frame throttling, timeline segment filtering, and playback architecture improvements.

Changes

Cohort / File(s)	Summary
Lint Configuration Cargo.toml	Removed manual_is_multiple_of = "deny" clippy lint setting from workspace lints.
Frame Emission & Rendering Timing apps/desktop/src/routes/editor/Editor.tsx	Introduced RAF-based frame coalescing with throttled/debounced frame emission; added rafId, pendingFrameTime, and lastProjectUpdateTime state for coordinated render scheduling; enhanced cleanup handlers to cancel pending RAF and clear throttled/debounced handlers.
Timeline Visibility Filtering apps/desktop/src/routes/editor/Timeline/context.ts	Memoized markingResolution and introduced visibleTimeRange computed value with render padding; added isSegmentVisible helper; exported new context members for segment visibility checks.
Clip Track Rendering Optimization apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx	Replaced full segment iteration with visible-segment-only rendering using visibleSegmentIndices memo; refactored all segment access to use per-segment wrapper segment() binding; updated drag/split logic to operate on memoized segment data.
Zoom Track Rendering Optimization apps/desktop/src/routes/editor/Timeline/ZoomTrack.tsx	Replaced zoom segment For loop with Show/Index-based rendering of visible indices; added isSegmentVisible filtering; refactored mouse drag, selection, and resize logic to use per-segment accessors; added fallback for missing segment data.
Audio Playback Prefetching crates/editor/src/playback.rs	Introduced async frame prefetching task with PrefetchedFrame struct and bounded channel buffer; added parallel decoding using FuturesUnordered with configurable task limits; implemented frame-skipping logic with logging to handle timing misalignment; enhanced playback loop to consume prefetched cache with fallback to active decoding.
Decoder Cache Sizing crates/rendering/src/decoder/mod.rs	Increased FRAME_CACHE_SIZE constant from 100 to 500.

Sequence Diagram(s)

sequenceDiagram
    participant Playback as Playback Loop
    participant Prefetch as Prefetch Task
    participant Decode as Decoder
    participant Channel as Prefetch Channel
    participant Cache as Prefetch Cache

    Playback->>Prefetch: spawn prefetch task
    Prefetch->>Prefetch: compute next frames to prefetch
    Prefetch->>Decode: request parallel decode (up to PARALLEL_DECODE_TASKS)
    activate Decode
    Decode->>Decode: decode frame async
    Decode->>Channel: send PrefetchedFrame
    deactivate Decode
    Channel->>Cache: buffer frame (bounded by PREFETCH_BUFFER_SIZE)
    
    Playback->>Cache: check for prefetched frame
    alt Frame in cache
        Cache-->>Playback: return prefetched frame
    else Cache miss
        Playback->>Decode: request active decode
        Decode-->>Playback: return frame
    end
    
    Playback->>Playback: advance playhead, apply frame-skip logic (max 3)
    Playback->>Cache: trim buffer if catching up

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~50–70 minutes

• Editor.tsx: Dense RAF and frame timing coordination with multiple state variables; requires careful analysis of throttle/debounce interaction and cleanup logic.
• ClipTrack.tsx & ZoomTrack.tsx: Extensive per-segment refactoring affecting rendering, drag handlers, and selection logic across both files; verify segment visibility filtering is correct and no regressions in interaction paths.
• playback.rs: Introduces async concurrency patterns with prefetching task, parallel decoding, and frame-skip logic; verify channel/buffer bounds, error handling, and frame pacing correctness.
• Heterogeneous scope: Changes span editor UI timing, timeline optimization, and audio playback architecture, requiring context switching between domains.

Possibly related PRs

• Fix trimming behavior #1273: Modifies ClipTrack.tsx per-segment access and drag/trim logic; overlaps with this PR's segment memoization refactor.
• run clippy #1301: Touches Cargo.toml Clippy lint settings, specifically affecting the manual_is_multiple_of lint (this PR removes its deny).
• Refine auto-zoom segment generation and cursor stability #1106: Modifies ZoomTrack.tsx and related timeline visibility APIs; directly overlaps with this PR's zoom segment rendering changes.

Suggested labels

Desktop

Suggested reviewers

• Brendonovich

Poem

🐰 Frames now throttle and prefetch with care,
Timeline segments filter what's fair,
Audio hops through a parallel decode,
Rendering smoother down the fast road,
One hop, two hops—optimization's here! ✨

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch cursor/optimize-editor-performance-for-large-videos-claude-4.5-opus-high-thinking-fb23

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 87ffbc2 and d654add.

📒 Files selected for processing (7)

• Cargo.toml (0 hunks)
• apps/desktop/src/routes/editor/Editor.tsx (4 hunks)
• apps/desktop/src/routes/editor/Timeline/ClipTrack.tsx (15 hunks)
• apps/desktop/src/routes/editor/Timeline/ZoomTrack.tsx (4 hunks)
• apps/desktop/src/routes/editor/Timeline/context.ts (2 hunks)
• crates/editor/src/playback.rs (7 hunks)
• crates/rendering/src/decoder/mod.rs (1 hunks)

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}