Skip to content

fix(export): recover video background fallback and smooth preview audio#452

Merged
meiiie merged 3 commits into
mainfrom
fix/export-background-preview-audio
May 8, 2026
Merged

fix(export): recover video background fallback and smooth preview audio#452
meiiie merged 3 commits into
mainfrom
fix/export-background-preview-audio

Conversation

@meiiie
Copy link
Copy Markdown
Collaborator

@meiiie meiiie commented May 8, 2026

Summary

  • fall back from decoder-backed video wallpaper/background frames to media-element sync when packet streaming fails mid-export
  • add regression coverage for the Lightning and legacy frame renderers so custom/video background exports recover instead of failing
  • reduce aggressive companion-audio playback-rate correction in the editor preview to avoid audible flutter/crackle while preserving seek-based sync correction
  • clean up two existing hook dependency warnings touched by the VideoEditor formatter pass

Validation

  • npm test -- src/lib/mediaTiming.test.ts src/lib/exporter/sourceAudioFallback.test.ts src/lib/exporter/modernFrameRenderer.test.ts src/lib/exporter/frameRenderer.test.ts
  • npx tsc --noEmit
  • npx biome check src/components/video-editor/VideoEditor.tsx src/lib/exporter/frameRenderer.ts src/lib/exporter/frameRenderer.test.ts src/lib/exporter/modernFrameRenderer.ts src/lib/exporter/modernFrameRenderer.test.ts
  • git diff --check

Summary by CodeRabbit

  • Bug Fixes

    • Video wallpaper/export now reliably falls back to media-element playback when decoder-backed streaming fails
    • Improved preview audio/video synchronization with parameterized drift/correction tolerances for more accurate playback
    • More robust export progress reporting and stability during background/wallpaper exports
  • Tests

    • Added tests covering decoder failure fallback and media-element fallback behavior to prevent regressions

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4596b53c-804f-4b0a-8e70-e5118cc891c0

📥 Commits

Reviewing files that changed from the base of the PR and between 7aefde7 and 31cb818.

📒 Files selected for processing (3)
  • src/components/video-editor/VideoEditor.tsx
  • src/lib/exporter/frameRenderer.ts
  • src/lib/exporter/modernFrameRenderer.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • src/lib/exporter/frameRenderer.ts
  • src/lib/exporter/modernFrameRenderer.ts
  • src/components/video-editor/VideoEditor.tsx

📝 Walkthrough

Walkthrough

Refines VideoEditor preview audio-sync parameters and smoke-export backend parsing; implements decoder→media-element fallback with readiness timeout in legacy and modern frame renderers, plus tests that assert decoder teardown and media-element fallback initialization.

Changes

Video Editor Audio Sync & Export Config

Layer / File(s) Summary
Audio Sync Parameter Constants
src/components/video-editor/VideoEditor.tsx
New constants govern source-audio preview seek-drift thresholds (playing vs. paused) and playback-rate synchronization tolerance/correction window/max-adjustment.
Audio Sync Usage in Preview
src/components/video-editor/VideoEditor.tsx
Preview audio sync computes drift thresholds conditionally based on play state and passes explicit tolerance/correction parameters to getMediaSyncPlaybackRate.
Smoke Export Backend Preference
src/components/video-editor/VideoEditor.tsx
Smoke-export MP4 backend preference now branches on pipeline model and useNativeExport setting to select breeze vs. webcodecs backend.
Project Load Session Initialization
src/components/video-editor/VideoEditor.tsx
applyLoadedProject explicitly branches: webcam source calls setCurrentRecordingSession; non-webcam calls setCurrentVideoPath before applying session presentation; related state setters grouped.
Code Refactoring & Dependencies
src/components/video-editor/VideoEditor.tsx
Condensed devOpenRecordingConfig useMemo, reformatted buildPersistedEditorState Partial type line structure, and adjusted callback dependency arrays.
Export Progress UI Labels
src/components/video-editor/VideoEditor.tsx
Export progress percent-phase translation call nesting restructured without changing displayed strings.

Frame Renderer Wallpaper Export Fallback

Layer / File(s) Summary
Legacy FrameRenderer Fallback Implementation
src/lib/exporter/frameRenderer.ts
setupBackground delegates wallpaper loading to loadBackgroundMediaElementSource; syncBackgroundFrame wraps decoder getFrameAtTime in try/catch to fallback to media-element and retry; adds helpers to teardown decoder state and prime a muted looping <video>.
ModernFrameRenderer Fallback Implementation
src/lib/exporter/modernFrameRenderer.ts
setupBackground and syncBackgroundFrame refactored to use shared fallback helpers; BACKGROUND_MEDIA_ELEMENT_READY_TIMEOUT_MS added to bound media-element readiness waiting.
Wallpaper Export Fallback Tests
src/lib/exporter/frameRenderer.test.ts, src/lib/exporter/modernFrameRenderer.test.ts
Tests hoist extra ForwardFrameSource mocks (cancel/destroy/getFrameAtTime), define synchronous requestAnimationFrame behavior in modern tests, clear mocks before key tests, and add tests that force decoder failure to assert cancellation/destroy and media-element fallback initialization.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

Checked

Poem

🐰 I hopped through frames and syncs today,
When decoders tripped, I found a way.
A muted loop, a patient wait,
The video element saves the state.
Hooray for fallbacks—hooray!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the two main changes: video background fallback recovery and preview audio smoothing, directly matching the core objectives.
Description check ✅ Passed The description includes a clear summary, motivation, validation steps, and comprehensive details; however, it does not follow the provided template structure with explicit sections.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/export-background-preview-audio

Comment @coderabbitai help to get the list of available commands and usage tips.

@meiiie
Copy link
Copy Markdown
Collaborator Author

meiiie commented May 8, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/components/video-editor/VideoEditor.tsx (1)

4624-4639: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Smoke backend pinning is bypassed when smokeUseNativeExport=1.

In the modern path, useExperimentalNativeExport flips this ternary to "auto" before the smoke-specific backendPreference override/defaults are consulted. That means smokeBackendPreference is ignored, and the "breeze" fallback here is effectively unreachable, so smoke runs cannot reliably force Breeze vs WebCodecs coverage.

Suggested fix
 					const backendPreference =
 						pipelineModel === "legacy"
 							? "webcodecs"
-							: useExperimentalNativeExport
-								? "auto"
-								: smokeExportConfig.enabled
-									? (smokeExportConfig.backendPreference ??
-										(smokeExportConfig.useNativeExport
-											? "breeze"
-											: "webcodecs"))
-									: (settings.backendPreference ?? exportBackendPreference);
+							: smokeExportConfig.enabled
+								? (smokeExportConfig.backendPreference ??
+									(smokeExportConfig.useNativeExport
+										? "breeze"
+										: "webcodecs"))
+								: useExperimentalNativeExport
+									? "auto"
+									: (settings.backendPreference ?? exportBackendPreference);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/components/video-editor/VideoEditor.tsx` around lines 4624 - 4639, The
ternary logic for backendPreference lets useExperimentalNativeExport force
"auto" before honoring smokeExportConfig.backendPreference, so smoke runs can't
pin Breeze/WebCodecs. Fix by treating smokeExportConfig.enabled as the
highest-priority branch for modern pipeline: when pipelineModel === "modern" and
smokeExportConfig.enabled, compute backendPreference from
smokeExportConfig.backendPreference ?? (smokeExportConfig.useNativeExport ?
"breeze" : "webcodecs"); otherwise (smoke disabled) fall back to the existing
useExperimentalNativeExport ? "auto" : (settings.backendPreference ??
exportBackendPreference). Update the backendPreference expression (referencing
pipelineModel, useExperimentalNativeExport, and smokeExportConfig) accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/components/video-editor/VideoEditor.tsx`:
- Around line 2143-2150: The else (plain-video) branch currently calls
setCurrentVideoPath then still reads getCurrentRecordingSession and applies it,
which can leak previous webcam session UI state; modify the else branch handling
around setCurrentVideoPath so that instead of calling
getCurrentRecordingSession/applySessionPresentation afterwards you explicitly
call applySessionPresentation(null) (i.e., reset session presentation) and only
call getCurrentRecordingSession()/applySessionPresentation(session) when
handling the webcam-backed branch; update code paths around setCurrentVideoPath,
getCurrentRecordingSession, and applySessionPresentation accordingly.

In `@src/lib/exporter/frameRenderer.ts`:
- Around line 906-953: The ready Promise created after calling
resolveMediaElementSource(videoSrc) can hang if loadeddata/canplay/error never
fire; modify the logic in frameRenderer.ts (around resolveMediaElementSource,
cleanupBackgroundSource, and the ready Promise that creates the video element)
to add a bounded timeout (e.g., configurable ms) that rejects/resolves false
after the timeout, ensure the timeout is cleared on success/error, always call
backgroundSource.revoke when the function returns false or on timeout, and
guarantee event listeners on the video element are removed in the timeout
handler and in the existing cleanup function so there is no leaked media source
or dangling listeners. Ensure the same teardown runs for the existing error path
and the new timeout path.

In `@src/lib/exporter/modernFrameRenderer.ts`:
- Around line 2107-2159: Guard the background video readiness with a timeout and
ensure the resolved source/video are revoked on all failure paths like the
webcam fallback: when awaiting resolveMediaElementSource(videoSrc) keep
this.cleanupBackgroundSource assigned, then in the Promise that waits for
"loadeddata"/"canplay"/"error" add a timeout (e.g., setTimeout ->
resolve(false)) and include that timer in the cleanup; on any failure branch
(error handler or timeout) call the revoke function
(this.cleanupBackgroundSource or backgroundSource.revoke) and stop/clear the
video (remove src, call load if needed) before resolving false; ensure the same
cleanup is performed if ready is false so no resolved source/video is left
alive; finally proceed to set this.backgroundVideoElement and call
ensureBackgroundSprite only when the Promise resolves true.

---

Outside diff comments:
In `@src/components/video-editor/VideoEditor.tsx`:
- Around line 4624-4639: The ternary logic for backendPreference lets
useExperimentalNativeExport force "auto" before honoring
smokeExportConfig.backendPreference, so smoke runs can't pin Breeze/WebCodecs.
Fix by treating smokeExportConfig.enabled as the highest-priority branch for
modern pipeline: when pipelineModel === "modern" and smokeExportConfig.enabled,
compute backendPreference from smokeExportConfig.backendPreference ??
(smokeExportConfig.useNativeExport ? "breeze" : "webcodecs"); otherwise (smoke
disabled) fall back to the existing useExperimentalNativeExport ? "auto" :
(settings.backendPreference ?? exportBackendPreference). Update the
backendPreference expression (referencing pipelineModel,
useExperimentalNativeExport, and smokeExportConfig) accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f44f6d4f-20e2-40dc-b9ed-eacad927af91

📥 Commits

Reviewing files that changed from the base of the PR and between d4dc0af and 7aefde7.

📒 Files selected for processing (5)
  • src/components/video-editor/VideoEditor.tsx
  • src/lib/exporter/frameRenderer.test.ts
  • src/lib/exporter/frameRenderer.ts
  • src/lib/exporter/modernFrameRenderer.test.ts
  • src/lib/exporter/modernFrameRenderer.ts

Comment thread src/components/video-editor/VideoEditor.tsx
Comment thread src/lib/exporter/frameRenderer.ts
Comment thread src/lib/exporter/modernFrameRenderer.ts
@meiiie
Copy link
Copy Markdown
Collaborator Author

meiiie commented May 8, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@meiiie meiiie marked this pull request as ready for review May 8, 2026 15:13
@meiiie meiiie merged commit 144698c into main May 8, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant