feat(desktop): Windows + Linux screen capture & recording (xcap) - #4
Merged
Conversation
…to JS) The in-tool region flow captured the frozen full-res screen, shipped the whole PNG to JS (10-30MB on a 5K display), base64'd it, decoded it, and cropped in a canvas — even for a tiny region. Now: capture_hold captures the frozen full frame once (while the main window is hidden) and keeps it in Rust; after selection, crop_held crops server-side with HiDPI scaling and returns only the small region as raw PNG bytes. release_held frees it on cancel. Memory is bounded to one held frame (cleared on next capture). physical_crop_rect (logical→physical, clamped) is a pure fn with 6 unit tests. Screenshot.tsx loads the crop via createImageBitmap — no base64, no full-res decode. Region button is Tauri-only so there's no browser path to preserve. cargo test: 19 passed; JS: 385 passed. Manual test: crop is pixel-accurate on HiDPI and correct on a secondary display. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013dqouzFP8vy9jaKhTDFj5H
Windows/Linux had no capture backend — every capture command returned "not yet implemented", so screen recording (which feeds on capture_screen_fast) produced zero frames and the audio pipeline had nothing to attach to. Adds xcap (0.9.4, wgc on Windows) as the Windows + Linux capture backend, wired into list_displays, capture_screen, capture_screen_fast, and capture_hold. macOS keeps its validated core-graphics path untouched. HeldCapture is unified on image::RgbaImage so crop_held is one platform-agnostic path; crop_rgba is a pure fn with unit tests. Verification: macOS builds + 21 unit tests pass locally. Windows/Linux can't be compiled on a macOS dev machine, so .github/workflows/desktop-check.yml compile-checks all three targets on CI (with the PipeWire/Wayland/dbus deps xcap needs on Linux). Runtime validation still needs real Windows/Linux hardware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013dqouzFP8vy9jaKhTDFj5H
xcap's Linux backend pulls libspa/spa-sys, which need PipeWire 1.0+ headers (spa_meta_region_is_valid, spa_video_info_raw.flags). ubuntu-22.04 ships an older PipeWire and failed to compile libspa. Bumped both the compile-check and release workflows to ubuntu-24.04 and added libpipewire-0.3-dev, libspa-0.2-dev, wayland/xcb, and libclang (spa-sys uses bindgen). Windows already compiled clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013dqouzFP8vy9jaKhTDFj5H
…r-frame) record_frames called capture_screen_fast per frame, which on Linux re-negotiates the xdg-desktop-portal screencast on every grab — far too slow for real-time recording (and it would re-prompt). macOS keeps its fast per-frame core-graphics loop; Windows/Linux now open ONE xcap VideoRecorder stream, consume frames from its channel, throttle to the target FPS, crop/encode each kept frame to JPEG, and push into the same buffer the encoder drains on stop. - record_frames is now macOS-only; record_frames_xcap handles Windows/Linux - start_recording spawns the right loop per platform - crop_rgba is pub(crate) so the recording loop can reuse the crop math - even-dimension enforcement so H.264/mp4 encoding accepts the frames macOS builds clean. Windows/Linux compilation verified via desktop-check CI. Runtime still needs real hardware (xcap Linux path needs the portal grant). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013dqouzFP8vy9jaKhTDFj5H
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the missing Windows + Linux screen-capture backend so recording (and the audio pipeline that rides on it) works beyond macOS. Previously every capture command returned "not yet implemented" on Windows/Linux, so recording captured zero frames.
Approach
xcap0.9.4 as the Windows (WGC/DXGI) + Linux (PipeWire portal) capture backend; macOS keeps its validated core-graphics path untouched.list_displays,capture_screen,capture_screen_fast(recording frames +capture_region), andcapture_hold.HeldCaptureunified onimage::RgbaImage→crop_heldis one platform-agnostic path.crop_rgba/physical_crop_rectare pure fns with unit tests.Verification
desktop-check.yml): ✓ macOS, ✓ Windows, ✓ Linux (ubuntu-24.04).release.ymlLinux build to ubuntu-24.04 with PipeWire/libspa/clang deps so real releases build.Stacking
Branched off Phase D (PR #3). Currently shows Phase D + these commits; once #3 merges to develop, this narrows to just the platform-capture commits.
🤖 Generated with Claude Code