Skip to content

Implement Linux desktop capture parity with process routing and exclusions#19

Merged
Brooooooklyn merged 14 commits into
Recappi:mainfrom
pengx17:codex/linux-support-test-coverage
Mar 31, 2026
Merged

Implement Linux desktop capture parity with process routing and exclusions#19
Brooooooklyn merged 14 commits into
Recappi:mainfrom
pengx17:codex/linux-support-test-coverage

Conversation

@pengx17

@pengx17 pengx17 commented Mar 22, 2026

Copy link
Copy Markdown
Contributor

Context

  • Extend @recappi/sdk from Linux decoder support into a real Linux desktop capture surface that matches the existing macOS / Windows API shape.
  • Keep the Linux implementation inside the native Rust/N-API layer so the generated package entrypoints remain the single source of truth.
  • Close the main review gaps around process routing, exclusion handling, startup rollback, and Linux capability reporting.

Collaboration Process

sequenceDiagram
    participant U as User
    participant C as Codex
    participant R as Recappi SDK
    participant D as Docker Linux Env
    participant CI as GitHub Actions

    U->>C: Push Linux support beyond decoder-only behavior
    C->>R: Move Linux desktop capture into the native Rust/N-API surface
    U->>C: Reject the hand-maintained JS wrapper approach
    C->>R: Restore generated package entrypoints and keep Linux logic in Rust
    U->>C: Push for a real Linux implementation instead of a stub
    C->>R: Add dedicated-sink Pulse routing for process capture and exclusion-aware global capture
    U->>C: Review the PR again from scratch
    C->>R: Tighten failure rollback, capability detection, and CI coverage based on the review findings
    C->>D: Re-run local and Docker Linux verification
    C->>CI: Push the updated implementation and trim duplicated bindings jobs
Loading

Design Discussion

  • Option A: keep Linux on a decoder-only or global-fallback implementation.
    • Pros: lowest implementation risk.
    • Cons: does not satisfy the existing ShareableContent API contract and leaves Linux semantics misleading.
  • Option B: add a Linux-specific launch helper and require callers to opt into a dedicated Pulse sink before starting the target process.
    • Pros: reliable when the caller controls process launch.
    • Cons: does not fit Recappi's existing tapAudio(processId) API, which attaches to an already-running process.
  • Option C: use Pulse sink rerouting inside the existing Linux ShareableContent implementation.
    • Pros: preserves the public API, supports already-running processes, and gives Linux a real best-effort per-process capture path.
    • Cons: still depends on Pulse metadata and movable sink-inputs at runtime.
  • Chosen approach: Option C.
    • Rationale: it is the best fit for the current Recappi API while staying honest about Linux being best-effort rather than macOS-equivalent under every runtime.

Final Approach

  • Keep Linux desktop capture in src/linux/mod.rs.
  • Implement Linux tapAudio(processId) with dedicated Pulse routing:
    • create a unique null sink per capture session
    • move matching sink-inputs for the target PID into that sink
    • loop the capture sink back to the target stream's original sink instead of always using the default sink
    • record the dedicated sink monitor through ffmpeg
    • restore moved inputs and unload temporary modules on stop
  • Implement Linux tapGlobalAudio(excludedProcesses) with exclusion-aware routing:
    • resolve the active monitor sink
    • create a dedicated capture sink for the monitored output
    • move only non-excluded sink-inputs into that sink
    • record the capture sink monitor plus optional real microphone input
  • Harden failure handling:
    • roll back partially moved sink-inputs if startup fails during Pulse rerouting
    • tear down partially created loopback routes instead of leaving Pulse state behind after a failed start
  • Harden capability reporting:
    • keep tapGlobalAudio gated by generic monitor-capture readiness
    • gate Linux tapAudio on readable sink-input state plus detectable module-null-sink / module-loopback support without mutating the live Pulse graph
  • Harden CI and Linux iteration support:
    • keep Docker-based Linux verification from macOS
    • run Linux build and bindings coverage in CI
    • remove duplicated node@22 bindings jobs from the matrix
flowchart LR
    A[ShareableContent.tapAudio(processId)] --> B[Create dedicated capture sink]
    B --> C[Move matching sink-inputs into capture sink]
    C --> D[Loop capture sink back to original source sink]
    D --> E[Capture monitor with ffmpeg]
    F[ShareableContent.tapGlobalAudio(excludedProcesses)] --> G[Create exclusion-aware capture sink]
    G --> H[Move only non-excluded sink-inputs]
    H --> I[Capture sink monitor plus optional mic]
    E --> J[Float32 samples via N-API callback]
    I --> J
Loading

Verification

  • cargo fmt --all
  • corepack yarn lint
  • cargo test
  • corepack yarn test
  • bash ./scripts/test-linux-docker.sh
  • <!-- Please add any manual verification performed outside this Codex session -->

Known Limitations / Follow-up

  • Published Linux artifacts currently target x86_64-unknown-linux-gnu only.
  • Linux capture still depends on a PulseAudio-compatible userspace plus pactl and ffmpeg being available at runtime.
  • Linux tapAudio(processId) and tapGlobalAudio(excludedProcesses) remain best-effort because they depend on movable Pulse sink-inputs and application.process.id metadata being exposed by the target streams.
  • Linux application list and microphone state notifications remain polling-based rather than native event streams.

Copilot AI review requested due to automatic review settings March 22, 2026 12:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR formalizes Linux as a supported build/test target for @recappi/sdk while clarifying that Linux support currently focuses on the platform-agnostic audio decoder path (not ShareableContent capture backends).

Changes:

  • Add an explicit Linux Rust module to the crate graph and wire Linux into the library’s OS-gated module exports.
  • Expand CI to run cargo test on Linux/macOS/Windows and build/test Linux native bindings (x86_64-unknown-linux-gnu).
  • Add Rust unit tests covering decoder mono-mixing and resampling behavior, and update docs/tests to reflect platform support boundaries.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/linux/mod.rs Introduces an explicit Linux module with documentation about current Linux support scope.
src/lib.rs Adds Linux module gating and re-exports for target_os = "linux".
src/audio_decoder.rs Adds cross-platform Rust unit tests for stereo→mono mixing and resampling behavior.
README.md Documents current platform support matrix and clarifies Linux is decoder-only for now.
package.json Adds x86_64-unknown-linux-gnu to napi.targets for Linux binding builds.
.github/workflows/CI.yml Adds cargo test matrix and Linux build/test coverage for native bindings.
__test__/index.spec.ts Updates AVA wording to reflect “decoder-only platforms” vs platforms with capture backends.

Comment thread src/linux/mod.rs Outdated
Comment thread src/lib.rs
Comment thread .github/workflows/CI.yml Outdated
@pengx17 pengx17 marked this pull request as draft March 22, 2026 13:00
@pengx17 pengx17 changed the title Add Linux target and decoder test coverage Add Linux shareable capture backend and Docker test workflow Mar 22, 2026
@pengx17 pengx17 changed the title Add Linux shareable capture backend and Docker test workflow Unify Linux desktop capture API with a Docker-verified PulseAudio backend Mar 22, 2026
@pengx17 pengx17 marked this pull request as ready for review March 22, 2026 13:27
@pengx17 pengx17 requested a review from Brooooooklyn March 22, 2026 13:29
@pengx17 pengx17 changed the title Unify Linux desktop capture API with a Docker-verified PulseAudio backend Unify Linux x64 desktop capture API with a Docker-verified PulseAudio backend Mar 23, 2026
@pengx17 pengx17 changed the title Unify Linux x64 desktop capture API with a Docker-verified PulseAudio backend Implement Linux x64 desktop capture in the native N-API surface Mar 23, 2026
@pengx17 pengx17 changed the title Implement Linux x64 desktop capture in the native N-API surface Implement Linux desktop capture parity with process-isolated tapAudio Mar 23, 2026
@pengx17 pengx17 changed the title Implement Linux desktop capture parity with process-isolated tapAudio Implement Linux desktop capture parity with process routing and exclusions Mar 23, 2026
@Brooooooklyn

Copy link
Copy Markdown
Contributor

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 6 comments.

Comment thread src/linux/mod.rs Outdated
Comment thread src/linux/mod.rs Outdated
Comment thread src/linux/mod.rs Outdated
Comment thread src/linux/mod.rs Outdated
Comment thread package.json Outdated
Comment thread .github/workflows/CI.yml
@Brooooooklyn

Copy link
Copy Markdown
Contributor

@pengx17 can https://github.com/Brooooooklyn/webcodecs-node replace the ffmpeg in tests?

@pengx17

pengx17 commented Mar 31, 2026

Copy link
Copy Markdown
Contributor Author

@Brooooooklyn I switched the Linux test fixture away from ffmpeg for helper-side tone generation. The fixture now generates PCM/WAV in Node and uses PulseAudio utilities / a small Node pipe writer for the synthetic sources.\n\nI kept ffmpeg in CI because the runtime under test still shells out to ffmpeg for the real Linux capture path, and I wanted the e2e coverage to stay honest.\n\nVerified locally with yarn test and bash ./scripts/test-linux-docker.sh.

@Brooooooklyn Brooooooklyn merged commit 4491184 into Recappi:main Mar 31, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants