fix(wgc): GPU DXGI encode path for Windows capture, with a fallback at every step - #305
fix(wgc): GPU DXGI encode path for Windows capture, with a fallback at every step#305EtienneLescot wants to merge 3 commits into
Conversation
The DXGI path is the right shape for issue #252: it removes the Map/Unmap readback the reporter's driver wedges inside. What it must not do is become a requirement, because it fixes one machine and every other one still has to record. So every step of it now falls back rather than returning: the encoding device, the NV12 video processor, the shared bridge texture, the sample allocator and the hardware sink writer each drop the whole pipeline and retry the exact chain a machine without a GPU path would have taken. Without that, `useDxgiInput` being the default made the software H.264 fallback unreachable for every recording, and a machine with no hardware encoder went from recording in software to not recording at all. `releaseDxgiPipeline()` puts device_/context_ back on the capture device, because the CPU path's staging texture has to live where the WGC frames do. The choice is no longer knowable from the outside, so callers ask `usesDxgiInput()` and the `encoder-selection` event reports `videoInput`. The bridge acquire was a 5s wait taken on the video-writer thread while it holds the frame lock -- the lock issue #252 is about, measured against an 8s watchdog step budget. It is now a few frame intervals, and a timeout skips the frame instead of ending the recording; the timestamp is stamped after the conversion so a skipped frame no longer stretches the timeline. Frames lost that way are counted and reported once at stop. Measured on a working machine, GPU path against CPU path: - 16.9 Mbps against 1.95 for the same desktop, because the D3D manager switches the sink writer onto a hardware MFT and those default to CBR, spending the full 18 Mbps budget on a static screen. Asking for VBR through ICodecAPI brings it to 2.2. MF_LOW_LATENCY was measured and made no difference, so it is gone. - Colour matches: raw luma 13/222.5/239 against 13/224.3/242, mean rendered RGB 245,240,245 against 246,242,246. The video processor is told full-range BGRA in, studio BT.709 out, and the media types carry the matching tags -- untagged, the driver default is BT.601 and a player reads 1080p as BT.709. - Stop latency 107ms, 0 contended frames over repeated runs, software fallback and preferSoftwareEncoder still land on the CPU path. Co-authored-by: Seb1900 <1712315938@qq.com>
📝 WalkthroughWalkthroughThe Windows recorder now supports DXGI/NV12 GPU encoder input with CPU fallback. It configures GPU conversion and hardware VBR, reports the selected input path, handles temporary bridge contention as dropped frames, and documents fallback and opt-out conditions. ChangesWindows DXGI encoder input
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant WGCSession
participant MFEncoder
participant VideoProcessor
participant SinkWriter
WGCSession->>MFEncoder: provide WGC texture
MFEncoder->>VideoProcessor: convert BGRA texture to NV12
VideoProcessor-->>MFEncoder: converted frame or bridge contention
MFEncoder->>SinkWriter: submit timestamped DXGI sample
SinkWriter-->>MFEncoder: encoding result
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
electron/native/wgc-capture/src/wgc_session.cpp (1)
66-111: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winDo not make
D3D11_CREATE_DEVICE_VIDEO_SUPPORTa hard requirement of capture.
createD3DDevicenow requestsD3D11_CREATE_DEVICE_VIDEO_SUPPORTunconditionally. If an adapter or driver rejects that flag,D3D11CreateDevicefails and the whole recording fails, including the CPU readback path that never needed video support. Only the_DEBUGbranch retries with a reduced flag set.The GPU path does not depend on this flag for the capture device:
initializeDxgiEncodingDevicecreates its own encoder device withD3D11_CREATE_DEVICE_VIDEO_SUPPORT(electron/native/wgc-capture/src/mf_encoder.cpplines 760-782), and the capture device only needs to create the shared keyed-mutex bridge texture. Retry without the flag so a machine that lacks video support still records on the CPU path.🛡️ Proposed retry
if (!succeeded(hr, "D3D11CreateDevice")) { - return false; + // Video support is only useful to the GPU encode path, which has its + // own device. Never let it cost the recording. + flags &= ~D3D11_CREATE_DEVICE_VIDEO_SUPPORT; + hr = D3D11CreateDevice( + nullptr, + D3D_DRIVER_TYPE_HARDWARE, + nullptr, + flags, + featureLevels, + ARRAYSIZE(featureLevels), + D3D11_SDK_VERSION, + &d3dDevice_, + &featureLevel, + &d3dContext_); + if (!succeeded(hr, "D3D11CreateDevice(no video support)")) { + return false; + } }Verify this on real Windows hardware before merge: CI runs only on Linux, so native capture changes need a manual smoke test. Based on coding guidelines: "Native capture changes require a manual smoke test on real macOS or Windows, because CI runs only on Linux."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@electron/native/wgc-capture/src/wgc_session.cpp` around lines 66 - 111, Update WgcSession::createD3DDevice to retry D3D11CreateDevice without D3D11_CREATE_DEVICE_VIDEO_SUPPORT when the initial creation fails, while retaining D3D11_CREATE_DEVICE_DEBUG handling in debug builds. Preserve the existing failure check and ensure devices without video support can continue through the CPU readback path. Manually smoke-test native capture on real Windows hardware.Source: Coding guidelines
🧹 Nitpick comments (2)
electron/native/wgc-capture/src/mf_encoder.cpp (1)
235-239: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueLog the hardware-transform attribute failure.
Every other failure branch in
createSinkWriterFromUrlprints the label and the HRESULT. This branch returns silently, so a failure to setMF_READWRITE_ENABLE_HARDWARE_TRANSFORMSproduces no diagnostic and is then reported under theConfigureDxgiManagerstage, which names a different step.♻️ Proposed change
hr = attributes->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, TRUE); if (FAILED(hr)) { + std::cerr << "ERROR: Set MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS(TRUE) failed (hr=0x" + << std::hex << hr << std::dec << ")" << std::endl; failedStage = SinkWriterCreateStage::ConfigureDxgiManager; return hr; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@electron/native/wgc-capture/src/mf_encoder.cpp` around lines 235 - 239, Update the hardware-transform attribute failure branch in createSinkWriterFromUrl to log the failure label and HRESULT before returning, consistent with the other failure branches. Use the correct stage label for setting MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS instead of reporting ConfigureDxgiManager.electron/native/wgc-capture/src/mf_encoder.h (1)
76-79: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the "success with no sample" result of
captureDxgiSample.
captureDxgiSamplereturnstrueand leavesoutSampleempty when the keyed-mutex bridge is contended (seemf_encoder.cpplines 1098-1101). A caller that only checks the return value writes nothing and does not know why. The neighbouringcaptureVideoSamplehas a detailed contract comment; state this one too, so the skip semantics stay discoverable from the header.📝 Proposed comment
+ // Returns false only on a real failure. A momentarily contended GPU + // bridge returns true with `outSample` empty: the caller must treat that + // as a skipped frame, not as a sample. bool captureDxgiSample( ID3D11Texture2D* texture, int64_t timestampHns, Microsoft::WRL::ComPtr<IMFSample>& outSample);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@electron/native/wgc-capture/src/mf_encoder.h` around lines 76 - 79, Add a contract comment immediately above captureDxgiSample documenting that it may return true with outSample empty when the keyed-mutex bridge is contended, and that callers must handle this as a skipped capture rather than a produced sample. Match the detail and style of the neighboring captureVideoSample documentation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@electron/native/README.md`:
- Line 88: The documentation incorrectly claims shared keyed-mutex texture
creation falls back to CPU encoding, although
MFEncoder::convertBgraTextureToNv12 creates it after initialization and failure
stops recording. Update electron/native/README.md:88-88,
technical-documentation/architecture/recording.md:72-72, and
electron/native/wgc-capture/src/mf_encoder.h:32-37 to remove that fallback claim
or explicitly state that bridge creation failure stops recording; no code change
is requested.
In `@electron/native/wgc-capture/src/main.cpp`:
- Around line 610-618: Update the encoderOptions.useDxgiInput condition to use
the resolved config.webcamEnabled value instead of webcamActive, while
preserving writeSeparateWebcam and the software-encoder and environment-variable
checks. Keep inline webcam PiP on the CPU path when webcamEnabled is true and no
separate webcam output is configured; verify with a real Windows webcam
recording.
In `@electron/native/wgc-capture/src/mf_encoder.cpp`:
- Around line 729-748: Update MFEncoder::finalize() to call
releaseDxgiPipeline() before MFShutdown(), then reset captureContext_ and
captureDevice_ before completing teardown. Preserve the existing
stagingTexture_, context_, and device_ cleanup, and verify the destruction order
with a real Windows hardware smoke test.
- Around line 941-994: Move the shared bridge-texture setup currently guarded by
captureBridgeTexture_ into initializeDxgiPipeline(), using width_, height_, and
the validated BGRA format to construct its descriptor without a WGC frame.
Ensure every creation, mutex, shared-resource, encoder-open, and input-view
failure causes initialization to select the existing CPU fallback rather than
returning Nv12ConvertResult::Failed from captureDxgiSample; keep per-frame
processing limited to using the already-initialized bridge resources.
- Around line 996-1001: Update the AcquireSync result handling in the
capture-side mutex path to return Nv12ConvertResult::Contended only when the
result is WAIT_TIMEOUT. Propagate or classify all other failure results,
including WAIT_ABANDONED and device errors, as non-recoverable using the
existing error-handling contract.
---
Outside diff comments:
In `@electron/native/wgc-capture/src/wgc_session.cpp`:
- Around line 66-111: Update WgcSession::createD3DDevice to retry
D3D11CreateDevice without D3D11_CREATE_DEVICE_VIDEO_SUPPORT when the initial
creation fails, while retaining D3D11_CREATE_DEVICE_DEBUG handling in debug
builds. Preserve the existing failure check and ensure devices without video
support can continue through the CPU readback path. Manually smoke-test native
capture on real Windows hardware.
---
Nitpick comments:
In `@electron/native/wgc-capture/src/mf_encoder.cpp`:
- Around line 235-239: Update the hardware-transform attribute failure branch in
createSinkWriterFromUrl to log the failure label and HRESULT before returning,
consistent with the other failure branches. Use the correct stage label for
setting MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS instead of reporting
ConfigureDxgiManager.
In `@electron/native/wgc-capture/src/mf_encoder.h`:
- Around line 76-79: Add a contract comment immediately above captureDxgiSample
documenting that it may return true with outSample empty when the keyed-mutex
bridge is contended, and that callers must handle this as a skipped capture
rather than a produced sample. Match the detail and style of the neighboring
captureVideoSample documentation.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 76007985-6f1d-43a8-b657-19d79c83e829
📒 Files selected for processing (6)
electron/native/README.mdelectron/native/wgc-capture/src/main.cppelectron/native/wgc-capture/src/mf_encoder.cppelectron/native/wgc-capture/src/mf_encoder.helectron/native/wgc-capture/src/wgc_session.cpptechnical-documentation/architecture/recording.md
| Encoder selection: by default the helper keeps the existing sink-writer path first. If that path fails while setting up H.264, it retries with the Microsoft software H.264 encoder (`mfh264enc.dll`). The key of this retry is registering that encoder locally in the helper process via `MFTRegisterLocalByCLSID`, which makes a software H.264 encoder available even when the machine's hardware encoders are missing or broken; hardware transforms are disabled for the retry only as a secondary guard so the sink writer prefers the locally registered software encoder, not as the fallback mechanism itself. Set `preferSoftwareEncoder: true` in the helper JSON, or set `OPENSCREEN_WGC_PREFER_SOFTWARE_ENCODER=true` before launching Electron, to force the software path from the first attempt. | ||
|
|
||
| The helper reports the outcome through the `encoder-selection` stdout event (`video` is `default`, `software-preferred`, or `software-fallback`). When the app sees `software-fallback` — the default encoder failed and the helper switched on its own — it shows a small dismissible notice in the recording HUD with a "Don't show again" option, because software encoding can raise CPU usage. An explicit `software-preferred` selection shows no notice, and the event stays available for diagnostics either way. | ||
| Frame input path: the helper feeds the encoder from the GPU when it can. On that path it copies the WGC frame across a keyed-mutex bridge to a second D3D11 device, converts BGRA to NV12 with the D3D11 video processor, and submits an allocator-owned DXGI sample to the hardware H.264 encoder, so no frame ever passes through system memory. The alternative is the original path: a staging texture, `Map(D3D11_MAP_READ)`, and a row-by-row copy into an `IMFMediaBuffer` — which is where a driver stall costs a recording (issue #252). The GPU path is a preference, never a requirement: it is skipped outright for `preferSoftwareEncoder` and for inline webcam PiP (both need the frame in system memory), and it degrades to the CPU path on its own if the encoding device, the NV12 video processor, the shared bridge texture, the DXGI sample allocator, or the hardware sink writer is unavailable. Set `OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1` to force the CPU path. Because the two paths land on different encoders and hardware MFTs default to constant bitrate, the GPU path asks for VBR through `ICodecAPI`; without it a static screen spends the full configured budget (measured 16.9 Mbps against 1.95 for the same desktop). |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
The "shared keyed-mutex texture" is not a fallback point. All three places state that the GPU path degrades to the CPU path when the shared keyed-mutex texture is unavailable. It does not. MFEncoder::convertBgraTextureToNv12 creates that texture on the first frame (electron/native/wgc-capture/src/mf_encoder.cpp lines 941-994), after initialize() already configured the sink writer for NV12, so a failure there fails the recording. Either move the bridge creation into initializeDxgiPipeline() as proposed on electron/native/wgc-capture/src/mf_encoder.cpp lines 941-994, or correct all three statements.
electron/native/README.md#L88-L88: remove "the shared bridge texture" from the list of automatic degrade conditions, or state that a bridge failure stops the recording.technical-documentation/architecture/recording.md#L72-L72: remove "no shared keyed-mutex texture" from the "degrades to the CPU one on its own at every step" list, or state the exception.electron/native/wgc-capture/src/mf_encoder.h#L32-L37: drop the claim that a driver which refuses shared keyed-mutex textures "records exactly as it did before the path existed", or qualify it.
📍 Affects 3 files
electron/native/README.md#L88-L88(this comment)technical-documentation/architecture/recording.md#L72-L72electron/native/wgc-capture/src/mf_encoder.h#L32-L37
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/README.md` at line 88, The documentation incorrectly claims
shared keyed-mutex texture creation falls back to CPU encoding, although
MFEncoder::convertBgraTextureToNv12 creates it after initialization and failure
stops recording. Update electron/native/README.md:88-88,
technical-documentation/architecture/recording.md:72-72, and
electron/native/wgc-capture/src/mf_encoder.h:32-37 to remove that fallback claim
or explicitly state that bridge creation failure stops recording; no code change
is requested.
| // Keep the CPU path for software encoding and inline webcam PiP: both need | ||
| // the frame in system memory, which is the one thing the DXGI path does not | ||
| // produce. The env var is the escape hatch for a machine where the GPU path | ||
| // misbehaves in a way the encoder's own probes do not catch -- a support | ||
| // answer instead of a hotfix. | ||
| encoderOptions.useDxgiInput = | ||
| !config.preferSoftwareEncoder && | ||
| (!webcamActive || writeSeparateWebcam) && | ||
| readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
webcamActive is always false here, so inline webcam PiP silently loses the overlay.
webcamActive is initialized to false at line 531 and set to true only at line 977, after encoder.initialize() at line 621. At line 617 it is therefore always false, and (!webcamActive || writeSeparateWebcam) is always true.
A recording with webcamEnabled: true and no webcamOutputPath then selects the DXGI path. Line 842 passes webcamFrame only in the CPU branch, so the picture-in-picture overlay is never composed. The recording succeeds and reports videoInput: "dxgi-nv12", so nothing signals the missing webcam.
Use the configuration value that is already resolved at this point.
🐛 Proposed fix
encoderOptions.useDxgiInput =
!config.preferSoftwareEncoder &&
- (!webcamActive || writeSeparateWebcam) &&
+ (!config.webcamEnabled || writeSeparateWebcam) &&
readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0;config.webcamEnabled is already cleared at line 551 when webcam initialization fails, and writeSeparateWebcam is assigned at line 558, so both are final by line 615.
Verify with a webcam recording on real Windows hardware: CI runs only on Linux. Based on coding guidelines: "Native capture changes require a manual smoke test on real macOS or Windows, because CI runs only on Linux."
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Keep the CPU path for software encoding and inline webcam PiP: both need | |
| // the frame in system memory, which is the one thing the DXGI path does not | |
| // produce. The env var is the escape hatch for a machine where the GPU path | |
| // misbehaves in a way the encoder's own probes do not catch -- a support | |
| // answer instead of a hotfix. | |
| encoderOptions.useDxgiInput = | |
| !config.preferSoftwareEncoder && | |
| (!webcamActive || writeSeparateWebcam) && | |
| readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0; | |
| // Keep the CPU path for software encoding and inline webcam PiP: both need | |
| // the frame in system memory, which is the one thing the DXGI path does not | |
| // produce. The env var is the escape hatch for a machine where the GPU path | |
| // misbehaves in a way the encoder's own probes do not catch -- a support | |
| // answer instead of a hotfix. | |
| encoderOptions.useDxgiInput = | |
| !config.preferSoftwareEncoder && | |
| (!config.webcamEnabled || writeSeparateWebcam) && | |
| readEnvInt("OPENSCREEN_WGC_DISABLE_DXGI_INPUT", 0) == 0; |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/wgc-capture/src/main.cpp` around lines 610 - 618, Update the
encoderOptions.useDxgiInput condition to use the resolved config.webcamEnabled
value instead of webcamActive, while preserving writeSeparateWebcam and the
software-encoder and environment-variable checks. Keep inline webcam PiP on the
CPU path when webcamEnabled is true and no separate webcam output is configured;
verify with a real Windows webcam recording.
Source: Coding guidelines
| void MFEncoder::releaseDxgiPipeline() { | ||
| bridgeInputView_.Reset(); | ||
| encoderBridgeMutex_.Reset(); | ||
| encoderBridgeTexture_.Reset(); | ||
| captureBridgeMutex_.Reset(); | ||
| captureBridgeTexture_.Reset(); | ||
| videoProcessor_.Reset(); | ||
| videoProcessorEnumerator_.Reset(); | ||
| videoContext_.Reset(); | ||
| videoDevice_.Reset(); | ||
| videoSampleAllocator_.Reset(); | ||
| dxgiDeviceManager_.Reset(); | ||
| dxgiResetToken_ = 0; | ||
| // Put the encoder back on the capture device. initializeDxgiEncodingDevice | ||
| // overwrites device_/context_ with the second device it creates, and the | ||
| // CPU path's staging texture has to live on the same device the WGC frames | ||
| // do or its CopyResource silently does nothing. | ||
| device_ = captureDevice_; | ||
| context_ = captureContext_; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Release the DXGI resources in finalize() as well.
releaseDxgiPipeline() is the only place that resets the new DXGI state, and initialize() calls it only on a fallback. On a successful GPU recording the resources live until the MFEncoder destructor runs.
finalize() (lines 1321-1338) resets stagingTexture_, context_, and device_, then calls MFShutdown(). Two consequences follow:
videoSampleAllocator_anddxgiDeviceManager_are Media Foundation objects that stay alive acrossMFShutdown().captureDevice_andcaptureContext_keep the WGC D3D11 device alive afterfinalize(), sosession.stop()inmain.cppno longer releases the last reference.
Call releaseDxgiPipeline() from finalize() before MFShutdown(), and reset captureDevice_/captureContext_ there too.
🧹 Proposed change in `finalize()`
stagingTexture_.Reset();
releaseDxgiPipeline();
captureContext_.Reset();
captureDevice_.Reset();
context_.Reset();
device_.Reset();
MFShutdown();Verify the teardown order on real Windows hardware: CI runs only on Linux. Based on coding guidelines: "Native capture changes require a manual smoke test on real macOS or Windows, because CI runs only on Linux."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/wgc-capture/src/mf_encoder.cpp` around lines 729 - 748,
Update MFEncoder::finalize() to call releaseDxgiPipeline() before MFShutdown(),
then reset captureContext_ and captureDevice_ before completing teardown.
Preserve the existing stagingTexture_, context_, and device_ cleanup, and verify
the destruction order with a real Windows hardware smoke test.
Source: Coding guidelines
| if (!captureBridgeTexture_) { | ||
| D3D11_TEXTURE2D_DESC bridgeDesc{}; | ||
| texture->GetDesc(&bridgeDesc); | ||
| bridgeDesc.BindFlags = D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET; | ||
| bridgeDesc.CPUAccessFlags = 0; | ||
| bridgeDesc.Usage = D3D11_USAGE_DEFAULT; | ||
| bridgeDesc.MiscFlags = D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX; | ||
| if (!succeeded( | ||
| captureDevice_->CreateTexture2D(&bridgeDesc, nullptr, &captureBridgeTexture_), | ||
| "CreateTexture2D(capture bridge)")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
| if (!succeeded(captureBridgeTexture_.As(&captureBridgeMutex_), "Query capture bridge mutex")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
|
|
||
| Microsoft::WRL::ComPtr<IDXGIResource> bridgeResource; | ||
| if (!succeeded(captureBridgeTexture_.As(&bridgeResource), "Query capture bridge resource")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
| HANDLE sharedHandle = nullptr; | ||
| if (!succeeded(bridgeResource->GetSharedHandle(&sharedHandle), "Get capture bridge handle")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
| if (!succeeded( | ||
| device_->OpenSharedResource( | ||
| sharedHandle, | ||
| __uuidof(ID3D11Texture2D), | ||
| reinterpret_cast<void**>(encoderBridgeTexture_.GetAddressOf())), | ||
| "Open encoder bridge texture")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
| if (!succeeded(encoderBridgeTexture_.As(&encoderBridgeMutex_), "Query encoder bridge mutex")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
|
|
||
| // The bridge is the only input this processor ever reads, so its view | ||
| // is built once here rather than per frame. Views describe a resource, | ||
| // they do not read it, so this needs no keyed-mutex ownership. | ||
| D3D11_VIDEO_PROCESSOR_INPUT_VIEW_DESC inputViewDesc{}; | ||
| inputViewDesc.FourCC = 0; | ||
| inputViewDesc.ViewDimension = D3D11_VPIV_DIMENSION_TEXTURE2D; | ||
| inputViewDesc.Texture2D.MipSlice = 0; | ||
| inputViewDesc.Texture2D.ArraySlice = 0; | ||
| if (!succeeded( | ||
| videoDevice_->CreateVideoProcessorInputView( | ||
| encoderBridgeTexture_.Get(), | ||
| videoProcessorEnumerator_.Get(), | ||
| &inputViewDesc, | ||
| &bridgeInputView_), | ||
| "CreateVideoProcessorInputView")) { | ||
| return Nv12ConvertResult::Failed; | ||
| } | ||
| } |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Bridge-texture setup failure ends the recording; it does not degrade to the CPU path.
This block creates the shared keyed-mutex texture, opens it on the encoder device, and builds the input view. It runs on the first frame, long after initialize() configured the sink writer for NV12. Every failure here returns Nv12ConvertResult::Failed, which captureDxgiSample reports as false, which main.cpp (lines 845-849) turns into encodeFailed plus a stop request.
The documented contract says the opposite. mf_encoder.h lines 32-36 state that "a driver that refuses shared keyed-mutex textures records exactly as it did before the path existed", and both electron/native/README.md and technical-documentation/architecture/recording.md list "no shared keyed-mutex texture" as an automatic degrade. No degrade is possible at this point.
Move the bridge creation into initializeDxgiPipeline(), where a failure still falls back to the CPU path. initialize() already knows width_, height_, and the BGRA format that captureDxgiSample validates, so the descriptor does not need a WGC frame.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/wgc-capture/src/mf_encoder.cpp` around lines 941 - 994, Move
the shared bridge-texture setup currently guarded by captureBridgeTexture_ into
initializeDxgiPipeline(), using width_, height_, and the validated BGRA format
to construct its descriptor without a WGC frame. Ensure every creation, mutex,
shared-resource, encoder-open, and input-view failure causes initialization to
select the existing CPU fallback rather than returning Nv12ConvertResult::Failed
from captureDxgiSample; keep per-frame processing limited to using the
already-initialized bridge resources.
| // Key 0 is the capture side's, key 1 the encoder's. Timing out here leaves | ||
| // key 0 exactly where it was, so the next frame simply tries again; that | ||
| // is the whole reason this one is recoverable and the one below is not. | ||
| if (FAILED(captureBridgeMutex_->AcquireSync(0, acquireTimeoutMs))) { | ||
| return Nv12ConvertResult::Contended; | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Classify only a timeout as Contended.
AcquireSync returns WAIT_TIMEOUT for a busy bridge, but it also returns hard errors such as DXGI_ERROR_DEVICE_REMOVED, E_FAIL, and WAIT_ABANDONED. FAILED(...) maps all of them to Nv12ConvertResult::Contended.
A permanently broken bridge then skips every remaining frame. main.cpp counts each skip and keeps going, so the recording ends with exit code 0, a recording-stopped event, and an MP4 that holds almost no frames. Treat only WAIT_TIMEOUT as recoverable.
🐛 Proposed fix
- if (FAILED(captureBridgeMutex_->AcquireSync(0, acquireTimeoutMs))) {
- return Nv12ConvertResult::Contended;
- }
+ const HRESULT acquireHr = captureBridgeMutex_->AcquireSync(0, acquireTimeoutMs);
+ if (acquireHr == static_cast<HRESULT>(WAIT_TIMEOUT)) {
+ return Nv12ConvertResult::Contended;
+ }
+ if (!succeeded(acquireHr, "Acquire capture bridge")) {
+ return Nv12ConvertResult::Failed;
+ }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/wgc-capture/src/mf_encoder.cpp` around lines 996 - 1001,
Update the AcquireSync result handling in the capture-side mutex path to return
Nv12ConvertResult::Contended only when the result is WAIT_TIMEOUT. Propagate or
classify all other failure results, including WAIT_ABANDONED and device errors,
as non-recoverable using the existing error-handling contract.
|
I tested the Windows x64 diagnostic helper from this PR artifact on the machine that reproduces #252 (artifact run 31257864130). Results:
System-audio capture still reproduces the failure on this machine. I reproduced it with both a 10-second run (15-second stop budget) and a 5-second run (8-second stop budget). The shorter run produced: The first system-audio run showed the same |
|
Update after testing the exact PR #305 helper copied into the standalone 1.9+3.5 package:
This exact helper is now packaged locally for further testing. The intermittent failure remains on the affected machine even without system audio; system-audio runs also reproduce it consistently. |
#252's reporter confirmed the GPU path fixes display and window capture on the machine that reproduces it, and found two failures left: one consistent with system audio, one intermittent without it. Both report `wgc-quiesce drained=false` then `video-writer-join phase=abandoned`, which means the video writer is stuck while holding the frame lock and every WGC callback is queued behind it. The system-audio one is a lock-order defect, and it predates the GPU path. `writerMutex_` is held across IMFSinkWriter::WriteSample by both submitVideoSample and writeAudio -- a synchronous encode -- while the capture* entry points took that same mutex just to stamp a sample, from inside main.cpp's frame lock. So an audio write on the mixer thread stalls the video writer, the writer stalls the WGC callbacks, and stop finds nothing drainable. The sample clock moves to a `timestampMutex_` of its own, which no blocking call is ever held across, and the sinkWriter_/finalized_ check goes away with it: submitVideoSample already makes that check before writing, so a sample built for a writer that has gone is discarded one step later instead of costing a lock. The intermittent one is not diagnosable from here, so instrument it rather than guess. The encoder now keeps a breadcrumb of the call it is inside, and the shutdown watchdog prints it: `phase=abandoned encode_stage=bridge-copy` says which driver call wedged, where `encode_stage=idle` says the writer never got into the encoder at all. That is the same move that made #252 legible in the first place. Verified by forcing the failure shape locally with OPENSCREEN_WGC_TEST_STALL_READBACK_MS: wgc-quiesce drained=false at 5001ms, video-writer-join abandoned at 8021ms, exit 3, and the breadcrumb correctly reads `idle` for a stall that is outside the encoder. Display, window, system audio, the software fallback knob and the CPU kill switch all still pass.
|
Thanks, that is exactly the data needed. Display and window fixed, two failures left, and both your traces say the same thing: Pushed two changes. The system-audio one is a lock-order defect, and it predates the GPU path. The intermittent one I cannot diagnose from here, so it is instrumented rather than guessed at. The encoder keeps a breadcrumb of the call it is inside and the watchdog prints it:
I reproduced your failure shape locally by forcing a stall under the frame lock: Could you re-run on the affected machine? Two things would settle it:
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@electron/native/wgc-capture/src/mf_encoder.cpp`:
- Around line 1271-1278: Move the video `encodeStage_ = "write-sample"`
assignment in the video write path to after `writerMutex_` is acquired, keeping
the existing idle cleanup. In the audio write method, set an audio-specific
stage immediately after locking `writerMutex_` and clear it after the audio
`WriteSample` call, including the early-return path, so the shutdown watchdog
reports the operation that currently owns the writer.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9e439471-82fa-4fd8-87de-3e7bc6c07e86
📒 Files selected for processing (3)
electron/native/wgc-capture/src/main.cppelectron/native/wgc-capture/src/mf_encoder.cppelectron/native/wgc-capture/src/mf_encoder.h
🚧 Files skipped from review as they are similar to previous changes (1)
- electron/native/wgc-capture/src/main.cpp
| encodeStage_ = "write-sample"; | ||
| std::scoped_lock writerLock(writerMutex_); | ||
| if (!sinkWriter_ || finalized_) { | ||
| encodeStage_ = "idle"; | ||
| return false; | ||
| } | ||
| return succeeded(sinkWriter_->WriteSample(videoStreamIndex_, sample), "WriteSample"); | ||
| const bool written = succeeded(sinkWriter_->WriteSample(videoStreamIndex_, sample), "WriteSample"); | ||
| encodeStage_ = "idle"; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Report the active writer operation.
Line 1271 sets encodeStage_ before writerMutex_ is acquired. If writeAudio owns the mutex in IMFSinkWriter::WriteSample, a waiting video thread overwrites the stage with "write-sample". writeAudio does not report an audio write stage. The shutdown watchdog can report the wrong blocking operation.
Set the video stage after acquiring writerMutex_. Set and clear an audio stage around the audio WriteSample call.
Proposed fix
bool MFEncoder::submitVideoSample(IMFSample* sample) {
- encodeStage_ = "write-sample";
std::scoped_lock writerLock(writerMutex_);
if (!sinkWriter_ || finalized_) {
encodeStage_ = "idle";
return false;
}
+ encodeStage_ = "write-sample";
const bool written = succeeded(sinkWriter_->WriteSample(videoStreamIndex_, sample), "WriteSample");
encodeStage_ = "idle";
return written;
}
bool MFEncoder::writeAudio(...) {
std::scoped_lock writerLock(writerMutex_);
// Validate and construct sample.
- return succeeded(sinkWriter_->WriteSample(audioStreamIndex_, sample.Get()), "WriteSample(audio)");
+ encodeStage_ = "write-audio";
+ const bool written =
+ succeeded(sinkWriter_->WriteSample(audioStreamIndex_, sample.Get()), "WriteSample(audio)");
+ encodeStage_ = "idle";
+ return written;
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@electron/native/wgc-capture/src/mf_encoder.cpp` around lines 1271 - 1278,
Move the video `encodeStage_ = "write-sample"` assignment in the video write
path to after `writerMutex_` is acquired, keeping the existing idle cleanup. In
the audio write method, set an audio-specific stage immediately after locking
`writerMutex_` and clear it after the audio `WriteSample` call, including the
early-return path, so the shutdown watchdog reports the operation that currently
owns the writer.
2 participants
Builds on @Seb1900's prototype in #304, rebased onto
main(that branch was cut fromrelease/v1.9.0and conflicts). Their commit is kept as-is; the second commit is the hardening.What #304 got right
The screen path encodes from a CPU readback:
MFVideoFormat_RGB32sink writer input, staging texture,Map(D3D11_MAP_READ), memcpy,Unmap— all on the same D3D11 device/context as WGC, under the shared frame lock. On the reporter's machine (Windows 10, WDDM 2.7, RTX 5070 Ti + AMD iGPU, two virtual display adapters)Unmapnever returns, so the writer thread holds the frame lock,wgc-quiescereportsdrained=false, andvideo-writer-joinis abandoned by the watchdog beforeencoder-finalize— an empty MP4. Their trace and ours agree on the step.The DXGI path removes that call entirely. It is the right fix.
What this PR changes
The GPU path is now a preference, never a requirement. In #304 every DXGI setup failure was a
return false, including a hard error placed between the default sink-writer attempt and the software H.264 retry. SinceuseDxgiInputis on by default for any recording without inline PiP, that made the software fallback unreachable: a machine with no hardware H.264 encoder (VM, RDP session, older iGPU) went from records in software to native recording fails. Now the encoding device, the NV12 video processor, the bridge texture, the sample allocator and the hardware sink writer each drop the whole pipeline and retry the exact chain a machine without a GPU path would have taken.releaseDxgiPipeline()restoresdevice_/context_to the capture device, because the CPU path's staging texture has to live where the WGC frames do.OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1forces it off.No multi-second wait under the frame lock. The bridge acquire was
AcquireSync(..., 5000), taken on the video-writer thread while it holds the very lock #252 is about, against an 8s watchdog step budget. It is now a few frame intervals, and a timeout skips the frame rather than ending the recording. The timestamp is stamped after the conversion, so a skipped frame no longer stretches the timeline. Skips are counted and printed once at stop.Which path ran is now observable. It is a per-machine outcome, so callers read
usesDxgiInput()instead of their own request, andencoder-selectioncarriesvideoInput: "dxgi-nv12" | "cpu-rgb32".Also: the injected-sink-writer-failure test knob now disables the GPU path, so it still proves what it was written to prove; per-frame processor rect/colourspace calls and the input view are hoisted out of the frame loop;
MF_LOW_LATENCYis dropped (measured, no effect).Measured
Verified end to end on a working Windows machine by driving the packaged helper directly. GPU path against CPU path, same idle desktop:
dxgi-nv12)cpu-rgb32)The bitrate one was the surprise: the D3D manager switches the sink writer onto a hardware MFT, and hardware MFTs default to CBR, so a static screen spent the full configured 18 Mbps budget — an 8x file.
MF_MT_AVG_BITRATEalone does not move them; asking for VBR throughICodecAPIdoes.Colour was the other risk, since #304 ran
VideoProcessorBltwith no colourspace set. The processor is now told full-range BGRA in, studio BT.709 out, with matching tags on both media types. The two paths measure the same.Also checked:
preferSoftwareEncoder: true→software-preferred+cpu-rgb32; injected sink-writer failure →software-fallback+cpu-rgb32;OPENSCREEN_WGC_DISABLE_DXGI_INPUT=1→cpu-rgb32; two consecutive recordings; 1080p30 and 60 fps.What we cannot verify
We have no hardware that reproduces #252, so none of the above proves the deadlock is gone — only that the GPU path is correct and the fallbacks work where we can run them. @Seb1900, could you confirm on the machine that fails? The
videoInputfield inencoder-selectionand the[frame-drops]line at stop should make it obvious which path ran.Supersedes #304. Closes #252 once confirmed.
Summary by CodeRabbit
New Features
Documentation