DepthForge NR

A native Windows experiment that opens a YouTube link in an app-owned WebView2 player, captures its compositor surface on the GPU, estimates live monocular depth with DirectML, and prepares the result for NVIDIA DLSS Neural Rendering experiments.

DepthForge NR is by SamG-Coder. It is an independent research project and is not an NVIDIA, Microsoft, YouTube, or Google product.

Current state

The native-only path currently provides:

• a sandboxed WebView2 YouTube player with an HTTPS youtube.com/youtu.be address bar;
• exact-window capture through Windows Graphics Capture;
• zero-copy D3D11-to-D3D12 sharing of the compositor frame;
• D3D12 resize and normalization;
• Depth Anything V2 Small FP16 inference through DirectML on the selected GPU;
• a persistent streaming pipeline that reuses capture, shaders, DirectML, ONNX Runtime, and GPU tensor allocations between frames;
• a continuously updating in-memory native depth-preview window with stable temporal exposure and transient-black-frame rejection;
• an integrated GPU Studio toolbar for source focus, output-stage status, inversion, range tracking, stabilization, black-frame hold, exposure, contrast, live FPS, and overlay collapse;
• a persistent double-buffered native presenter that composes the image and toolbar off-screen and publishes them with one paint operation;
• a pinned, hash-verified model downloader; and
• a one-click build/run launcher.

The continuous preview is presently the Depth stage. The project does not label that preview as DLSS output. Persistent guide textures, motion, the feature-1004 call, and the native A/B presenter are the next integration stage. The current depth-only stage measured approximately 55-60 FPS after warm-up on the development RTX 5080 system; results vary by system and workload.

Architecture

YouTube URL
    |
    v
app-owned sandboxed WebView2 player
    |
    v
Windows Graphics Capture (exact HWND)
    |
    v
shared ID3D11Texture2D -> same allocation opened on D3D12
    |
    v
D3D12 resize + RGB normalization
    |
    v
Depth Anything V2 Small via ONNX Runtime DirectML
    |
    v
GPU-resident relative-depth tensor
    |
    +--> current native live depth preview
    |
    `--> planned color/motion/depth guides -> local DLSS NR -> native compare

TypeGPU, WebGPU, getDisplayMedia, browser extensions, and arbitrary browser GPU-process texture scanning are not runtime dependencies. WebView2 only hosts the controlled YouTube page. Native C++ owns capture, inference, processing, and output.

Run it

Double-click:

Run-NativeYouTube.cmd

On the first run the launcher:

1. downloads the Apache-2.0 Depth Anything V2 Small FP16 ONNX model;
2. verifies SHA-256 2DF6223F206B5164E21F664ACE61DABEB9BB6A49B8B5A3E00510B4807D0F5B04;
3. configures and builds the two native binaries;
4. opens the sandboxed YouTube player;
5. captures that exact player window; and
6. opens the continuously updating native depth window.

Close the Live Native Depth window or press Escape in it to stop. The launcher then closes the player it created.

To start with a particular video:

.\scripts\Run-NativeYouTube.ps1 `
  -Url "https://www.youtube.com/watch?v=VIDEO_ID"

Build

cmake -S . -B build
cmake --build build --config Release --parallel

The clean configuration prints:

DepthForge native YouTube player: enabled
DepthForge native D3D12/DirectML engine: enabled
TypeGPU/WebGPU runtime dependency: none

Outputs:

build/bin/Release/DepthForge.YouTubePlayer.exe
build/bin/Release/DepthForge.YouTubeGpuEngine.exe

Native components

Component	Responsibility
src/player/main.cpp	URL validation, WebView2 lifetime, YouTube player window
src/native/youtube_gpu_probe.cpp	exact-window capture, D3D11/D3D12 sharing, live preview lifetime
src/native/youtube_depth_pipeline.cpp	D3D12 preprocessing, DirectML inference, preview normalization
scripts/Get-DepthModel.ps1	pinned model download and SHA-256 verification
scripts/Run-NativeYouTube.ps1	build, player startup, HWND binding, continuous engine startup

Requirements

• Windows 10 or 11 x64 with Windows Graphics Capture
• Visual Studio/MSVC and a current Windows SDK
• CMake 3.24 or newer
• Microsoft Edge WebView2 Runtime
• a DirectX 12-capable GPU
• network access for YouTube and the first model download

The Microsoft WebView2 and Windows App SDK ML packages are pinned by CMake and are restored from NuGet when absent from the local cache.

DLSS Neural Rendering additionally requires the user's own lawfully obtained, signed NVIDIA/Streamline runtime files. Those files are not downloaded, committed, or redistributed by this project.

Security and content boundaries

• Only HTTPS youtube.com, www.youtube.com, m.youtube.com, and youtu.be top-level navigation is accepted by the player.
• Capture is bound to the exact app-owned HWND.
• The native engine also verifies the target process image.
• The preview does not save source video frames.
• Protected content may appear black; DepthForge does not attempt to bypass DRM or protected-video restrictions.
• YouTube playback remains subject to YouTube's terms and the content owner's rights.

Read SECURITY.md before weakening process, navigation, model, or runtime validation.

Model and licensing

The default depth model is the FP16 ONNX export of Depth Anything V2 Small. The Small model is Apache-2.0 licensed. Model weights are stored under models/, which is ignored by Git.

Larger Depth Anything V2 variants have different licensing and are not silently substituted by the launcher.

DLSS status

The native engine can optionally compile the feature-1004/NGX feature-18 adapter when local Streamline SDK and signed runtime directories are supplied to CMake. The runtime files are copied only into the local build output and are never committed or redistributed. On the development RTX 5080 system the signed binaries, feature-18 mapping, Streamline initialization, D3D12 device attachment, and feature-1004 support query all pass; the toolbar reports this as FEATURE 1004 READY. A successful readiness query alone is not proof of visual improvement. DLSS Output remains locked until all of the following are true for the same persistent D3D12 device:

• color is R16G16B16A16_FLOAT;
• motion is R16G16_FLOAT and honestly represents the current implementation;
• depth is R32_FLOAT with an explicit near/far convention;
• output is a separate R16G16B16A16_FLOAT UAV;
• the feature evaluation is submitted and retired successfully; and
• Source, Output, Compare, and Difference views show the live result.

Until then, the Studio continues to identify the visible stage as Depth.

Configure a local DLSS-enabled build without placing machine paths in source:

cmake -S . -B build-native-dlss `
  -DDEPTHFORGE_STREAMLINE_SDK_ROOT="C:/path/to/streamline" `
  -DDEPTHFORGE_NGX_RUNTIME_DIR="C:/path/to/signed/runtime"
cmake --build build-native-dlss --config Release

License

DepthForge NR source is released under the MIT License. Third-party components, model weights, services, and NVIDIA runtime files remain under their respective terms. NVIDIA, RTX, DLSS, Microsoft, WebView2, Google, and YouTube names and marks belong to their respective owners.