How It Was Built Part 3

Part of How ShadePlay was built

Part 3: the picture

Chapter 1: capturing the screen

The picture never leaves the graphics card. This chapter is about getting every new screen image, in HDR, at up to 360 frames per second, without disturbing the game.

Desktop Duplication, and HDR frames

September 24

Problem. Which way of capturing the screen can deliver HDR at 240 or 360 fps, reliably, for days?

What we found.

• Desktop Duplication hands a program each new desktop image directly on the graphics card. It delivered about 359.8 images per second on the 360 Hz monitor, and checking for a new one takes a few microseconds.
• Asked the modern way (DuplicateOutput1), it delivers an HDR desktop as 16-bit floating-point frames: the real colours, including highlights far above normal white.
• Asked the old way, or for 8-bit frames only, it hands over an HDR desktop clipped to SDR: everything brighter than 80 nits came out white.
• The frame format can change from one frame to the next. One test run unexpectedly got 10-bit and 8-bit frames.
• The alternatives fell short. Windows' newer capture interface is capped at about 60 fps unless a setting is used that the toolchain's headers didn't have, and it draws a yellow border unless borderless capture is granted. FFmpeg's own Desktop Duplication filter doesn't recover after a display change, and its pacing topped out at about 155-161 fps in a test. Hooking into games, as some recorders do, injects code into them, which anti-cheat systems block.

Fix. ShadePlay asks for 16-bit HDR frames first and 8-bit frames second, and checks the format of every frame. It never injects anything into a game.

The card that owns the monitor

September 24

Problem. On the test PC, capture failed on some graphics adapters with a vague "invalid argument" error.

What we found. Windows listed two adapters named "NVIDIA GeForce RTX 5080", and only one of them had the monitor attached. There was also the processor's integrated GPU, and there were virtual monitors from VR streaming apps. Capturing from any adapter except the one that owns the monitor fails.

Fix. ShadePlay finds the monitor it records and creates its graphics device on the adapter that owns that monitor: never by name, never by position in the list, never by Windows' per-app GPU preference (which can quietly move an app to the integrated GPU). The choice is made again after every display change. It is also why the README asks you to plug your monitor into the graphics card: a monitor on the motherboard makes the integrated GPU the recorder.

DPI awareness or nothing

September 24

Problem. The modern capture call failed with "unsupported" for every format list, while the old call worked.

What we found. DuplicateOutput1 only works for programs that declare themselves "per-monitor DPI aware, version 2", a setting about screen scaling that seems to have nothing to do with capture. Worse, "unsupported" is the same error Windows returns during a brief display-mode change, so an app that treats it as temporary retries forever and never learns why.

Fix. ShadePlay's manifest declares per-monitor-v2 awareness, and the capture code checks it (and sets it if needed) before duplicating.

Let go of the frame at once

September 24

Problem. Microsoft's guidance says to keep a captured frame until just before asking for the next one.

What we found. Doing that halved the capture rate. At 240 ticks a second, releasing the frame at once got roughly 440 to 460 new images out of 480; holding it got about 230. Other open-source recorders release right after copying, too.

Fix. ShadePlay copies each new frame into its own texture and releases the original straight away. The colour conversion reads the copy.

Drawing the mouse pointer ourselves

September 24

Problem. Desktop Duplication's image doesn't contain the mouse pointer.

What we found. Windows hands over the pointer's shape and position separately. Drawing it into an HDR frame needs care: FFmpeg's capture filter draws it at the brightness of 80 nits, six times too dark next to the test PC's 480-nit SDR white, and another open-source tool assumes 300 nits.

Fix. ShadePlay draws the pointer itself, inside the colour-conversion shader, scaled to the real SDR white level. That needs no extra copy and also covers frames in which only the pointer moved. The pointer can be switched off in the advanced settings.

A steady 240 frames per second

September 24

Problem. Games don't deliver frames at a steady rate, but video editors and YouTube want a constant frame rate, and the sound must never drift away from the picture.

What we found.

• A high-resolution Windows timer woke up on time: typically 0.28 ms late, and at worst about 1 ms, once Windows' power throttling was switched off for the recorder.
• Without that opt-out, a normal test program was up to 11-14 ms late.

Fix. A clock-driven loop. A timer ticks at the chosen frame rate, and each tick either converts a new image or sends the previous frame again, which costs no graphics work at all. The time stamp of every frame comes from its tick, not from when the image arrived, and a late tick skips forward instead of piling up. Picture and sound therefore share one timeline and can't drift apart. In the first hands-on test on September 27, a 60.8-second clip at 240 fps had 14,589 frames, and only 3 ticks came late.

One video thread owns every graphics, capture and encoder call, because NVIDIA's guide warns against splitting its encoder and Direct3D work across threads. When another program briefly blocks the encoder, capture waits too, but since the time stamps come from the clock, such a stall only shows up as a few skipped frames.

Monitor off, lock screen, UAC

September 24

Problem. Two ordinary situations looked like failures:

• The lock screen and UAC prompts run on a secure desktop that only a system process may capture. The recorder took the refusal as a fault and restarted itself every 60 to 75 seconds until the PC was unlocked.
• No monitor at all (unplugged, or switched off so that Windows drops it): a fault, a restart every 10 minutes, a red tray icon for hours and thousands of warnings in the log.

Fix. Both are calm standby states, not faults: a quiet tray icon, no restarts, one line in the log. The tray says that the screen is locked and that recording resumes by itself. Audio keeps running. While there is no picture, a save is refused rather than writing old video. Every retry looks for displays afresh, so a monitor that comes back is found without waiting for a Windows event. ShadePlay never keeps the display awake itself.

A monitor that is asleep, not gone

September 28

Problem. During the release work, with the monitor asleep, the recorder reported "No video (starting)" and restarted every 60 seconds.

What we found. A monitor that is asleep, or switched off but still attached, is not the same as one that is gone. Windows still lists it and screen capture still starts, but Windows draws nothing for it, so not even a first image ever arrives. The earlier fix only covered a missing display. Windows' own "display off" notification didn't arrive in a test copy either, so it couldn't be relied on.

Fix. A capture that never gets its first image now means "waiting for a display", exactly like a missing monitor: no restart and no "not recording" warning. The capture is re-created on a slowing schedule, from every 3 seconds up to once a minute, and recording starts again as soon as a picture arrives. A test copy checked it with the monitor asleep: over 80 seconds it waited calmly, with 0 restarts, where the build before had restarted every minute. Two new tests keep it that way.

Alt-tab without broken clips

September 24, seen live on September 27

Problem. Switching in and out of a fullscreen game, a UAC prompt or a display-mode switch makes Windows cancel the screen capture ("access lost").

What we found. If the size and the format stay the same, only the capture needs to be re-created. The encoder, and the clip's timeline, can keep running on repeated frames in the meantime.

Fix. An "access lost" that changes neither size nor format keeps the encoder and its epoch (Epochs: any change re-opens the encoder), so the clip doesn't split. In the first hands-on test, Overwatch reset the capture several times (going fullscreen, alt-tabbing), and each time recording resumed within milliseconds without breaking the clip.

Driver resets that restarted nothing

September 24

Problem. After a graphics driver reset, the recorder should rebuild everything. It never did.

What we found. The recorder checked "waiting for a display" before "device lost", and after a driver reset the video pipeline sits in its waiting state with the driver's error text. The first check always won.

Fix. The two checks were swapped, with a regression test that was proven to fail on the old order. This one was found while the encoder was being built, before the first review.

Chapter 2: colour and compression

From the desktop's colours to HDR10

September 24

Problem. An HDR desktop is composed in "scRGB": linear light, in which 1.0 means 80 nits. HDR video wants something else entirely: the BT.2020 colour range, the PQ brightness curve and 10-bit values.

What we found.

• Normal (SDR) windows on an HDR desktop are shown at the user's SDR white level. With the test PC's settings that was 480 nits, brighter than the panel's 456-nit peak. Many tools assume 80, 200 or 300 nits, which makes SDR content far too dark or too bright. Windows sends no event when it changes, so ShadePlay reads it about once a second.
• The 10-bit values must be stored with their low bits clean. One open-source recorder's formula leaves garbage in the low bits of 1,015 of the 1,024 values.
• FFmpeg's NVIDIA wrapper never states where the colour samples sit, and players assume "left", so the shader samples colour to the left.

Fix. ShadePlay's own small GPU programs (shaders) convert every frame on the graphics card: linear light, then BT.2020 colours, then absolute brightness, then PQ, then 10-bit video values, written straight into the encoder's input surface. SDR desktops go to BT.709, 10-bit by default. The results were checked against an independent reference: every value within one step, and 99.94 % exactly equal on a 1440p HDR frame. The conversion costs about 47 µs of GPU time per frame.

The shader that stalled the first frames

September 24

Problem. The first real clip dropped 5 frame ticks at the start, with a 25 ms gap between its first two frames.

What we found. Two causes, one hiding behind the other:

1. The shaders were compiled when they were first needed, about 90 ms in total.
2. Even after compiling them in advance, the first conversion still waited 35.7 ms: the NVIDIA driver translates a shader on its first draw, not when it is created.

Fix. Every shader variant is compiled once at start, on a background thread, and then drawn once into a tiny 2 x 2 target to warm the driver up: 4.6 ms when the driver's cache is warm, 31-52 ms when it is cold. After that the first conversion waits 0.1 to 0.4 ms. At the maintainer's first real start, all 13 variants were ready in 280 ms, and the review's live runs had 0 dropped ticks.

An SDR game on an HDR desktop

September 24

Problem. Alt-tabbing into an SDR game split the clip in two.

What we found. While an SDR game runs fullscreen on an HDR desktop, Windows can hand the screen to the game directly ("independent flip"), and Desktop Duplication then delivers 8-bit SDR frames. The recorder took that as a new format and started a new SDR segment.

Fix. On an HDR desktop, 8-bit frames go through the HDR path too: decoded, scaled to the SDR white level and turned into HDR10, the way Windows itself shows SDR content on an HDR desktop. The encoder settings stay the same, so alt-tabbing never splits a clip. A GPU check showed SDR white landing on exactly the same value as the desktop's own white. A live test with a real SDR game in that mode is still open.

Zero copies

September 24

Problem. Copying frames to the processor and back would cost time, memory bandwidth and CPU.

Fix. The picture never leaves the graphics card. The captured frame is copied into ShadePlay's own texture, the shader writes straight into a surface from FFmpeg's pool of Direct3D 11 frames, and the encoder reads that surface. FFmpeg's default pool of texture arrays failed on this driver, so each surface is a single texture. Only the compressed packets come back to the processor, into the replay buffer in memory. Nothing is written to disk until you save.

Why AV1

September 24

Problem. Which codec should be the default? The maintainer asked for AV1, and for "as detailed as possible" at 240 fps.

What we found (on the RTX 5080 at 1440p, with synthetic test content):

• AV1 is faster on this card: about 404 fps on one encoder engine at the chosen speed setting, against 304 for HEVC.
• AV1 is smaller: HEVC needed about 8-9 % more bits for the same quality, and H.264 about 21 % more.
• H.264 fell short of 240 fps at 1440p at a comparable setting (184-215 fps, partly measured while other programs used the encoder), and it can't split frames across both engines.

Fix. AV1 by default, HEVC as an option and as the automatic fallback (No AV1? Use HEVC), and H.264 for SDR compatibility only. AV1 clips need a recent player: VLC, or Microsoft's free AV1 Video Extension for the Windows Media Player app.

360 fps: both encoder engines at once

September 24

Problem. One encoder engine manages about 400 AV1 frames per second at 1440p. That leaves too little headroom for 360 fps.

What we found. The RTX 5080 can split each frame across both of its engines ("split-frame encoding"). Forced on, it reached about 700 fps. But it costs about 2 % efficiency, it switches on by itself only for much taller videos, and it collapses to about 236-250 fps as soon as any other program uses the encoder.

Fix. Split-frame encoding only above 300 fps. At 240 fps one engine does the work and the other stays free for Discord or anything else. H.264, which can't split frames, uses the fastest preset above 300 fps. A live 10-second test at 360 fps in HEVC ran at 359.8-360.07 fps with split-frame encoding and no dropped ticks.

Settings that cost more than they give

September 24

Problem. NVIDIA's encoder has many quality options. Which of them fit a real-time recorder at 240 fps?

What we found.

• FFmpeg's default quietly keeps the preset's own B-frames. That cost 21 % of the speed (317 instead of 404 fps) and reorders the frames, so "no B-frames" must be set explicitly.
• Lookahead occupies the second engine, multipass costs 22-55 % of the speed, the "ultra high quality" mode runs at about 170 fps, and temporal filtering needs B-frames. None of them fit.
• Spatial adaptive quantization raised the bitrate by 25-70 % at the same quality setting, runs on the GPU's shader cores (so it takes a little time from the game), and the quality steps were measured without it.

Fix. Preset p4 (the middle of seven), no B-frames, no lookahead, no multipass, and spatial adaptive quantization off by default. The advanced settings can turn it on; it is never used with split-frame encoding.

HDR metadata the encoder doesn't write

September 24

Problem. An HDR video should carry "mastering display" information: which display it was made on and how bright that display gets. Players and YouTube use it to show the clip correctly.

What we found. NVIDIA's AV1 encoder (driver 610.62) writes no HDR metadata into the video stream, although FFmpeg asks for it. The MP4 file can carry the same information in a label of its own, which YouTube accepts, but players that read only the video stream miss it. HEVC does get it inside the stream, on every keyframe.

Fix. At save time, ShadePlay inserts its own HDR10 metadata block after the header of every AV1 keyframe, and leaves packets that already carry one alone. It was tested by decoding a raw AV1 stream on its own: the decoder found the HDR metadata in the stream alone.

One label is left out on purpose. HDR files can also state the brightest pixel of the content (MaxCLL and MaxFALL), and the first build set them to the panel's peak. But ShadePlay's conversion doesn't clip content to that peak, and on the test PC the SDR white (480 nits) is brighter than the peak (456 nits), so any number would be a guess. Clips say "unknown" instead, and YouTube turned them into HDR without it.

Epochs: any change re-opens the encoder

September 24

Problem. Can a new bitrate or quality setting be applied to the running encoder?

What we found. For HEVC, a live bitrate change silently switched the stream to a different tier and changed its headers, while the file's header kept the old ones. One MP4 track can't change its codec settings halfway through.

Fix. Every encoder start creates an epoch: a fixed set of settings with its own time origin. Any change of codec, frame rate, size, HDR, bit depth or quality re-opens the encoder and starts a new epoch, and a clip never mixes two. A second bug surfaced in the first review: a clip used only the newest epoch, so any re-open cut the replay short. Now a clip uses the epoch that best covers the minute you asked for, and the clip's small sidecar file says when the capture restarted.

Quality you can turn up

September 24, simplified on September 27

Problem. "As detailed as possible", without wasting memory on a still desktop.

What we found.

• A constant bitrate spends its full rate on an idle desktop and still limits fast action.
• Pure constant quality has no ceiling at all: one test produced 1.65 Gb/s and slowed the encoder down.
• Constant quality with a cap does both: an idle desktop costs a few Mb/s, fast action gets constant quality up to the cap, and the cap bounds the memory.
• The HEVC encoder refuses caps above 800 Mb/s.

Fix. Four quality steps. The caps below are for 1440p at 240 fps and scale with resolution and frame rate:

Step	AV1 quality value	HEVC and H.264 value	SDR cap	HDR cap
Lite	32	26	120 Mb/s	150 Mb/s
Default	27	21	250 Mb/s	300 Mb/s
High	23	18	400 Mb/s	500 Mb/s
Insane	19	15	600 Mb/s	750 Mb/s

Lower values mean higher quality. In the first hands-on test, Overwatch averaged about 41 Mb/s at the default step. The simple settings page shows the steps as one slider, from "Smaller files" to "Best quality", with an estimate such as "Up to about 2.1 GB per minute of clip." (Too complicated). A fixed bitrate is in the advanced settings.

A keyframe every second

September 24

Problem. A saved clip must start on a complete picture, a keyframe. How often should one come?

What we found. With a keyframe every second, a 60-second clip starts at most one second early, and video editors never have far to seek. Keyframes cost 1.9-8.6 % of the bitrate at one per second, against 0.9-4.3 % at one every two seconds: cheap, and storage was never the concern.

Fix. One keyframe per second, and the replay is cut only at keyframes.

What recording costs

Measured on September 24 and 27

These are the numbers in the README's Performance section, with their background. All were measured on the test PC, recording 1440p HDR at 240 fps in AV1.

• Processor: about 0.4 % of the whole processor while recording (about 6 % of one core) with the first FFmpeg build, and about 0.9 % (14 % of one core) with ShadePlay's own build, measured on the desktop on September 28. Saving adds a burst in the background: in one test, 8 % of the processor for about 1.3 seconds, while four audio tracks were encoded in parallel.
• Graphics card: the compression runs on the card's separate video engine, about 30 % busy as NVIDIA's own monitoring tool reports it. That figure is an average over the card's two engines, so the one engine a recording uses is busier than it looks. The colour conversion adds about 0.05 ms of graphics work per frame.
• Memory: about 250 MB for the recorder, plus the replay: from a few hundred MB on a quiet desktop to a few GB for a minute of fast action at high quality, and never more than a quarter of your RAM.
• Reliability: recording within half a second of starting, a 60-second clip saved in 3 to 4 seconds in the background, and 3 of about 14,600 frames late over a full minute at 240 fps.

The effect on a game's frame rate has not been measured yet. The maintainer agreed to a test with and without ShadePlay later: "yes let's do the fps test later" (Part 6).

---

---

<- Part 2: how we worked | Contents | Part 4: sound, saving and the app ->