Latency

Latency

Measured on a StarTech PEXHDCAP60L: 1080p60 sustained, around 1 ms between a frame arriving from the card and being drawn. That figure is the age of the frame at the moment the draw call is issued, read from the statistics overlay, which computes it from the QPC timestamp taken when the sample was handed over.

Four decisions account for it. Each is a place where the obvious implementation would have added a queue.

1. A renderer filter that does not render

src/capture/frame_sink.h, src/capture/frame_sink.cpp

The stock DirectShow renderers — VMR9, EVR — schedule samples against the graph clock and keep a queue in front of them. That is correct for playback and exactly wrong here. FrameSink is a minimal IBaseFilter with one input pin (SinkPin, implementing IPin and IMemInputPin) that does the opposite.

SinkPin::Receive forwards to FrameSink::OnSample, which:

1. Checks the sample for an attached media type. A mid-stream format change arrives that way; when the width, height or subtype differ from the current format, all three slots are reallocated and the ready and read indices are reset to −1 so nothing half-sized is ever handed out.
2. Picks a write slot under the lock — PickWriteSlotLocked() returns the one index of three that is neither the slot waiting to be picked up (readyIdx_) nor the slot the reader is holding (readIdx_).
3. Copies outside the lock. The chosen slot cannot be touched by anyone else, and only the upstream streaming thread calls Receive, so there is no second writer. The memcpy — the expensive part — therefore never blocks the render thread.
4. Retakes the lock only to publish: if readyIdx_ was still set, that frame never reached the screen and dropped_ is incremented. Then readyIdx_ becomes the new slot, sequence_ and received_ advance, and lastArrivalQpc_ is stamped.
5. Signals frameEvent_, an auto-reset event, so the render loop can sleep on a handle instead of polling.

Receive returns immediately in every case. Frames arriving faster than they can be shown are overwritten, not buffered, so delay cannot accumulate — the dropped counter in the statistics overlay is the visible consequence.

AcquireFrame is the reader half. It moves readyIdx_ into readIdx_, clears readyIdx_, and returns true when the frame is one the reader has not seen. The returned FrameView always describes the frame currently held, even when that is the previous one — so a render pass triggered by something other than a new frame still has something valid to draw.

ReceiveCanBlock returns S_FALSE, and GetMiscFlags reports AM_FILTER_MISC_FLAGS_IS_RENDERER, which is what stops the graph from inserting anything helpful upstream.

Why three slots

Two would deadlock the copy-outside-the-lock trick: with the reader holding one and one waiting to be picked up, there is no third to write into. Three is the minimum that lets the writer always have a free slot without ever waiting for the reader, and there is no benefit to four — a fourth slot is a queue, which is the thing being avoided.

2. No graph clock

FrameSink::SetSyncSource accepts the clock and stores it, but the graph is run without one: src/capture/video_capture.cpp calls IMediaFilter::SetSyncSource(nullptr) before running. With no reference clock, no filter holds a sample back until its presentation time, and the capture filter delivers as soon as the driver hands it something.

The cost is that DirectShow's own timestamps become advisory. Nothing in qBlank uses them for the display path; the recorder derives its timeline from the audio instead (see Recording).

3. Flip-model swap chain, one frame of queue

src/render/display_win32.cpp

• DXGI_SWAP_EFFECT_FLIP_DISCARD
• IDXGIDevice1::SetMaximumFrameLatency(1)
• DXGI_SWAP_CHAIN_FLAG_ALLOW_TEARING where the adapter supports it, present with DXGI_PRESENT_ALLOW_TEARING and a sync interval of 0
• VSync off by default

Two details are worth stating because both were found the hard way:

A sync interval of 0 without the tearing flag still blocks. On a flip-model swap chain, DXGI treats "no vsync" as "no vsync if you have told me you can tear". Without ALLOW_TEARING on both the swap chain and the present call, the present waits for the next vertical blank anyway. display_win32.cpp queries IDXGIFactory5::CheckFeatureSupport(DXGI_FEATURE_PRESENT_ALLOW_TEARING, …) and records the answer in tearingSupported().

SetMaximumFrameLatency applies to the device, not to a swap chain. With one frame of queue and two swap chains — which happened when the settings lived in their own window on the same device — each present waited on the other's frame to retire. That is covered on The settings window. The settings window has its own device now, with its own queue of three (ui_surface_win32.cpp), and the picture's device keeps its queue of one.

Vulkan

src/render/display_vulkan.cpp, src/render/vulkan_device.cpp, src/render/render_passes_vulkan.cpp

Since 5.2 the picture can also be drawn with Vulkan, under Settings → Display → Graphics. Direct3D 11 stays the default and the fallback. The Vulkan backend draws the same passes with the same shaders, compiled to SPIR-V at build time, and keeps the same rules:

• Two swapchain images, or the driver's minimum if it is higher.
• IMMEDIATE when VSync is off, the Vulkan word for tearing. MAILBOX only if the driver has no IMMEDIATE, with a warning in the log. FIFO with VSync.
• One frame in flight. One command buffer with one fence; the next frame's commands wait for the previous submission.
• A longer chain kept to one frame ahead. With more than two images, the acquire first waits up to 100 ms for the previous present, through VK_KHR_present_wait where the device has it.

Measured on the analogue card at 576i, deinterlaced to 50 frames a second. The second field of every frame is drawn 20 ms after the first, which pulls the mean frame age up to about 11 ms:

Build	Mean frame age
5.1.0, Direct3D 11	11.3 ms
5.2, Direct3D 11	11.3 ms
5.2, Vulkan	10.7 ms

Vulkan is a little ahead, but not by enough to make it the default. Direct3D 11 has years of drivers behind it, and both renderers get their shaders compiled by the build. vulkan-1.dll is delay-loaded, so a PC without a Vulkan loader still starts, on Direct3D 11. If Vulkan fails to come up for any reason, the program starts on Direct3D 11 and says why in the log.

4. Format conversion on the GPU

src/render/shaders.h, pass 1 (kConvertPS)

YUY2, UYVY, YVYU, NV12, planar 4:2:0, RGB24 and RGB32 are unpacked in a pixel shader. The bytes go from the capture buffer into a D3D11_USAGE_DYNAMIC texture with Map/Unmap and are decoded on the way out.

The one exception is RGB24, because no texture format holds three bytes a pixel. It goes into the same BGRA texture as RGB32 and gains its alpha on the way in: one four byte read and one four byte write per pixel, the read's fourth byte being the next pixel's blue, which the alpha covers. That brought 1080p from 2.9 ms of frame age to 1.9 ms. Unpacking the three bytes in the shader instead measured 1.3 ms, but it made the clean pass a third larger for everyone, its first compile two seconds longer and the exe 12 KB bigger, for a format that is only ever used when chosen by hand.

Everything in that pass uses texel fetches (Load) rather than samplers, so no hardware filtering can smear anything before it is meant to. Sampling with a filter enabled would interpolate across the chroma subsampling boundary, which is a colour error, not a softness.

Chroma between samples

In 4:2:2 and 4:2:0 a colour sample sits on every even pixel, and the odd pixel lies halfway between two of them. On a digital source the odd pixel takes the midpoint of its two samples. Its colour edges are as sharp as its brightness, and copying the left sample moved each of them half a pixel and turned it into a step. An analogue source keeps the copy: composite colour has no edges that sharp, and every composite filter was measured on it. Only the clean pass's own fetch does this, one branch on a constant, so the deinterlacers' many fetches compile exactly as before.

The render loop

App::Run in src/app_frame.cpp waits on the sink's frame event with a timeout, and draws when one of three things is true:

const bool wokeOnPicture = lastWait_ == WAIT_OBJECT_0 && lastWaitHadEvent_;
const bool fieldDue      = secondFieldPending_ && nowQpc >= secondFieldQpc_;
const double idleFloor   = IdleFloorMs();
if (wokeOnPicture || fieldDue || sinceRenderMs >= idleFloor) {
  lastRenderQpc_ = nowQpc;
  RenderFrame();
}

• wokeOnPicture — a new frame arrived. This is the normal case and the only one that matters for latency.

• fieldDue — a bob-style deinterlacer is showing the second field of the frame it already has, and that field's turn has come. Scheduled from FrameSink::lastArrivalQpc(), not from when the first field was drawn: scheduling from the draw is how a second field ends up due after the next frame has already arrived.

  This condition used to read lastWait_ == WAIT_TIMEOUT && secondFieldPending_, and that was a bug with a very specific footprint. MsgWaitForMultipleObjectsEx returns WAIT_TIMEOUT only when nothing else woke the loop. Open the settings window and there is a steady stream of messages, so the wait returns "input available" every time and the timeout branch never runs. The next arriving picture then takes the haveNewFrame branch, which resets fieldIndex_ — so every second field was silently dropped for as long as the dialog was open, and only on interlaced sources.

  It is the same shape of mistake as the WM_TIMER that could not compete with the drag loop's message flood (see The settings window): a schedule must not be conditional on the message queue being quiet. Asking the clock instead is both correct and cheaper.

• sinceRenderMs >= idleFloor — a floor, so something on screen that has to move keeps moving when no pictures are arriving at all. App::IdleFloorMs() returns 200 ms for the video, which needs no floor of its own — nothing arrived, so there is nothing new to show — and 16 ms whenever the frame contains something that animates by itself: the embedded settings panel, the crop picker, a toast, the volume display.

  The distinction is not whether a signal is present but whether anything on screen has to move. With a source running, its own rate sets the pace and this floor never applies. Without one, the embedded panel used to be drawn by this loop at the video's floor and ran at a measured 4.7 frames a second.

  The wait below is capped to match: a floor is worthless if the loop sleeps for longer than the floor is.

Everything else the message loop wakes up for — mouse moves, timer messages, the settings window's own traffic — deliberately does not trigger a redraw. Before that was true, the pipeline measured 235 draws a second on a source delivering 25, because a second window on screen produces a steady stream of messages and every one of them was being treated as a reason to redraw.

These same three conditions are what the settings window's drag loop applies as well. While a window is being dragged, Windows keeps the thread to itself and this loop does not run at all, so the preview is driven from a message the dialog receives instead — but the decision of whether to draw is identical, down to consuming the same auto-reset capture event with a zero wait. Two different ways in, one rule. See The settings window for the two wrong versions that came before it, both of which were a number.

Nothing waits on a disk

Since 6.0 the render thread waits for no disk, driver or shell. A drive that has gone to sleep takes a second or more to answer; at 240 fps a frame lasts 4 ms.

src/job_thread.h: one thread per area, so work that does not belong together never waits on each other. Jobs in one area run in order. A job that has an answer returns what to do with it, and Collect() runs that on the render thread, once a frame, for one atomic load when there is none. That way toasts, the config and the settings window are only ever touched by the thread that draws them.

Thread	Owner	What
configJobs_	App	writing the settings (serialised on the render thread)
screenshotJobs_	App	reading out, saving and freeing screenshots
folderJobs_	App	the folder to open: made again, or the default
disk_	RecordingControl	start checks, free space, file size
worker_	SourceCapture	opening and closing a recording source (one per source)
deviceJobs_	Settings window	device lists, probes, the card's own audio
setupJobs_	Settings window	the shortcuts, the virtual camera's registration

Opening a link, a folder or a file runs on a short thread of its own: the shell wants a single-threaded apartment, and the render thread is multithreaded because of DirectShow.

Measured against 5.8.0, the frame age is the same within the noise: 0.5 ms on Vulkan, 1.1 to 1.2 ms on D3D11. The gain is in the frames that used to stall, not in the average.

Where the remaining milliseconds are

The overlay's Pipeline row is the span qBlank is answerable for: from the moment a frame lands in the sink to the moment Present returns and the frame belongs to the compositor. Upload, deinterlacing, filters, scaling, drawing, the present queue and — with VSync on — the wait for the flip are all inside it.

It used to be measured to the start of the draw instead, and read around 1 ms. That was not wrong so much as uninformative: the loop sleeps on the capture event, so a draw almost always begins a fraction of a millisecond after the frame arrives, and the number mostly reported that the render thread had woken up. Everything the program actually does with a frame happened after the measurement stopped.

Measured on a GameCube over composite, 480i at 29.97 fps, VSync on:

	Pipeline
Deinterlacing off	0.6 – 1.1 ms, excursions to about 7 as the source's 29.97 Hz drifts against a 60 Hz flip
Bob deinterlacing	8.8 – 9.5 ms, peaks 20 – 27

The second row is not a cost that bob added and the first row was hiding. Bob draws each arrived frame twice, and the second draw is deliberately placed half a frame later so the two fields appear at the times they were captured. That draw shows a frame that has been in the building for about 16.7 ms, and now the number says so. Averaged over both draws, (0.9 + 16.7) / 2 is the 8.8 that turns up.

What is outside it, and what qBlank cannot measure from where it stands:

• the card's own field capture, DMA and driver latency, ahead of the sink — on 480i this is the largest term in the whole chain, because the card delivers woven frames and the first line of one is a full 33.4 ms older than the last
• DWM composition, when not in fullscreen
• the cable, the display's processing, and the panel's own pixel response

Fullscreen removes the second of those. The rest is the card's and the monitor's, and no number this program prints can stand in for them: this is a pipeline delay, not the age of the light.