Skip to content

Colour range and matrix

NuclearMeltdown edited this page Sep 8, 2026 · 6 revisions

Colour range and matrix

src/render/video_renderer.cpp, VideoRenderer::AnalyzeRange()

Both default to automatic. The matrix is worked out in three steps; the range is measured from the pixels, and that is the part worth explaining.

The matrix

Automatic asks three questions in order, in VideoRenderer::Draw():

  1. Does the media type say? DXVA_ExtendedFormat::VideoTransferMatrix in a VIDEOINFOHEADER2 — 1 is BT.709, 2 is BT.601. Believed when present.
  2. Is the subtype HDYC? That is UYVY carrying BT.709 by definition.
  3. Otherwise, height decides. croppedHeight_ >= 720 means BT.709, below it BT.601. This is the case that actually fires on most cards.

Note that it is the cropped height, not the capture height — a 480i console letterboxed inside a 1080-line capture is still standard definition, and cropping to it is what tells the renderer so.

The setting can be forced either way.

The range: why it cannot be read

What a capture card hands over is limited range (16–235) or full range (0–255) depending on what the source is sending. It is a console setting, not a property of the pixel format the card was asked for: a console set to full range delivers full range whether the card produces NV12 or RGB32.

DirectShow has a place to say which one it is — DXVA_ExtendedFormat::NominalRange in a VIDEOINFOHEADER2 — and plenty of cards, this one included, attach no colour description at all. There is nothing to read, so the answer has to come from the pixels.

The measurement

AnalyzeLevels() runs on the raw capture buffer before upload, on the CPU, and costs almost nothing because it samples rather than scans.

Where the luma is. LumaLayout() returns an offset and a step into the buffer for each packed format:

Format Offset Step
YUY2, YVYU 0 2
UYVY 1 2
NV12, planar 4:2:0 0 1
P010 1 2
RGB24 1 3
RGB32 1 4

For RGB there is no luma channel, so green stands in for it: it carries most of the brightness and it costs one read per pixel instead of three. P010 parks its ten bits at the top of sixteen, so the upper byte is the upper eight — the same scale as everywhere else, and the 16/235 boundaries hold unchanged.

Until 3.6.1 this function had a second copy of that table, and the copy had drifted. It walked RGB byte by byte, which in RGB32 made every fourth sample the unused fourth channel. That byte is zero, so a quarter of the evidence landed in the "below 16" column before the signal was looked at.

Where in the frame. Between the detected picture bounds, and this matters more than anything else in the measurement. Blanking is not picture: everything outside the active line digitises below black — on a PEXHDCAP60L it pins to exactly 0 in RGB and exactly 1 in Y, over 1,2 million samples without one outlier — and on a PAL capture it is an eighth of the frame. Counted along with the picture it answers the question by itself. The same capture reads 13,1 % below 16 taken whole and 0,22 % taken inside the picture, against a threshold of 0,2 %.

The bounds are ready in time. They settle after five windows of nine frames, so 45 frames, and no range verdict is due before 120. Until they are, the fallback is a sixteenth off each side, which clears the 22- and 26-point border this card leaves without knowing anything about it. If either edge would leave less than 32 points the whole frame is used.

How it samples. Every third frame (kRangeSampleEvery = 3), 8192 samples per frame (kRangeSamplesPerFrame), over 40 analysed frames (kRangeFramesWanted = 40) — roughly four seconds at 30 fps.

The sample stride is forced odd, and each row starts one further along than the last:

size_t stride = (size_t)(x1 - x0) * (size_t)(y1 - y0) / kRangeSamplesPerFrame;
if (stride < 1) stride = 1;
if (stride % 2 == 0) ++stride;
...
for (int x = x0 + (int)((size_t)y % stride); x < x1; x += (int)stride)

A stride that divides the row length evenly lands on the same column every row, which on a UI-heavy picture reads one vertical stripe and calls it the whole frame. A prime-ish stride on a shifting start walks the samples across.

What it counts. Minimum, maximum, samples below 16, samples above 235. The log line names the window it used, so a disagreement is one line away from being settled:

Wertebereich erkannt: voll 0-255 (min 0, max 255, 3.305 % unter 16, 330830 Proben aus x 24..691, y 21..558)

Two ways the evidence is worthless

Both are checked before any verdict:

if (rangeMin_ > 40 || (rangeMax_ - rangeMin_) < 64) { /* discard, start over */ }
  • No dark pixels at all (min > 40). There is nothing to tell the two apart: limited range piles its blacks at exactly 16, full range goes below it. With no blacks in the picture, neither statement can be checked.
  • No contrast at all (max − min < 64). A console asleep, a card between signals. An entirely black frame reads as 100 % below 16 and would otherwise be written down as a confident verdict of full range on no evidence whatsoever.

In both cases everything is thrown away and the measurement starts fresh, so whatever appears later is judged on its own. Until something decides, Draw() falls back to the rule of thumb for the pixel format — limited for YUV, full for RGB.

The verdict

const double below = (double)rangeBelow16_ / (double)rangeSamples_;
rangeVerdict_ = below > 0.002 ? RangeVerdict::Full : RangeVerdict::Limited;

The black end decides, and only the black end. Values above 235 prove nothing: limited-range signals are allowed to carry superwhites and plenty of sources do. Two tenths of one per cent of samples below 16 is enough, because in a genuine limited-range signal there should be none at all.

Why it is frozen

Once decided, the verdict is not revisited until the format changes or ResetAnalysis() is called. A verdict that keeps changing with the content would be worse than a wrong one held steady — the picture would visibly shift brightness as scenes came and went.

ResetAnalysis() is called on a format change and on a crossbar switch. The latter is the important one: a crossbar switch puts a different signal on the same pins without the media type changing, so nothing measured about the old signal survives it.

Thawing it by hand

There is a third case, and no automatic reaches it: an option in the card's own driver. Moving a card between full and limited output shifts every black level in the picture and leaves the media type exactly as it was, so the frozen verdict goes on describing rules that no longer apply.

Measure again does it on demand — beside the verdict in Settings → Picture, in the right-click menu, and on F6. It calls ResetRangeAnalysis(), deliberately not ResetAnalysis(): interlacing and the picture bounds are measured in the same pass, they depend on nothing outside qBlank, and discarding them as well would buy a re-detected deinterlacer and a re-detected crop — both plainly visible — in exchange for nothing. Since 3.6.1 there is a second reason: the range measurement samples between those bounds, so keeping them means a re-measure starts with its window already correct rather than on the fallback inset.

The measurement needs forty frames, during which the display says what it said before, so the action is confirmed with a toast rather than left to look like nothing happened.

What the card's own option does, measured

Worth knowing before reaching for it, because the label is ambiguous and the answer is not reversible.

On a PEXHDCAP60L the driver's colour-range setting describes its input, not its output. Full passes the decoder through; Limited means "the source is 16–235, stretch it to 0–255", and the stretch happens before anything qBlank can see. Measured on a PAL composite source, 90 frames, 37 million samples per format:

RGB32 (green) YUY2 (Y)
min / max 0 / 255 1 / 254
occupied codes 256 of 256 254 of 256
gaps between min and max 0 0
blanking border exactly 0 exactly 1

No comb, so the card is not stretching an already quantised 16–235 onto 0–255 — the gain change happens before the ADC and every code is reachable. What it does lose is the two ends. In the picture interior the blacks pile on the floor (bin 1 counts five times its neighbours) and the whites pile on the ceiling (bin 254 counts five times its neighbours), and a pile-up is clipping: those values were distinct going in and are one value coming out. There is nothing downstream that puts them back, in qBlank or anywhere else.

qBlank on Limited while the driver is on Limited expands twice, which is worse again. The setting to leave the driver on is Full.

The numbers behind it

Under the verdict, while the range is on automatic, stand the figures it fell out of: lowest and highest sample, the share below 16 and the share above 235, and how many samples that was. VideoRenderer::rangeNumbers() returns them without a lock, the same way the verdict itself is read — written while measuring, read while drawing, and worst case a set of figures from two consecutive frames, which for a readout does not matter.

They are there because the verdict folds two cases into one word. Black at 0 and black at 16 both read as detected, and while setting a card up those are exactly the two that have to be told apart. The same figures go to the log, but once, in the moment they settled.

What the shader does with it

Pass 1 (kConvertPS in src/render/shaders.h) reads the decision out of its constant buffer as three numbers — yOffset, yScale, cScale. Limited range sets them to 16/255, 255/219 and 255/224; full range sets them to 0, 1 and 1, so the same shader code runs either way with no branch. The matrix follows as four coefficients from MatrixCoefficients().

The statistics overlay at detail level 2 shows what was decided, so a wrong verdict is visible rather than merely felt.

Related

The same "is there anything in this frame" question, asked continuously rather than once, is what drives the empty state — see Signal detection.

Clone this wiki locally