Deinterlacing

Deinterlacing

src/render/shaders.h (pass 2, kConvertPS), src/render/video_renderer.cpp (AnalyzeInterlace)

Detection

Whether the source is interlaced is measured rather than believed. A media type is entitled to say so and frequently does not — a plain VIDEOINFOHEADER has nowhere to put it, and cards that use VIDEOINFOHEADER2 often leave the interlace flags at zero. A 480i console can arrive described as progressive.

AnalyzeInterlace() asks two questions of the same three row reads. It walks odd y only, so rowA = y−1, rowM = y, rowB = y+1: rowA/rowM are the two rows of one pair, and rowM/rowB straddle the boundary into the next. Sixty-four columns across the width, every third frame (kCombSampleEvery = 3), over twenty analysed frames (kCombFramesWanted = 20).

Only the luma plane is looked at. Chroma is subsampled vertically in half these formats, which would blur away the very thing being measured.

Question 1: do the two rows of a pair hold the same line?

A 240p or 288p console packed into a 480 or 576 line frame arrives that way — the card takes two consecutive progressive pictures and interleaves them, so rows 2k and 2k+1 are the same picture line at two different moments.

The two mean absolute differences are accumulated separately:

inner += |rowA − rowM|     within a pair
outer += |rowM − rowB|     across the pair boundary

On a co-sited source one of those is near zero and the other is not:

if (hi >= kDoubleFloor && lo * kDoubleRatio < hi) { coSitedFields_ = true; … }

with kDoubleFloor = 4.0 and kDoubleRatio = 3.0 — the larger difference has to be at least 4 levels per line pair, and at least three times the smaller. Which of the two is smaller gives coSitedPhase_ (0 or 1); either phase is legitimate, because which row of a pair comes first is a property of where the capture happened to start, not of the signal.

Such a source is interlaced in the sense that matters: the two fields are half a field apart in time and comb on anything that moves. What it does not need is spatial reconstruction — nothing is missing.

The measurement is taken in buffer order and used in picture order. Those differ for bottom-up layouts, but only by a mirror, and mirroring an even number of rows maps a pair starting on an even row to a pair starting on an even row. Capture heights are even, so the phase carries over unchanged.

Question 2: is there combing?

Asked only when the first question said no.

const int comb   = (lm - la) * (lm - lb);
const int detail = abs(la - lb);
if (comb > 900 + detail * 12) ++hits;

The product is positive exactly when the middle line lies outside the range its two neighbours span — bright, dark, bright — and near zero on a smooth gradient. The detail term raises the bar in proportion to genuine vertical detail, so a finely striped but static image is not read as combing.

Combing is judged per frame, not pooled:

if ((double)hits / (double)samples > kCombThreshold) ++combFrameHits_;   // 0.030
…
if (combFrameHits_ >= kCombFramesNeeded) interlaceVerdict_ = Interlaced;  // 2

Combing lives where something moved, so it is concentrated in the few frames that had movement in them. Spreading those hits across a window full of still ones is exactly how a real interlaced source gets called progressive.

What latches and what does not

• Interlaced latches. Once combing has been seen there is no reason to doubt it again, and AnalyzeInterlace() returns immediately on every later call.
• Co-sited latches, for the same reason — it is structural.
• Progressive never latches. A paused game or a title screen has nothing moving in it, and calling that progressive and sticking to it would leave the deinterlacer off for the rest of the session. The verdict is reported once, then the window resets and the measurement keeps running.

Field order

Asked of the media type first (AM_INTERLACE_FIELD_PATTERN_FIELD1FIRST and friends) and overridable in the Picture tab. Guessing wrong does not soften the picture — it makes it judder, because the fields are shown in the wrong temporal order and the picture steps backwards every second field.

The five modes

gDeinterlaceMode in the pass-2 constant buffer. Everything below reads the cleaned picture from pass 1, so the deinterlacers know nothing about pixel formats or composite artefacts — see The composite filter for why that ordering is not optional.

The figures are vertical movement between consecutive frames, measured on a 480i console:

Mode	gDeinterlaceMode	Vertical movement
Off (weave)	0	none	combing on anything that moves
Bob	1	1.0 line	full rate, no latency, no interpolation
Bob interpolated	2	0.56	the alternation between sharp and interpolated lines, not the picture moving
Motion adaptive	3	0.005	weaves what is still, interpolates what is not
Edge directed	4	0.69	follows edges; meant for pixel art, weakest on composite
YADIF	5	0.002	best quality; keeps one frame in memory

The co-sited shortcut

When gCoSitedPhase >= 0, every mode collapses to the same and only correct answer:

int base = ((row - gCoSitedPhase) & ~1) + gCoSitedPhase;
rgb = FetchRgbAt(int2(srcX, clamp(base + gFieldIndex, rowTop, rowBottom)));

Take the line belonging to the field being shown and use it for both rows of its pair. No interpolation, no ghosting, and — because the two fields are not offset from each other — no vertical step. On such a source, bob is the sharpest option available and costs nothing.

Bob (mode 1)

float t = (float(row) - float(gFieldIndex)) * 0.5;
int r = int(floor(t)) * 2 + gFieldIndex;

floor, not round. Rounding lands on exactly .5 for every second output row, and which way it goes depends on the field — so the whole picture shifts by a line each time the field changes, sixty times a second. Flooring is half a line low for both fields equally, which is invisible; a line of difference between fields is not.

That still leaves the measured 1.0 line of movement, and it is unavoidable in this mode: the two fields of a real interlaced signal sit half a line apart, a distance nearest-neighbour doubling cannot represent. Either the block boundaries move or the content does, and the measured line is the former.

Bob interpolated (mode 2)

The same t, but linearly interpolated between the field's own rows either side of it. The 0.56 figure is the alternation between sharp and interpolated lines, not the picture moving.

Motion adaptive (mode 3)

Rows belonging to the field being shown are used exactly as captured. Rows between them come from the other field, half a field time away, and are only trustworthy where nothing moved — so the mode measures how much moved and crossfades.

float da = lw - la;
float db = lw - lb;
float comb = (da * db > 0.0) ? min(abs(da), abs(db)) : 0.0;
motion += saturate((comb - 0.003) * 60.0);

Two things here were arrived at by measurement:

min(|da|, |db|) rather than da × db. The obvious form multiplies the two deviations together, which is quadratic in contrast, and on soft anti-aliased material it collapses: against this card the typical comb value came out at 0.00008 with the threshold at 0.00422 — fifty times too high. The mode wove essentially everywhere and left the picture interlaced wherever anything moved. Hard pixel edges cleared it and 3D rendering never did. Taking the smaller deviation is linear in contrast and works on both.

It also needs no allowance for genuine vertical detail: on a real edge the middle line lies between its neighbours, so the deviations have opposite signs and the expression reads zero by construction.

Three columns, not one. A composite signal carries enough noise that a single column flickers between weave and interpolate from frame to frame, and that flicker is more visible than the combing it was trying to remove.

Sweeping the two remaining numbers against real frames, with movement established independently by comparing the same field across frames, 0.003 and 60 reach 47 % of what genuinely moves while touching 4.7 % of what does not.

Where it does interpolate, it uses EdgeDirected rather than a vertical average — straight averaging was leaving jagged steps on things moving quickly across the picture.

Edge directed (mode 4)

EdgeDirected() looks for the direction in which the line above and the line below agree, instead of straight down the column. On a diagonal edge the vertical average produces a staircase; following the edge does not.

Three things make it usable on an analogue signal:

• Scored over three pixels, not one. A single pair matches by coincidence all over a flat or noisy picture — dozens of directions come out near zero and the winner is essentially arbitrary. That was the speckle this mode produced originally.
• Scored on LumaSmooth, a three-tap horizontal average. An unsmoothed score follows the composite shimmer's own diagonal instead of the picture's: the shimmer's dots line up at an angle and the search reports that as an edge. The value of a pixel still comes from the pixel; only the decision about which direction to take it from is made on the smoothed version.
• A slant penalty of d × 0.018 per step, so a genuinely vertical edge is not talked out of being vertical by a marginally better diagonal match.

Finally the result is clamped into the range its own two vertical neighbours span. A pixel interpolated from some diagonal outside that range did not come from this edge — it came from wherever the search wandered off to. That clamp is what removes the gross misses rather than merely making them rarer.

Search range is ±3 pixels of slant.

YADIF (mode 5)

Yadif() predicts the missing line spatially with YadifSpatial() — the same three-pixel smoothed scoring, but widening to two pixels of slant only when one pixel already looked better than straight down — then refuses to let that prediction stray further from the temporal evidence than the surrounding lines say it plausibly could.

float3 tPrev = FetchRgbPrev(int2(x, row));   // missing line, one frame ago
float3 tCur  = FetchRgbAt(int2(x, row));     // missing line, woven into this frame
float3 d = (tPrev + tCur) * 0.5;
…
return clamp(spatial, d - diff, d + diff);

diff starts as how much this part of the picture is moving — measured on the lines that exist as well as on the line being guessed — and is then widened by the local vertical gradient two lines out in each direction (b and f), which is the standard YADIF spatial check.

This implementation is one-sided. Textbook YADIF also looks at the frame after the current one, which would mean holding every frame back until its successor arrives. In a viewer whose entire point is latency that trade is not worth making. Where the one-sided version costs accuracy, it falls back towards the spatial prediction, which is the safe direction to be wrong in.

YADIF is the only mode that needs history — DeinterlaceNeedsHistory() in src/config.h returns true only for it — and therefore the only one that costs anything to have switched on, in the form of one extra frame kept in GPU memory and copied each frame.