The composite filter
src/render/shaders.h, pass 1 (kCleanPS)
A composite signal carries colour and brightness on one wire, and the two leak into each other. That crosstalk shows up as two different artefacts, and they need two different treatments because they live in two different places:
| Artefact | What it is | Where it lives |
|---|---|---|
| Rainbow shimmer | dense brightness detail the decoder mistook for colour | the chroma |
| Dot crawl | the colour subcarrier leaking into brightness | the luma |
Blurring the chroma removes the first and does nothing at all for the second.
Everything on this page happens before any deinterlacer has touched the picture. That is the entire reason pass 1 exists as a separate pass.
A cleanup that runs afterwards has to work out which source line the pixel in
front of it came from, and for a mode that interpolates there is no single
answer. It ends up subtracting a pattern that pixel never carried, which is not
a cleanup but a new artefact. Doing it first removes the question — and it is
also why TemporalDelta() and DotDemodDelta() are written as deltas
computed on the woven pixels, kept apart from the decision of where to apply
them.
This was found the hard way: the filter worked with deinterlacing off and fell apart with it on.
Composite carries colour at roughly a quarter of the bandwidth it carries brightness, so a picture decoded from it has no fine colour detail in it to begin with. Averaging the colour sideways therefore costs nothing real, and it takes the shimmer with it.
float3 soft = sum / n; // horizontal box average
return soft + (Luma(rgb) - Luma(soft)); // brightness put back sharpOnly the colour is softened; the original brightness is added back on top.
The loop is deliberately not unrolled. Every FetchRgbAt expands into a
pixel-format branch and a four-way choice of which frame to read, so seventeen
unrolled copies cost the shader compiler seconds — which the user waits through
at every start. A real loop over the actual radius costs nothing at runtime and
compiles in a fraction of the time.
The Picture tab has a checkbox and a slider, and they hand over to each other.
They are two different bargains, which is why they are two controls:
- Averaging costs nothing and only helps where the picture is standing still.
- Demodulation is the only thing that helps where something is moving, and it costs sharpness.
Behind the single slider they used to share, the first was switched on by the very first step above zero and the whole rest of the travel was the second getting gradually softer — which was exactly what it felt like to use, and left no way to have the free half on its own.
The averaging locks on whenever demodulation is asked for (the checkbox ticks itself and greys out when the slider leaves zero). Without it the demodulator works over the still parts of the picture as well, paying sharpness there for something that is free.
float3 TemporalDelta(int x, int row) {
float3 f0 = FetchRgbFrame(int2(x, row), 0);
…
return (f0 + f1 + f2 + f3) * 0.25 - f0;
}The number came off the card, not out of a textbook. Measured on a PAL-60 console, the mean frame-to-frame difference of a still picture:
| Lag | 1 frame | 2 frames | 3 frames | 4 frames |
|---|---|---|---|---|
| Mean difference | 1.91 | 2.62 | 1.90 | 0.78 |
A clean four-frame cycle. Averaging two frames therefore cancels nothing — it picks two arbitrary points of that cycle. Averaging four covers it exactly, and what is left is the picture.
This half costs no sharpness at all. Analogue noise, being uncorrelated between frames, goes the same way for free.
A console generates its line timing and its subcarrier from the same clock, so the phase walk per frame is a fixed fraction of a turn, and which fraction depends on the colour system rather than on the frame rate:
| Phase per frame | Sequence | Left after 2 frames | after 4 | |
|---|---|---|---|---|
| PAL B/G/I, 50 Hz | 270° | 4 frames | 0.71 | 0.00 |
| PAL-60 | 270° (measured) | 4 frames | 0.71 | 0.00 |
| PAL-M | 90° | 4 frames | 0.71 | 0.00 |
| NTSC-M | 180° | 2 frames | 0.00 | 0.00 |
So a 50 Hz PAL console behaves exactly like the 60 Hz one this was built against — same colour system, same four-frame walk — and NTSC repeats twice as fast, which four frames also covers.
The only difference 50 Hz makes is that four frames span 160 ms instead of 133, so the "nothing is moving here" judgement has to hold for a little longer.
TemporalGate() decides how much of the correction to trust at each pixel.
This is the part that took two attempts. Dot crawl crawls: it is not still, so any detector that simply asks "did this pixel change" calls it movement and switches the filter off exactly where the artefact is. Measured that way, the filter took 48 % off the mild half of the picture and 15 % off the worst one per cent — the wrong way round, and therefore invisible.
What separates them is not amount but scale. The shimmer is a fine pattern a few pixels across; a picture that is really moving moves in broad shapes.
for (int dx = -3; dx <= 3; ++dx) { s0 += LumaFrameAt(…, 0); … } // seven samples
float mhi = max(max(s0, s1), max(s2, s3)) * 0.142857;
float mlo = min(min(s0, s1), min(s2, s3)) * 0.142857;
return 1.0 - saturate((mhi - mlo - 0.02) * 12.0);Seven horizontal samples is a little over two cycles of the colour subcarrier, so
the shimmer averages itself away before it is ever asked about, and real movement
comes through untouched. Then five levels out of 255 of slack (0.02), and full
suppression twenty-odd levels later (× 12).
DotDemodDelta(). This is the slider.
Dot crawl is the subcarrier amplitude modulated by however much colour is at that point in the picture — which is why it only appears at colour edges. Measured on this card, isolating the pattern through its own four-frame cycle and looking at its spectrum, a band of thirty-two bins around the peak holds under half its energy. The modulation smears it far too wide for any narrow filter to catch, and a wide one takes the picture with it.
Synchronous detection does not care about the smearing. Multiplying the line by the carrier and by the same carrier a quarter cycle over moves the pattern down to zero frequency, where a short average recovers exactly how much of it is there; multiplying back up reconstructs it, and it is subtracted.
float w = kTwoPi / gCarrierPeriod;
…
acc += hann * l * cos(w * (x + k)); // in phase
accQ += hann * l * sin(w * (x + k)); // quadrature
…
float pattern = 2.0 * (acc * cos(ph0) + accQ * sin(ph0)) / norm;
return -pattern; // subtracted from all three channelsPicture detail is not tied to the carrier's phase and survives, apart from whatever happened to sit right on the frequency. Subtracting the same amount from all three channels moves brightness and leaves colour alone, which is the half of the picture this belongs to.
Against this card's own frames, at a window of nine samples this removes 81 % of the pattern for 17 % of the horizontal sharpness, where the three-tap notch it replaced managed 67 % for 18 %.
There are two passes over the window. The first exists only to find the local average:
float mean = 0.0, norm = 0.0;
for (…) { mean += hann * Luma(FetchRgbAt(…)); norm += hann; }
mean /= norm;The carrier does not sum to zero over a window of a few samples, so a flat area of picture correlates with it and gets a pattern invented on top of itself. Measured on a constant field of 128, leaving this out produced swings of ninety levels. That was the "picture goes oddly bright" bug.
A raised cosine (Hann) weighting is applied over the window in both passes, so the ends do not ring.
float cycles = 8.2 - gDotNotch * 5.9; // a little over eight, down to just over two
int r = (int)floor(cycles * gCarrierPeriod * 0.5 + 0.5);
r = clamp(r, 2, 12);| Window (samples) | 25 | 15 | 11 | 9 | 7 | 5 |
|---|---|---|---|---|---|---|
| Pattern removed | 42 % | 61 % | 74 % | 81 % | 90 % | 99 % |
| Horizontal sharpness | −7 % | −11 % | −14 % | −17 % | −25 % | −39 % |
In practice a value of 0.1 on the slider is already enough for most material.
The window is measured in cycles of the carrier, and that matters because the two are not the same thing across standards: NTSC's carrier occupies 3.77 samples of a 720 pixel line where PAL's occupies 3.04. A window fixed at a pixel count therefore hands NTSC a fifth fewer cycles to detect with.
Simulated against an amplitude modulated pattern — which is what dot crawl actually is — that cost NTSC 61 % removal at the far end of the slider where PAL got 69 %, and it invented a quarter more pattern of its own doing it. Counted in cycles the two land at 67 % and 72 %.
The same scaling covers capture widths other than 720, which is why
gCarrierPeriod arrives already multiplied by srcW / 720
(VideoRenderer::effectiveCarrierPeriod()).
gCarrierPeriod comes from VideoStandardSubcarrierSamples() — see
Source and signal. Nothing is detected from the picture.
With the carrier assumed wrong — PAL's filter against an NTSC signal or the reverse — removal falls from about 70 % to 34 % at the middle of the slider. That is the reason to care about the Source tab beyond getting a picture at all.
float handled = 0.0;
if (gTemporal > 0.0 && gHistCount >= 3) {
handled = TemporalGate(p.x, p.y) * gTemporal;
rgb += TemporalDelta(p.x, p.y) * handled;
}
if (gDotNotch > 0.0) rgb += DotDemodDelta(p.x, p.y) * (1.0 - handled);Both work out how much pattern is present from the captured pixels. Where the four-frame average has already taken it away, the demodulator would go and subtract it a second time — putting an inverted copy back. That is why turning both up used to make the picture worse than either one alone.
So the temporal half reports how much of the job it did, and the demodulator handles only the rest:
- Still picture — the average does it all, at no cost in sharpness.
- Moving picture — the average steps back and the demodulator takes over.
Not handled. SECAM's colour is frequency modulated on two carriers that alternate line by line, so there is no single frequency to demodulate against. The temporal half still applies, and still works.
This is roughly where composite restoration generally stops. The established offline filters — TComb, DeDot, LUTDeCrawl, Checkmate — are all temporal and say so themselves about moving content. Going further needs motion compensation of the QTGMC sort, which is not a real-time proposition.