Scaling and sharpening

Scaling and sharpening

src/render/shaders.h, pass 3 (kScalePS)

The pipeline is split so that scaling never sees a card format. Pass 1 cleans the signal, pass 2 deinterlaces and applies geometry, and only then does pass 3 resample onto the window. Every scaling filter therefore works on real pixels rather than on half-decoded chroma, and is independent of what the card happened to deliver.

The five filters

ScaleFilter in src/config.h: Nearest, Bilinear, Bicubic, Lanczos3, SharpBilinear.

Filter	Taps	Notes
Nearest	1	Point fetch. Exact pixels, hard edges, uneven pixel sizes at non-integer factors
Bilinear	2×2	The hardware sampler
Bicubic	4×4	Catmull-Rom, so it interpolates through its samples rather than approximating them
Lanczos3	6×6	Sharpest of the resampling filters; slight ringing at hard edges
Sharp bilinear	2×2	Nearest up to the integer factor, bilinear only across the remainder

Catmull-Rom rather than Mitchell or B-spline: it passes through its control points, so a pixel that lands exactly on a source pixel comes back unchanged. For a signal made of hard pixel edges that is what you want.

Sharp bilinear is the right answer for pixel art at non-integer window sizes. It gives you the crispness of nearest neighbour with the uneven-pixel artefact confined to a single blend across each boundary, instead of some pixels being drawn twice as wide as others.

Both Catmull-Rom and Lanczos3 are written as real loops with weights computed per tap, not as unrolled tap lists — for the same shader-compile reason as the composite filter's chroma loop.

Integer scaling

AspectMode::Integer in src/config.h. The picture is scaled by the largest whole-number factor that still fits the window, and centred. Combined with nearest neighbour this is the only combination that reproduces the source exactly — every source pixel becomes an n×n block, all blocks the same size.

The other aspect modes are Source (use what the card reports), Force16x9, Force4x3 and Stretch (fill the window, ignore aspect).

Contrast adaptive sharpening

float3 sharp = c + (c * 4.0 - n - s - w - e) * (amount * 0.25);
return clamp(sharp, lo, hi);

A cross-shaped unsharp mask, sampled one source texel away.

The clamp is what makes it adaptive: lo and hi are the minimum and maximum of the centre pixel and its four neighbours, so the result can never leave the range the neighbourhood already spans. Flat areas stay clean and edges cannot ring, which is what separates this from a plain unsharp mask that halos everything.

A consequence worth knowing: a perfect step edge is left alone at any setting. The centre is already the local extreme and the clamp will not push it past its own neighbours. There is nothing to sharpen about an edge that is already hard — the filter works on the bandwidth-limited ramps a composite signal actually delivers.

It samples through a linear sampler rather than by texel fetch, because at this point in the pipeline the coordinates are fractional anyway.

The offset used to come from the wrong size

This line read 1.0 / gDstSize until it was measured. uv runs over the source texture, so an offset of one destination pixel steps srcSize / dstSize source texels — less than one whenever the picture is enlarged.

Both halves then collapse together. The neighbours are sampled between texels, so the unsharp sum shrinks; and the clamp is built from those same samples, so its headroom shrinks with it. The filter faded out in proportion to how large you made the window.

Measured against a bandwidth-limited edge on a 720×576 source, mean change per pixel across the edge zone at full strength:

Window	Factor	1/gDstSize	1/gSrcSize
720×576	1×	0.71 levels	0.71 levels
1440×1152	2×	0.38	1.52
1920×1536	2.7×	0.25	1.55
2880×2304	4×	0.12	1.77
3840×3072	5.3×	0.06	1.90
480×384	0.67×	1.43	0.92

At 1:1 the two agree, which is why the filter was not obviously broken. Beyond that it faded to nothing — and at 4K, 0.06 levels out of 255 is invisible. That is exactly the regime a 240p or 576i picture is normally watched in.

The last row is the other half of the same error: shrinking the picture stepped more than one texel, reaching across detail and over-sharpening.

Counted in source texels the response holds between 0.7 and 1.9 levels across the whole range, which is what a scale-independent sharpener should do.

Sharpening is deliberately not applied to what the recorder, the screenshots or the virtual camera receive. It is a property of viewing at a particular size, not a property of the picture.

Where the picture lands

VideoRenderer::videoRect() reports the rectangle the picture occupies in client pixels, which is what the crop overlay, the mouse hit-testing and the toolbar all work from.

SetTopInset() reserves pixels at the top of the window for the toolbar. The picture is fitted below them rather than drawn underneath, so the bar never covers what you are playing.