Disable dithering for full range 8 to 16 bit conversions

[kgrabs]

Upsampling from full range 8 bit (0-255) to full range 16 bit (0-65535) is x * 257, which doesn't need dithering.

[kgrabs]

Just realized int16 <-> float16 and float32 -> float16 don't dither by default either (though the latter case it seems to have been done on purpose)

I tested all three cases on some clips with Expr and all cause information loss so I'm gonna go ahead and change it

[Frechdachs]

At least with the python logic, depth would dither for int to f32, which I would explicitly disable.
Zimg might not do it anyway, but I would like it to be obvious when reading the python code, too.

Upsampling from full range 8 bit (0-255) to full range 16 bit (0-65535) is x * 257, which doesn't need dithering.

I don't think that's correct for chroma, since 128 * 257 is no longer centered.

[kgrabs]

I don't think that's correct for chroma, since 128 * 257 is no longer centered.

I hadn't thought of that, I'll have a look at the chroma conversions too

would dither for int to f32

oops

[kgrabs]

It seems like chroma is safe in this case. This test script sttill returns a green clip

clips = []
for x in range(256):
    clips += [core.std.BlankClip(width=5, format=vs.YUV444P8, color=[x]*3)]
clip = core.std.StackHorizontal(clips)
test = clip.resize.Point(range_in=1, range=1, format=clip.format.replace(bits_per_sample=16)).resize.Point(range_in=1, range=1, format=clip.format.replace(bits_per_sample=8))
core.std.Expr([clip,test], 'x y = 0 255 ?').set_output()

Edit: float 16 can handle up to int 10

[kgrabs]

(clip.format.sample_type != sample_type and (curr_depth + 6) > bitdepth)

this conditional should mean that:
A) every conversion from float to integer will dither by default
B) converting from integer to half float will dither by default only if the input depth is above 10
C) converting to float 32 from any format will never dither by default

[kageru]

Please add a comment explaining why this case is excluded (i.e. summarize what has been discussed here). That might prevent confusion in a few months when someone else looks at the code.

[OrangeChannel]

Just realized int16 <-> float16 and float32 -> float16 don't dither by default either (though the latter case it seems to have been done on purpose)

B) converting from integer to half float will dither by default only if the input depth is above 10

Dithering isn't used for conversions to float formats so this was the correct behavior.
Adding a and sample_type == vs.INTEGER to the end of the conditional (outside the or's) would probably clear this up too.

should_dither = (range_in != range  # full -> limited  OR  limited -> full
                 or clip.format.sample_type == vs.FLOAT  # float -> int
                 or (range_in == Range.FULL and not (curr_depth, bitdepth) == (8, 16))  # full, int -> full, int but specifically not for 8 -> 16
                 or curr_depth > bitdepth) and sample_type == vs.INTEGER  # limited, int -> limited, int if downsampling

@Frechdachs @kgrabs does this seem correct?

1. Will always dither for range conversions
2. Will always dither for FLOAT to INT
3. Will dither for full range INT to INT but specifically not for 8 -> 16 upsampling
4. Will dither for anything else only if downsampling

[kgrabs]

float16 only has precision up to about 10-11 bit integer, so converting from 11-16 bit integer to float 16 is downsampling

[OrangeChannel]

But does zimg even dither when converting anything to float, even half?
https://github.com/sekrit-twc/zimg/blob/c41203224c71b5162a53d5238034211d6d1aca44/src/zimg/depth/depth.cpp#L47

VapourSynth docs say no and this seems to also imply that but this is a big codebase that I’m not too interested in learning about.

[kgrabs]

So it seems. Float output likely never dithers

# Higher accuracy than half float
clip = clip.resize.Spline64(format=vs.YUV444P16)

# Create high-precision information (will cause dither to create false positives in diff later in script)
clip = clip.std.BoxBlur(hradius=5,vradius=5)

a = clip.resize.Point(format=vs.YUV444PH, dither_type='error_diffusion')
b = clip.resize.Point(format=vs.YUV444PH, dither_type='none')

# No difference despite dither settings
core.std.Expr([a,b], 'x y = 0 1 ?').set_output()

same thing for format=vs.YUV444PS too, so both high precision int -> lower precision float, and single precision float to half precision float are affected

[OrangeChannel]

Might want to bring this up in zimg’s repo then if you think it should allow dithering to half float, but for now I think the code block I suggested is accurate for when dithering is needed/possible.