Feature request: add tetrahedral interpolation

[pdr0github]

ffmpeg's default lut3d tetrahedral interpolation can produce better results in some cases , such as this example posted by ErazorTT

http://forum.doom9.org/showpost.php?p=1972815&postcount=102
http://forum.doom9.org/showthread.php?t=175552&page=6

[FranceBB]

As of my post in Doom9 from August 2022 https://forum.doom9.org/showthread.php?t=175552&page=6
currently it's not possible to pick which kind of interpolation is used to produce the result in Cube().
I think that currently Cube() is using trilinear which produces worse (yet acceptable) results than tetrahedral interpolation.
Would it be possible to introduce an option in Cube() like String Interpolation where it can have one of the following values Interpolation="Tricubic", Interpolation="Trilinear", Interpolation="Tetrahedral", while leaving the default to the current interpolation method to avoid breaking compatibility?

It could be something as easy as:

Cube("myLUT.cube", fullrange=true, Interpolation="Tetrahedral")

As a simple example on why Tetrahedral can produce better results than Trilinear, take a look at the following script:

#Indexing HLG BT2020 content
LWLibavVideoSource("D:\4K_Ama\05_DI SERVIZIO - EBU UHD HDR COLOR BARS\EBU_HDR_COLOUR_BARS_2160p_v210.mov")
propClearAll()

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#From HLG to BT709 with 16bit precision
Cube("A:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV420 8bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV420", colorspace_op="rgb:709:709:limited=>709:709:709:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

Here's the result (Left interpolation is Trilinear, the current default behavior for Cube, while the Right is Tetrahedral):

As you can see from the graph, the result on the right hand side is much smoother and it's generally what studios expect, which is why getting trilinear interpolation approved by production houses adds unnecessary complications.

[FranceBB]

So, in a nutshell, @sekrit-twc will you be keen to add tetrahedral interpolation as an option for users?

[sekrit-twc]

@FranceBB it has been implemented in af682b6 only for cpu=0. Can you verify?

[FranceBB]

Thank you so much! :D
I'm gonna test on Monday as soon as I'm back in front of a Sony reference monitor. :)
I made a temporary build with your latest changes in the meantime, in case anyone else wants to give it a go as well ;)
vscube.zip

[FranceBB]

Good news.
In the test pattern we tested it on, vscube produced results on par with the ffmpeg lu3d filter when set to tetrahedral.
poisondeathray, tormento and I tested it and it worked fine, well done. :)
By the way, I don't know if you knew, but there were attempts by the AVS community to port vscube to Avisynth. Given the number of dual plugins out there that can work on both VapourSynth and Avisynth, I think it would make sense to have both versions (VS and AVS) in one single repository, yours, so that they're gonna be kept updated at the same time and it's gonna be less hassle for everyone. :)
timecube.zip

[videoh]

"there were attempts by the AVS community to port vscube to Avisynth"

For the record, the AVS support was done by me (Donald Graft = DG). It works perfectly, and it has been successfully used by many for some time now. It's not an "attempt".

sekrit-twc, if you want to absorb it and maintain it going forward, that's fine with me. If not, I am willing to continue tracking your releases in my AVS version.

Regards and thank you for your work.

[sekrit-twc]

Implemented cpu=1...3 in latest commit. Looks correct on my side, but please verify. Took a look at avscube.cpp, and it needs to be updated due to internal API changes.

[videoh]

cpu=1...3 is working for me.

sekrit-twc, did you plan to modify avscube.cpp for the new API or would you like me to do it?

Thank you.

[sekrit-twc]

Please submit an updated avscube in a separate issue.

[FranceBB]

Sorry for the late reply, but I unfortunately was busy at work and Tormento couldn't test 'cause he only has AVX (no AVX2 nor AVX512). Anyway, I grabbed your latest changes and made a new vscube build to test on my Intel Xeon Gold 6238R (which properly supports AVX512). I applied the same LUT with all the cpu modes and I saved the result as tiff, then I compared the deviation with a lattice point graph. According to the preliminary results, they all seem to match.
Very well done and thank you a lot, sekrit, for this. :)

vscube_07072023.zip

[videoh]

Guys, here is a test version for AVS+ support (64-bit):

https://rationalqm.us/cube/AVSCube_test.rar

Will submit code when verified.

[FranceBB]

Thanks Donald :) I'm gonna be testing first thing on Monday to see if the results are consistent.
I'll make two tests: one in v210 lossless with simple tiles which I'll be able to compare with lattice points and one with a real life footage which I'll just compare and playback on a reference monitor.

The latter being a football game from last season:

#Indexing UHD 50p BT2020 HLG 1000 nits live feed
LWLibavVideoSource("Test_HLG.mxf")

#Screw frame properties
propclearall()

#From 4:2:2 10bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion",
use_props=0)

#From HLG to BT709 with 16bit precision
Cube("A:\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=1, fullrange=1)

#From RGB 16bit planar Narrow Range to YUV422 8bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422", colorspace_op="rgb:709:709:limited=>709:709:709:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#Downscaling to FULL HD
SinPowResizeMT(1920, 1080)

#Interlace to 25i
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()

I'll try to report back over the rest of the week and let you guys know how it went, but needless to say everyone is gonna appreciate it. Tormento and I were thrilled about the news and so was everyone else.
If you were both here (Donald and sekrit), I would have said that breakfast and lunch were on me this Monday.

[FranceBB]

Something doesn't feel right.

LWLibavVideoSource("K:\4K_Ama\05_DI SERVIZIO - EBU UHD HDR COLOR BARS\EBU_HDR_COLOUR_BARS_2160p_v210.mov")

propClearAll()

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:std-b67:2020:limited=>rgb:std-b67:2020:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

tetrahedral=Cube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=1, fullrange=1).VideoTek()
trilinear=Cube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", interp=0, fullrange=1).VideoTek()
donald=DGCube("B:\avdb\Server\encoder\encoder\Processors\avs_plugins\LUTs\9c_HLG_BT709_Type3_Display_DownMapping_SuperWhite_nocomp-v1_5.cube", in="full", out="full", lut="full", interp="tetrahedral").VideoTek()

StackVertical(trilinear, tetrahedral, donald)

They produce three VERY different results.

What I don't understand is why Cube() tetrahedral and DGCube() tetrahedral are so much different.

Souce: https://we.tl/t-xqG6XosULK
Cube_Tetrahedral: https://we.tl/t-GLjsVvJ6Ad
DGCube_Tetrahedral: https://we.tl/t-5L8KCVRfmg
Cube_Trilinear: https://we.tl/t-iBhR03kbFP

[FranceBB]

Also, DGCube with Tetrahedral is actually the one that gets it right.
In the Cube version, however, it looks like a bit of both versions is applied. You can see the ripples of the trilinear and also the smoothness of the tetrahedral. Why?

Cube Trilinear (rightly showing ripples):

DGCube Tetrahedral (no ripples, actually correct):

Cube Tetrahedral (something in-between, NOT correct):

Cube Tetrahedral is something in between...? O_O

[videoh]

Good to know I did it right. I'm willing to look into the timecube code but I'll need a link to the cube file you used.

Did you try cpu=0 for timecube tetrahedral? If it is broken there too I'll be able to troubleshoot it. If cpu=0 works but cpu=1...3 does not that will be up to sekrit-twc. I can link the document I used to implement tetrahedral if it will help anyone.

DG

[videoh]

https://rationalqm.us/misc/AMPAS-CLF-LUT-pseudoCode.pdf

[FranceBB]

Very good guess! (About the cpu modes, I mean).
It is indeed an assembly related issue.

The normal C++ code is actually correct.

The SSE4.1 assembly code is also correct.

The AVX2 assembly code is wrong.

The AVX512 assembly code is correct.

So it looks like there's something wrong only in the AVX2 code.

[sekrit-twc]

There was a bug in the AVX2 code that was silently patched. Resync to master.

[videoh]

Thank you. Here is a test build for Avisynth+ support:

https://rationalqm.us/cube/AVSCube_test.rar

[FranceBB]

Thank you, gents!
First tests look good.

C++

SSE4.1

AVX2:

AVX512:

I'll test further as soon as I get back from lunch. :)

[FranceBB]

Hi guys, I noticed that I never replied here.
Anyway, long story short, all tests were successful and the results were signed off by the content providers / production houses.
I'm effectively using it in production.
I've updated the wiki too: http://avisynth.nl/index.php/AVSCube
Thanks sekrit, thanks Donald.
Have a lovely rest of the day. :)