Add support for TIFF YCbCr subsampled planar images

[ccodere]

Is your feature request related to a use case or a problem you are working on? Please describe.
A common issue when doing video encoding is that we need to prepare the images in the YCbCr color space. Encoding is usually in planar format, such as 4:2:2 or 4:2:0 or event 4:1:0 subsampled with different matrixes with studio or full range. I do not know of any image format that permits to support different subsampling, planar data, custom matrixes and subsampling position information and compression except for the TIFF format. Hence I have my image encoder when doing YCbCr image space encodes directly in planar format, not in packed format with full LZW and predictor information. Furthermore, when decoding images from movies, we can dump the images in a TIFF file, with one image per frame, this is what my tool does.

Today, I have not seen ANY public support by any tool for planar 4:2:0/4:1:0/4:2:2 TIFF compressed formats (i may be wrong, but GIMP, your tool and XnView does not support it).

Describe the solution you'd like
Support for reading TIFF YCbCr planar images which are subsampled. The TIFF specification is quite unclear how to manage the rowsPerStrips in that case, I just know that each plane should be in 1 strip. My current implementation uses the subsampling field to divide rowsPerStrip for the chroma planes. Of course, I am unsure if this is the correct approach and my metadata could be wrong, and i will propose the same to the libTIFF mailing list, and maybe you already support subsampled planar images, but in that case would need your guidance, what is wrong with my image format that I am attaching.

Describe alternatives you've considered
TIFF YCbCr Packed formats of course will work, but incurs a minor performance hit to prepare for encoding for videos.

Additional context
TIFFInfo for my image:
TIFF Directory at offset 0x112cec (1125612)
Image Width: 1392 Image Length: 968
Resolution: 73, 73 pixels/inch
Bits/Sample: 8
Compression Scheme: LZW
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Rows/Strip: 968
Planar Configuration: separate image planes
DateTime: 2022:05:04 00:11:44
YCbCrCoefficients: 0.299000,0.587000,0.114000
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255
Predictor: horizontal differencing 2 (0x2)
marbles_cropped_II_yuv420p8_jpeg_Planar_LZW.zip

[haraldk]

Hi @ccodere

Can libTIFF read your image? According to LibTIFF TIFF 6.0 Specification Coverage it should be capable of reading YCbCr (Class Y) type images with planar configuration 2 and subsampling (and using LZW compression). However, the page isn't clear on the subject of using planar and subsampling together.

This is quite a narrow use case, but I'll look into it, and see if it makes sense! 😀

[haraldk]

PS: Can you also attach the same image without LZW and predictor? Having the data and lengths uncompressed makes things a bit easier to reason about. Thanks!

[ccodere]

I do not think libtiff supports it, since GIMP probably uses libtiff. But i have asked on the mailing list, and it may even be possible that the standard does not permit this type of configuration. In all cases you can find the uncompressed version of my generated file attached.

marbles_cropped_MM_yuv420p8_jpeg_Planar_Uncompressed.zip

In all cases, its just a nice to have to support, my tool will still support it on my side...

TIFFInfo:
TIFF Directory at offset 0x1ed7d2 (2021330)
Image Width: 1392 Image Length: 968
Resolution: 73, 73 pixels/inch
Bits/Sample: 8
Compression Scheme: None
Photometric Interpretation: YCbCr
YCbCr Subsampling: 2, 2
Samples/Pixel: 3
Rows/Strip: 968
Planar Configuration: separate image planes
DateTime: 2022:05:12 02:01:43
YCbCrCoefficients: 0.299000,0.587000,0.114000
Reference Black/White:
0: 0 255
1: 128 255
2: 128 255

[haraldk]

Thanks!

With my newly created planar YCbCr upsampler stream and planar YCbCr-RGB conversion, the uncompressed version reads fine. It looks just like the resources/tiff/marbles.tif reference image (without the black border).

The compressed version looks very strange, with every second row alternating and some serious color clipping. So I'm not sure about that one. But it's likely just my predictor that doesn't understand the subsampling... Will report back. 😀

[haraldk]

Yup. With the predictor taking the subsampled plane width into consideration, both images look the same! 👍🏻
Will need to clean up the code a bit and add some test cases before adding the code to the library.

But the main conclusion: I think your images are correct as per the spec (even though not much supported in software).

[ccodere]

Thanks for the quick response and to have confirmed my image format makes sense, and am happy i will have support by one library/software on internet!! :) Let me know if you ever need other test images.

My main use-cases are: Uncompressed and LZW compressed (with and without predictor), 4:2:0, 4:2:2 and 4:1:0 with full range and partial range matrixes. (My partial range matrixes starts at 16 instead of 15 though for ReferenceBlackWhite).

p.s: And yes, it is the original marbles image indeed that i cropped to be divisible by four, as required by planar formats and video encoders...
Thanks again!

[haraldk]

Let me know if you ever need other test images.

If you could create some smaller images (140x60?), that would be great, as it would speed up the tests.

I'm not sure if I do the YCbCr -> RGB 100% correct with regards to coefficients and reference black/white, but the images looks reasonable.

[ccodere]

Regarding YCbCr conversion, actually if you look at the TV Studio matrixes it starts at 16, not 15, as defined in TIFF for the luma matrixes for ReferenceBlackWhite. Even though i am not expert, but i feel that the upscale conversion formula defined in the TIFF specification is lacking accuracy, but i did not do deep tests. The 127 as coding range for CbCr is the thing that bugs me, but i cannot pinpoint why, i did some tests a while back, and could not get back to same values as if you were using the scaled matrixes. In all cases, this is a minor detail, and is ok for me, and you will notice my ReferenceBlackWhite is 16 luma black for TV range values in my sample.

Here is a testsuit with all subsampling formats i support and reference image in BMP format, hope it helps.
tiff_planar_ycbcr_testsuit.zip

[haraldk]

Thank you!

Fixed in master. Available in the latest snapshot for now (you should be able to use the imageio-tiff-3.9.SNAPSHOT with the 3.8.2 version). Will be included in the next release.