Question: AVIF input with dimensions that are not a multiple of 8

[lovell]

Hello, processing the cosmos_frame01000_yuv420_10bpc_bt2020_hlg_q50.avif sample image with libheif:

• heif_image_handle_get_height( heif_handle ) returns 858
• heif_image_get_height( heif_image, heif_channel_interleaved ) returns 864

I suspect this may be due to the AV1 decoder working with 8x8 blocks internally.

The heif-info tool appears to use the former (from the ispe box?):

$ heif-info cosmos_frame01000_yuv420_10bpc_bt2020_hlg_q50.avif
MIME type: unknown
image: 2048x858 (id=1), primary
  color profile: nclx
  alpha channel: no
  depth channel: no

The heif-convert tool appears to use the latter:

$ heif-convert cosmos_frame01000_yuv420_10bpc_bt2020_hlg_q50.avif out.png && file out.png 
File contains 1 images
Written to out.png
out.png: PNG image data, 2048 x 864, 16-bit/color RGB, non-interlaced

Note the pixel "stretching" at the bottom of this output from row 859 onwards:

Is this difference in dimensions the expected behaviour of libheif's public API and CLI tools?

[farindk]

Hello.
Right, this AVIF seems to store a different dimension in the ispe box than the actual compressed image has. I will have to dive into the AVIF / AV1 standards to check whether that is correct. Obviously, the image in AV1 format is extended at the bottom to get a multiple of 8, 16, or 32.
Either the crop window in the AV1 is not specified, or a crop transformation is missing in the AVIF, or AV1 implicitly assumes that the image is cropped to the ispe size.

[novomesk]

I am facing the similar situation in libheif-based GIMP plug-in.

heif-info fox.profile2.12bpc.yuv422.monochrome.odd-width.odd-height.avif 
MIME type: image/avif
image: 1203x799 (id=1), primary
  color profile: no
  alpha channel: no
  depth channel: no

But in GIMP I have 1208x800 and the green line on the right:

I think it should be cropped to 1203x799, the values returned by heif_image_handle_get_width ( handle ), heif_image_handle_get_height ( handle )

Small I perform the crop in my code or libheif will do it in the future?

[farindk]

Actually, I consider this a bug in the encoder and/or specification. The ispe box is specified as a purely descriptive box that should not be used to trigger cropping, as that should be specified in a 'clap'. The ispe box should simply match output image size, but apparently the encoder specified the wrong AV1 image size. On the other hand, I am aware that e.g. h265 cannot specify odd image sizes with 4:2:0 as the image size is specified in chroma units (see #191).
Hence. libheif probably has to handle 'wrong' input by cropping according to the ispe.

[novomesk]

For example I saved 100x100 AVIF via libheif.
libavif based applications opens it as 100x100 image as expected
But when I open the libheif-saved file via libheif again I have suddenly 104x104.

I understand that encoder(libaom) works like that. If the difference is below 8 pixels we can consider it as feature.

[farindk]

I tried to add writing a 'clap' box that will remove the padded border. But the problem is that in order to write the 'clap' box, I need to know the size of the encoded image. However, that is not easily available. I've asked the rav1e whether there is a ways to get this without parsing the AV1 stream: xiph/rav1e#2477

[farindk]

For AVIF this should now be resolved with 07b7c81.
I'll still have to add a solution for HEIF, because h265 cannot store odd sized images.

[adityapatadia]

Awesome, it would be great if you can do a bug-fix release.

[farindk]

2f91ec1 should now handle odd image sizes well for AVIF and HEIF. It also should handle well HEIF image sizes < 16 pixels.
I'll leave this open until people have tested it and confirm that it's working.

[adityapatadia]

I can test it. Should I compile develop branch and deploy it?

[farindk]

Yes, please. I'd like to merge this soon to 'master' and tag a new release.

[novomesk]

Loading AVIF with any dimensions works for me now. Just when saving it is cropped to a multiple of 2.

[farindk]

@novomesk Right, that was still old code in the AOM encoder plugin. I've changed it in the commit above. The rav1e encoder plugin didn't do the rounding.

[lovell]

@farindk I've tested the latest code on the develop branch and can see it is now cropping output to the ispe box dimensions. Thank you very much for investigating this and all your work helping to maintain libheif. I'll leave this issue open for now to allow others to confirm.

[farindk]

@lovell Thanks. BTW: it is not cropping according to the ispe box, but there was a bug reading the displayed image resolution from libaom. When saving, it is now writing proper clap boxes for heif as h.265 cannot have odd image sizes and x265 does not support images <16 pixels width/height.

[adityapatadia]

I have tried it but I get following error when I compile vips with it and try to encode image:

heif_image.cc:146: bool heif::HeifPixelImage::add_plane(heif_channel, int, int, int): Assertion `bit_depth <= 16' failed

[farindk]

@adityapatadia Yes, sorry. During some apparently simple code cleanup, I added a wrong assert(), not related to this issue. Should be fixed now.