AVIF media attachment is broken.

[tateisu]

Steps to reproduce the problem

1. on mastodon.social, v4.1.2+nightly-20230508

2. on Chrome Windows, 113.0.5672.93（Official Build） （64 bit）

3. open Web UI

4. Drag & Drop AVIF file. sample:
   kimono.mirror-vertical.zip

5. NG: thumbnail on post compose area is filled black.

6. post.

7. NG: thumbnail on post on home TL is bluehashed.

8. click thumbnail.

9. NG: browser shows "broken image" icon.

Expected behaviour

thumbnail on post compose area is shown, humbnail on post on home TL is shown, browser can show large image.

Actual behaviour

NG: thumbnail on post compose area is filled black. NG: thumbnail on post on home TL is bluehashed. NG: browser shows "broken image" icon.

Detailed description

AVIF issue is also exists #20834 ,
but in this issue no errors thrown, silently image broken.

this is log from mastodon.juggler.jp (4.1.2)

[paperclip] Trying to link /tmp/RackMultipart20230516-15-5zx8s6.avif to /tmp/2d100ea1d27a6740cd042fb39589e0d320230516-15-pa9ccs.avif
[paperclip] Trying to link /tmp/2d100ea1d27a6740cd042fb39589e0d320230516-15-pa9ccs.avif to /tmp/9e7d5b7e2f670d57b85e4bc12588e16520230516-15-minhe3.avif
Command :: file -b --mime '/tmp/9e7d5b7e2f670d57b85e4bc12588e16520230516-15-minhe3.avif'
[paperclip] Content Type Spoof: Filename 0dce26e97f3c438f.avif (image/avif from Headers, ["image/avif"] from Extension), content type discovered from file command: application/octet-stream. See documentation to allow this combination.
Command :: identify -format %m '/tmp/2d100ea1d27a6740cd042fb39589e0d320230516-15-pa9ccs.avif[0]'
Command :: convert '/tmp/2d100ea1d27a6740cd042fb39589e0d320230516-15-pa9ccs.avif[0]' -auto-orient -resize "1209x1715>" -quality 90 +profile "!icc,*" +set modify-date +set create-date '/tmp/2e58edf9d1dc8f861a1f197dfbab5ed520230516-15-mt8md1'
[paperclip] Trying to link /tmp/2e58edf9d1dc8f861a1f197dfbab5ed520230516-15-mt8md1 to /tmp/bd978beca502d7715a3b85816c4e731920230516-15-ytesig
Command :: identify -format %m '/tmp/bd978beca502d7715a3b85816c4e731920230516-15-ytesig[0]'
Command :: convert '/tmp/bd978beca502d7715a3b85816c4e731920230516-15-ytesig[0]' -auto-orient -resize "403x572>" -quality 90 +profile "!icc,*" +set modify-date +set create-date '/tmp/a7ee9521317e1774055e8fa7969e21dc20230516-15-xgdyqn'
Command :: convert '/tmp/a7ee9521317e1774055e8fa7969e21dc20230516-15-xgdyqn' -depth 8 RGB:-
[paperclip] Trying to link /tmp/a7ee9521317e1774055e8fa7969e21dc20230516-15-xgdyqn to /tmp/5d8c55252da1da996110055a5bce661c20230516-15-m12z9b
[paperclip] saving media_attachments/files/110/376/356/428/173/683/original/0dce26e97f3c438f.avif
[paperclip] saving media_attachments/files/110/376/356/428/173/683/small/0dce26e97f3c438f.avif
method=POST path=/api/v2/media format=html controller=Api::V2::MediaController action=create status=200 duration=3312.56 view=2.45 db=4.89

no errors detected, but image was broken.

Specifications

mastodon.social, v4.1.2+nightly-20230508
Chrome Windows, 113.0.5672.93（Official Build） （64 bit）

[tateisu]

browser screen shots for evidence.

[tateisu]

I test ImageMagick's command that is shown in paperclip logs.

$ docker build -t im6 .
$ docker run -v $PWD:/images -it im6 bash avifConvert.sh

Dockerfile

## from mastodon master branch
#ARG NODE_VERSION="16.20-bullseye-slim"
#FROM ghcr.io/moritzheiber/ruby-jemalloc:3.2.2-slim as ruby

# from mastodon 4.1.2
ARG NODE_VERSION="16.18.1-bullseye-slim"
FROM ghcr.io/moritzheiber/ruby-jemalloc:3.0.6-slim as ruby

# just install imagemagick
RUN apt-get update && apt-get install -y --no-install-recommends imagemagick

WORKDIR /images
VOLUME ["/images"]

avifConvert.sh

#!/bin/bash
set -ex

fileReceived=kimono.mirror-vertical.avif
outOrig=outOrig.avif
outSmall=outSmall.avif

identify -format %m\\n $fileReceived
convert $fileReceived[0] -auto-orient -resize "1209x1715>" -quality 90 +profile "!icc,*" +set modify-date +set create-date $outOrig
identify -format %m\\n $outOrig
convert $outOrig[0] -auto-orient -resize "403x572>" -quality 90 +profile "!icc,*" +set modify-date +set create-date $outSmall
convert $outSmall -depth 8 RGB:- > rgb8.data

kimono.mirror-vertical.avif
kimono.mirror-vertical.zip

result:

• input image and outOrig.avif and outSmall.avif can be shown on Chrome browser.
• rgb8.data can't shown on Chrome browser.

[tateisu]

sample image is downloaded from https://github.com/link-u/avif-sample-images

[tateisu]

paperclip also runs file -b --mime, but it can't recognize AVIF mime type.

file -b --mime kimono.mirror-vertical.avif
=> application/octet-stream; charset=binary

[noellabo]

On mastodon.social, v4.1.2+nightly-20230508, when I try to download an uploaded file, it returns a response with mime-type as image/heic, even though the extension is avif, and actually A binary file in HEIC format is downloaded.

This causes the browser to recognize it as a broken image.

If you download the file by specifying the URL directly, you will get the file.

[noellabo]

On the other hand, if you upload an avif attachment on another server that works as expected (e.g. fedibird.com) and it is received by mastodon.social via ActivityPub, it will also be treated as an avif by mastodon.social. (Expected behavior)

[tateisu]

This causes the browser to recognize it as a broken image.

why? both of actual data and mime type is image/heic. browser should not see extension part in url.

[tateisu]

ImageMagick/ImageMagick#5159
There is a high possibility that ImageMagick is related to the problem that the converted HEIC file is misdetected by AVIF

[Shanesan]

This also happens to apngs. Please also add this to the argument for changing the background processing of images in general because it needs an overhaul.

[ClearlyClaire]

There is a high possibility that ImageMagick is related to the problem that the converted HEIC file is misdetected by AVIF

As far as I can tell, HEIC and AVIF both use the same container format (HEIF) with different codecs (HEVC vs AV1), and ImageMagick handles them through the same code module (called coder in ImageMagick parlance), but it appears that some ImageMagick versions (or versions of libraries it depends on) may not be capable of encoding AVIF files and the shared coder for HEIC and AVIF will instead create HEIC files.

I unfortunately don't think we can make a code change (except ditching ImageMagick for something else) to address the issue. Instead, we need to use an ImageMagick version that handles AVIF correctly.

why? both of actual data and mime type is image/heic. browser should not see extension part in url.

Because most browsers do not support HEIC.

[mcclure]

I unfortunately don't think we can make a code change (except ditching ImageMagick for something else) to address the issue. Instead, we need to use an ImageMagick version that handles AVIF correctly.

I wonder if there is some way the Mastodon software could detect the installed ImageMagick has this problem, at installation or upgrade time say, and alert the admin in some way.

Anyhow, the problem is reproducing on mastodon.social as of today.

https://data.runhello.com/janelle.avif

[mcclure]

Post today's Imagemagick security fix (v4.1.3+nightly-20230706), I tested again with janelle.avif. It still does not work, but the symptom is different. Uploading janelle.avif I get this popup:

"422Validation failed: File has contents that are not what they are reported to be".

(Formatting error in original)

Although this is actually a little more user friendly than the black box, it is still wrong and still not an appropriate failure mode (for example "AVIF not supported." would be a better error).

[c960657]

Fixed by #26264.