Issue with uploading HEIC from Samsung phones

[mishari]

Steps to reproduce the problem

Upload a HEIC file from Samsung S22

Expected behaviour

Upload should succeed

Actual behaviour

500 error

Detailed description

The HEIC file causing the error can be found in the attached zip file
20230211_112032.heic.zip

Error message as follows:

[paperclip] An error was received while processing: #<Paperclip::Errors::NotIdentifiedByImageMagickError: Paperclip ::Errors::NotIdentifiedByImageMagickError>
method=POST path=/api/v2/media format=html controller=Api::V2::MediaController action=create status=500 error='IOError: closed stream' duration=17.17 view=0.00 db=1.07
  
IOError (closed stream):
  
app/lib/fast_geometry_parser.rb:5:in `from_file'
lib/paperclip/attachment_extensions.rb:21:in `block in post_process_style'
lib/paperclip/attachment_extensions.rb:20:in `each'
lib/paperclip/attachment_extensions.rb:20:in `inject'
lib/paperclip/attachment_extensions.rb:20:in `post_process_style'
app/controllers/api/v2/media_controller.rb:5:in `create'
app/controllers/concerns/localized.rb:11:in `set_locale'
lib/mastodon/rack_middleware.rb:9:in `call'

Specifications

Mastodon v4.1.0

[ClearlyClaire]

I can reproduce this with any HEIC file, not sure why this is happening, but I'm investigating.

[ClearlyClaire]

So, I'm pretty sure your version of ImageMagick does not handle HEIC, you can check with convert -list format | grep HEIC.

The IOError is caused as a result of the ImageMagick issue, though I'm not too sure how that happens.

[mishari]

The admin says they're using the docker image from https://hub.docker.com/r/tootsuite/mastodon

Does this help?

$ convert -list format | grep HEIC
AVIF* HEIC      rw+   AV1 Image File Format (1.11.0)
HEIC* HEIC      rw+   Apple High efficiency Image Format (1.11.0)

[leahoswald]

We see the same problem here with the same output from convert like in the comment above.

@worker01:~# convert --version
Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org
Copyright: (C) 1999-2021 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP(4.5) 
Delegates (built-in): bzlib djvu fftw fontconfig freetype heic jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib

[smunaut]

Interestingly looking at a machine that can process those HEIC file, I get :

$ convert -list format | grep HEIC
     AVIF  HEIC      rw+   AV1 Image File Format (1.13.0)
     HEIC  HEIC      rw+   High Efficiency Image Format (1.13.0)
     HEIF  HEIC      rw+   High Efficiency Image Format (1.13.0)

(i.e. one more line).

Note this is just my laptop convert, nothing from the docker or anything, but at least it shows recent ImageMagick can actually process those files just fine. So something is wrong with the docker ImageMagick if it can't.

[tateisu]

first, I test ImageMagick.

Dockerfile

## from mastodon master branch
#ARG NODE_VERSION="16.20-bullseye-slim"
#FROM ghcr.io/moritzheiber/ruby-jemalloc:3.2.2-slim as ruby

# from mastodon 4.1.2
ARG NODE_VERSION="16.18.1-bullseye-slim"
FROM ghcr.io/moritzheiber/ruby-jemalloc:3.0.6-slim as ruby

# just install imagemagick
RUN apt-get update && apt-get install -y --no-install-recommends imagemagick

WORKDIR /images
VOLUME ["//images"]

create image.

docker build -t im6 --progress=plain .

test script.

#!/bin/bash
set -ex

IDENTIFY="identify -format %wx%h,%[exif:orientation]\\n"
# "image/jpeg",
$IDENTIFY 20210119_022935.jpg
# "image/png",
$IDENTIFY 8ba37e53b89b158d.png
# "image/gif",
$IDENTIFY 0020-1238049173545.gif
# "image/webp",
$IDENTIFY 6510300-trim.webp

# "image/avif",
$IDENTIFY kimono.mirror-vertical.avif
# "image/heif",
$IDENTIFY dwsample-heif-640.heif
# "image/heic",
$IDENTIFY 20230514_113330.heic

• avif test file produce error or broken image on mastodon.social.
• heic test file produce error HTTP 500 on mastodon.social.

running test.

$ docker run -v $PWD:/images -it im6 bash identifyAll.sh

+ IDENTIFY='identify -format %wx%h,%[exif:orientation]\n'
+ identify -format '%wx%h,%[exif:orientation]\n' 20210119_022935.jpg
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
1152x975,
+ identify -format '%wx%h,%[exif:orientation]\n' 8ba37e53b89b158d.png
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
256x256,
+ identify -format '%wx%h,%[exif:orientation]\n' 0020-1238049173545.gif
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
340x283,
340x283,
340x283,
340x283,
340x283,
340x283,
340x283,
340x283,
+ identify -format '%wx%h,%[exif:orientation]\n' 6510300-trim.webp
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
405x879,
+ identify -format '%wx%h,%[exif:orientation]\n' kimono.mirror-vertical.avif
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
722x1024,
+ identify -format '%wx%h,%[exif:orientation]\n' dwsample-heif-640.heif
identify-im6.q16: unknown image property "%[exif:orientation]" @ warning/property.c/InterpretImageProperties/4051.
640x427,
+ identify -format '%wx%h,%[exif:orientation]\n' 20230514_113330.heic
4000x3000,1

Maybe it's not problem of ImageMagick.

NotIdentifiedByImageMagickError is raised from mastodon's app/lib/fast_geometry_parser.rb , that use FastImage repo. https://github.com/sdsykes/fastimage/blob/master/lib/fastimage.rb#L141
https://github.com/sdsykes/fastimage/blob/master/lib/fastimage.rb#L588

[tateisu]

then I test FastImage

$ bundler init
$ vi Gemfile
$ bundle install --path vendor/bundle
$ bundle exec ruby test.rb 20230514_113330.heic

Gemfile

source "https://rubygems.org"
gem 'fastimage'

test.rb

require "rubygems"
require 'fastimage'
ARGV.each do |a|
  p a,FastImage.size(a)
end

result

"20230514_113330.heic"
nil

it's FastImage 's problem.

sample HEIC file.
20230514_113330.zip

[davidcelis]

This is still occurring and happens whenever trying to upload an image from my iPhone without explicitly choosing to convert it to a JPG from iOS' built-in photo picker. Looking at the actual response, this is what I get back any time I accidentally try uploading an HEIC directly:

Error processing thumbnail for uploaded media

[rafadc]

FYI in case someone ends up googling this thread since sdsykes/fastimage#140 has been merged already once a new release of Fastimage is done this will be fixed in Mastodon. I have git's version in my server and is working flawlessly with Samsun's heic.

[ClearlyClaire]

This should be fixed by #28488.

[Cysioland]

Sadly, the issue reappeared in 4.2.8 when trying to upload a .heic image created by a Samsung S23 Ultra through a computer.

 [paperclip] An error was received while processing: #<Paperclip::Errors::NotIdentifiedByImageMagickError: Paperclip::Errors::NotIdentifiedByImageMagickError>
[333ceb5a-3434-4407-9e8a-76723b9089ac] IOError (closed stream):
[333ceb5a-3434-4407-9e8a-76723b9089ac]   
[333ceb5a-3434-4407-9e8a-76723b9089ac] app/lib/fast_geometry_parser.rb:5:in `from_file'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/paperclip/attachment_extensions.rb:21:in `block in post_process_style'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/paperclip/attachment_extensions.rb:20:in `each'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/paperclip/attachment_extensions.rb:20:in `inject'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/paperclip/attachment_extensions.rb:20:in `post_process_style'
[333ceb5a-3434-4407-9e8a-76723b9089ac] app/controllers/api/v2/media_controller.rb:5:in `create'
[333ceb5a-3434-4407-9e8a-76723b9089ac] app/controllers/concerns/localized.rb:11:in `set_locale'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/mastodon/rack_middleware.rb:9:in `call'
[333ceb5a-3434-4407-9e8a-76723b9089ac] lib/public_file_server_middleware.rb:18:in `call'

[ClearlyClaire]

This has not made its way to a Mastodon release yet, so the issue has not reappeared, it wasn't gone to begin with.

[smunaut]

sad 😢