Support for HEIF

[pythonicrubyist]

Do you have any plans to support HEIF image format, recently introduced by apple?

[wiredfool]

I would welcome support, but I'm not aware of any appropriately licensed HEIF or HEVC libraries to do encoding/decoding.

[CounterPillow]

The reference HEVC encoder/decoder is licensed as BSD 3-clause (but obviously extremely slow and tuned for PSNR), the openHEVC decoder is licensed as LGPL2.1.

Patent licenses are the bigger issue; the current patent licensing situation with 3 competing pools means that it'd be practically impossible for any FOSS project to produce binaries including a HEVC decoder which don't violate some patents.

So even if someone were to do the work to implement it, you wouldn't find it in most Pillow builds, because nobody wants to get harassed by the patent trolls.

[wiredfool]

The HEIF container decoder is also not a free license: https://github.com/nokiatech/heif/blob/master/LICENSE.TXT .

[CounterPillow]

There's an initial HEIF support patch for FFmpeg floating around on their mailing list, but as the post itself says, it's early work and the whole format is batshit insane. The container is based off of MP4 for a start.

I don't think it'll be possible to support HEIF in Pillow in a sane manner based on the container and codec complexity alone, and even if someone were to make a demuxer+decoder in one library with a license Pillow could use, you wouldn't be able to build a product on it for around 30 more years, assuming the patents remain unlicenseable for anyone without big money lawyers.

[wiredfool]

I'm definitely -1 on batshit insane.

I think that the only way it could work is if we shipped support that was dynamically linked and based off of OS libraries, e.g. CoreImage on OSX. But it's not clear that would work, and it would only be the one platform at this point.

I'm going to close this, if the world comes around to the point that it's reasonable, we can look at it.

[rspeed]

What about libheif? It takes care of the packaging while using libde265 for reading and x265 for writing. All three libraries are LGPL.

My understanding is that there would be no patent issues whatsoever for Pillow. The HEIF container itself has a blanket patent waiver, and the HEVC licensing authority doesn't require a license for software that isn't bundled with devices.

Under the software initiative, HEVC Advance will not seek a license or royalties on HEVC functionality implemented in application layer software downloaded to mobile devices or personal computers after the initial sale of the device, where the HEVC encoding or decoding is fully executed in software on a general purpose CPU. Examples of the types of software within the policy include browsers, media players and various software applications.

As far as I can tell, there's nothing blocking this other than the current instability of libheif's API.

Edit: Found a better link for the royalty-free license announcement and added a summary.

[farindk]

libheif encoder API is stable since version 1.1.0. It will appear in Debian testing in a few days:
https://packages.qa.debian.org/libh/libheif.html

I have not much experience with python, but if you need any assistance, feel free to contact me (I'm the libheif author).

[CounterPillow]

@rspeed HEVC Advance is only one of three competing HEVC licensing authorities.

[rspeed]

@CounterPillow So we'd need similar guarantees from the others?

[CounterPillow]

Yes, or just hope none of them feel like trying to enforce their patents against you. Though I don't think patent issues should prevent HEIF support from being added to Pillow. It's up to the companies or individuals building commercials products using Pillow to ensure their product doesn't infringe on any patents; I was simply pointing out that Linux distros like RedHat or OpenSuSE likely will want to ship Pillow without HEIF support.

[ncalligaro]

It seems Nokia released an open source HEIF Javascript reader, and also C libraries to read and write HEIF: https://github.com/nokiatech/heif/wiki . Wouldn't it be possible to use this to add support to the library?

[hugovk]

@ncalligaro Unfortunately no, see above: #2806 (comment).

[oebilgen]

Sorry, why are you not using libheif?

[radarhere]

@oebilgen See #2806 (comment)

[Cykooz]

May be I wrong, but this license only applies to Nokia's software, not the HEIF container format.

Nokia Technologies Ltd (“Nokia”) hereby grants to you a ... license,
under its copyrights and Licensed Patents only to, use, run, modify (in a way that still complies
with the Specification), and copy the Software within the Licensed Field.

libheif is not based on Nokia`s code and don't uses Nokia's library. This is an independent implementation of decoder/encoder of HEIF images under GPLv3 license. This library are included in the official repositories of Debian, Ubuntu and other Linux distros.

[radarhere]

@Cykooz I feel like this has already been mentioned - #2806 (comment)

[rspeed]

@radarhere I'm not sure what you're referring to. Nothing I said in that comment disagrees with the point @Cykooz was making.

[r0bdiabl0]

I think it's time to implement libheif if not as a decoder only. I'm noticing a lot of products out there support the conversion of heic files to jpg but not the other way around. I think this is how they are getting away with not worrying about patent issues, etc... But as @rspeed noted in #2806 , there are no issues if you are using the format in the application layer as long as the device already has a license in it's initial software:

Under the software initiative, HEVC Advance will not seek a license or royalties on HEVC functionality implemented in application layer software downloaded to mobile devices or personal computers after the initial sale of the device, where the HEVC encoding or decoding is fully executed in software on a general purpose CPU. Examples of the types of software within the policy include browsers, media players and various software applications.

ImageMagick is using it so why not Pillow.

[instantlinux]

This issue, I think, very definitely needs revisiting in 2020. A large percentage of all image/video files in the world are now generated, managed and stored natively in heic/heif format. The python community needs a tool for coping with this reality.

[simonw]

If anyone needs a workaround for this, I managed to open a HEIC file using the pyheif library and convert it to a Pillow Image like this:

from PIL import Image
import pyheif

heif_file = pyheif.read_heif("IMG_7424.HEIC")
image = Image.frombytes(mode=heif_file.mode, size=heif_file.size, data=heif_file.data)
image.save("IMG_7424.jpg", "JPEG")

[bagipriyank]

is this something that PIL might support in the future?

[mara004]

This issue, I think, needs revisiting in 2021 ;)
As far as I can see, including libheif as dependency should be safe, as even Debian have added it to their main repository, and they have a legal team that checks every package very carefully.

[a-w-1806]

There is now a Pillow plugin for HEIF available, pyheif-pillow-opener.

https://pypi.org/project/pyheif-pillow-opener/

Finally I probably get this working. If you use this pyheif-pillow-opener, you should install pyheif, which depends on libheif. See pyheif's page for how to install that to your OS. And, if you are using Linux OS or container, you probably want to pay attention to something called libde265, or pyheif still installs successfully but won't work with HEIC/HEIF. For a discussion on this, see strukturag/libheif#90.

I can confirm it works.

[mara004]

I just tried using pyheif_pillow_opener, but can't get it to work.

>>> from PIL import Image
>>> from pyheif_pillow_opener import register_heif_opener
>>> register_heif_opener()
>>> img = Image.open(".../heif.heic")
...
PIL.UnidentifiedImageError: cannot identify image file

I'm using a Ubuntu 20.04 derivative, pyheif 0.5.1, libheif1 1.11.0, and libde265-0 1.0.8.

[radarhere]

@mara004 I would suggest asking that over at https://github.com/ciotto/pyheif-pillow-opener

[mara004]

Yeah, good point. I'll do that.

[Cykooz]

You can also try my package https://pypi.org/project/cykooz.heif
I built "manylinux" wheels for Python 3.6-3.9. They contains all required libraries (libheif, libde256 and libx265).
I have been using this package in production for two years now.

from PIL import Image
from cykooz.heif.pil import register_heif_opener

register_heif_opener()
img = Image.open('data/test.heif')
assert isinstance(img, Image.Image)
assert img.size == (3024, 4032)
assert img.mode == 'RGB'
assert img.getpixel((100, 100)) == (73, 74, 69)
img.save('test.jpg', 'JPEG')

[mara004]

I now tried https://github.com/bigcat88/pillow_heif/ which works well

[homm]

I see there are plenty of alternatives and forks. Nevertheless, I want to present my own solution.

First of all, this is not just a fork with some fixes, I've found people who can be responsible for pyheif development. There are already three versions released after reincarnation. So changes more fundamental, like lazy loading or better pypy memory management.

So solution consist of pyheif itself, which have binary wheel for popular platforms and pure-python Pillow plugin:

heif-image-plugin

It also supports AVIF for decoding. You can find more changes in Changelog section.

[Simon566]

this heif-image-plugin somehow crashes:

Collecting albums, done.
  Sorting albums  [####################################]  100%
   Sorting media  [####################################]  100%
Collecting files  [####################################]  100%
Processing files  [------------------------------------]  0/2
Traceback (most recent call last):
  File "/home/simon/python_virt_sigal/env/bin/sigal", line 8, in <module>
    sys.exit(main())
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/sigal/__init__.py", line 157, in build
    gal.build(force=force)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/sigal/gallery.py", line 756, in build
    for status in self.pool.imap_unordered(worker, media_list):
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
    put(task)
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 211, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle 'module' object

[homm]

@Simon566 I noticed than the Issues page was disabled for https://github.com/uploadcare/heif-image-plugin. Fixed it. Please fill the report with minimal reproducible code there if you think this is heif-image-plugin problem.

[bigcat88]

@Simon566 can you please check if the bug is reproducible with pillow-heif ?
If yes, then give me some info(OS(win,mac,unix), cpu and test file) - i will be glad to fix it if so.

[aapris]

I now tried https://github.com/bigcat88/pillow_heif/ which works well

This works for me too, and it was easy to install. I could't get cykooz.heif compiled at all on Mac OS.

[Simon566]

Hi,

I'm running it on Debian 11 in a python virtual environment. I'm using sigal (static image gallery generator) which uses pillow.

I updated to pillow_heif and still it isn't working. How is this heif plugin correctly registered with pillow ?

Traceback (most recent call last):
  File "/home/simon/python_virt_sigal/env/bin/sigal", line 8, in <module>
    sys.exit(main())
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/sigal/__init__.py", line 174, in build
    gal.build(force=force)
  File "/home/simon/python_virt_sigal/env/lib/python3.9/site-packages/sigal/gallery.py", line 852, in build
    for status in self.pool.imap_unordered(worker, media_list):
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
    put(task)
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 211, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle 'module' object

I'm registering this in sigal.conf like:

from PIL import Image, ImageSequence
from pillow_heif import register_heif_opener

register_heif_opener()

here are my packages:
blinker==1.4
cffi==1.15.0
click==8.1.2
importlib-metadata==4.11.3
Jinja2==3.1.1
Markdown==3.3.6
MarkupSafe==2.1.1
natsort==8.1.0
piexif==1.1.3
pilkit==2.0
Pillow==9.1.0
pillow-heif==0.2.1
pip==22.0.4
pkg_resources==0.0.0
pycparser==2.21
pyheif==0.7.0
setuptools==62.1.0
sigal==2.3
zipp==3.8.0

regards,
Simon

[Simon566]

working with the testcode on console nevertheless works ...:

from PIL import Image
import pillow_heif

if pillow_heif.is_supported('input.heic'):
    heif_file = pillow_heif.open_heif('input.heic')
    for img in heif_file:  # you still can use it without iteration, like before.
        img.scale(1024, 768) # `libheif` does not provide much operations, that can be done on image, so just scaling it.
    heif_file.add_thumbnails([768, 512, 256]) # add three new thumbnail boxes.
    # default quality is probably ~77 in x265, set it a bit lower and specify `save mask`.
    heif_file.save('output.heic', quality=70, save_all=False) #save_all is True by default.
    exit(0)

so it looks like a issue with sigal or the way I register the plugin ( i trigger this in the sigal conf file )

[hugovk]

@Simon566 Please report problems with sigal to https://github.com/saimn/sigal and with pillow-heif to https://github.com/bigcat88/pillow_heif.

[BubbleFish-FightClub]

It took me super long to find this thread >_<' I have a good 100+ images that fall under this issue, all of the images have a ".jpg" extension which made finding this even more difficult.

The current error that is provided for these images is here:

Pillow/src/PIL/Image.py

Lines 3094 to 3096 in c4325c8

	raise UnidentifiedImageError(
	"cannot identify image file %r" % (filename if filename else fp)
	)

Should we at least update the code to check Image.format == "HEIF" to give a better error message or at least link to this issue for other to find a solution?

[mara004]

Should we at least update the code to check Image.format == "HEIF" to give a better error message or at least link to this issue for other to find a solution?

I think as long as there is no decoder, Pillow has no way of recognising that your image is HEIF.

[BubbleFish-FightClub]

@mara004 good point. My mistake, I found the Image.format working after having installed the plugin.

[aapris]

I think as long as there is no decoder, Pillow has no way of recognising that your image is HEIF.

I've only .HEIC files generated by iPhone, but all of them contain bytes
0000000 \0 \0 \0 f t y p h e i c \0 \0 \0 \0
in the header. So reading bytes [4:12] of the file may give some kind of a hint if the file is HEIF or not.

I don’t know if such research is Pillow’s job or not.

>>> with open("~/Downloads/IMG_1234.HEIC", "rb") as f:
...   x = f.read()
... 
>>> x[4:12]
b'ftypheic'

More details: strukturag/libheif#83

[mara004]

Analysing the file header for a HEIF signature might be possible. However, I don't think there's much point in adding this to Pillow. Unsupported means unsupported after all, and you don't need Pillow to determine the file type.

[aapris]

This is true. I use python-magic to identify file mime type. It has been very useful.

[bigcat88]

I think as long as there is no decoder, Pillow has no way of recognising that your image is HEIF.

I've only .HEIC files generated by iPhone, but all of them contain bytes 0000000 \0 \0 \0 f t y p h e i c \0 \0 \0 \0 in the header. So reading bytes [4:12] of the file may give some kind of a hint if the file is HEIF or not.

I don’t know if such research is Pillow’s job or not.

>>> with open("~/Downloads/IMG_1234.HEIC", "rb") as f:
...   x = f.read()
... 
>>> x[4:12]
b'ftypheic'

More details: strukturag/libheif#83

Pillow decodes files in two stages:

It loops over the available image plugins in the loaded order, and calls the plugin’s _accept function with the first 16 bytes of the file. If the _accept function returns true, the plugin’s _open method is called to set up the image metadata and image tiles.

pillow-image-plugin

Presence of b'ftypheic' is not a guarantee that images inside file are in x265(HEVC) encoding.

Pillow already exposes get_format_mimetype function for supported files to get mimetype.
And for HEIF it works fine(with right plugin of course).

[lupsin]

I now tried https://github.com/bigcat88/pillow_heif/ which works well

This works for me too, and it was easy to install. I could't get cykooz.heif compiled at all on Mac OS.

Maybe someone will post this information in the documents, because it also took me about two days to find this topic and understand which of the HEIF plugins work for Pillow and support image encoding.
Everyone on stackoverflow mentions pyheif, but it looks like it's poorly maintained and a working pillow-heif is down there somewhere.
And anyone looking for it needs to test every plugin, not work.