Performance, no multithreading for decoding

[jtressle]

Describe the bug

Hi,

Thanks for developing this extension to Pillow. I had a question regarding performance converting a .heic to a .jpg file. Running heif-convert is about twice as fast as doing:

import pillow_heif.HeifImagePlugin

img = Image.open(path_heic)
img.save(path_jpg)

The command line I used was:

for file in *.heic; do heif-convert $file ${file/%.heic/.jpg}; done

Is this expected? For 100 images (of the smaller variety I attached), heif-convert took 42 seconds, pillow_heif took 75 seconds (threaded), and 208 seconds using one thread.

I've attached two files from two datasets I've observed this on.

Thanks,

Steps/Code to Reproduce

import pillow_heif.HeifImagePlugin

img = Image.open(path_heic)
img.save(path_jpg)

images.zip

Expected Results

Performance should mirror heif-converte

Actual Results

About two times slower.

Versions

0.8.0
{'version': {'libheif': '1.14.0', 'x265': 'x265 HEVC encoder (3.5+1-f0c1022b6)', 'aom': 'AOMedia Project AV1 Encoder v3.5.0'}, 'decoders': {'HEVC': 1, 'AV1': 1, 'AVC': 0}, 'encoders': {'HEVC': 1, 'AV1': 1, 'AVC': 0}}

[bigcat88]

Will look tomorrow, just a quick question:
you run python code in cycle something like this?
e.g.:

for i in range(100):
    img = Image.open(path_heic)
    img.save(path_jpg)

[jtressle]

Here is what I did the for the threaded code:

requirements:
pip install pymp-pypi

code:

import pymp
import multiprocessing as mp

with pymp.Parallel(mp.cpu_count()) as p:
    for i in p.range(0,len(image_paths)):
        img = Image.open(path)
        ...
        img.save(path_jpg)

I ran my single threaded code just as you had. The approximate times I got were 75 seconds threaded and 208 seconds single-threaded. I did notice heif-convert is multi-threaded.

Thank you and I appreciate you looking into it.

[bigcat88]

Looks like it is something wrong with speed(multithreading/optimization is disabled?) of compiled version of libheif that bundled in wheels.
If builded from source for example with libheif from ubuntu's repo, or ppa:strukturag the speed is ~equal.
Currently investigating further...
Edited: On Alpine this bug(speed lower 2x) present too, so can say it is present on all Linux...

Bench code:

from subprocess import DEVNULL, run

import pytest
from PIL import Image

import pillow_heif

pillow_heif.register_heif_opener()
print(pillow_heif.libheif_info())

HEIF_CONVERT_PATH = "heif-convert"
# "C:/msys64/mingw64/bin/heif-convert.exe" 


def libheif_convert(f_input, f_output):
    run([HEIF_CONVERT_PATH, f_input, f_output], stderr=DEVNULL, stdout=DEVNULL, check=False)


def pillow_convert(f_input, f_output):
    Image.open(f_input).save(f_output)


@pytest.mark.benchmark(group="small_image")
def test_libheif_convert_small(benchmark):
    benchmark.pedantic(
        libheif_convert, args=("small_image.heic", "small_image_libheif.jpg"), rounds=100, warmup_rounds=3
    )


@pytest.mark.benchmark(group="small_image")
def test_pillow_convert_small(benchmark):
    benchmark.pedantic(
        pillow_convert, args=("small_image.heic", "small_image_pillow.jpg"), rounds=100, warmup_rounds=3
    )


@pytest.mark.benchmark(group="large_image")
def test_libheif_convert_large(benchmark):
    benchmark.pedantic(
        libheif_convert, args=("large_image.heic", "large_image_libheif.jpg"), rounds=50, warmup_rounds=3
    )


@pytest.mark.benchmark(group="large_image")
def test_pillow_convert_large(benchmark):
    benchmark.pedantic(
        pillow_convert, args=("large_image.heic", "large_image_pillow.jpg"), rounds=50, warmup_rounds=3
    )

[bigcat88]

Alpine Linux:

Current build:

---------------------------------------- benchmark 'large_image': 1 tests --
Name (time in s)                 Min     Max    Mean  StdDev  Median     IQR
----------------------------------------------------------------------------
test_pillow_convert_large     2.5099  2.5591  2.5232  0.0123  2.5207  0.0075
----------------------------------------------------------------------------

--------------------------------------------- benchmark 'small_image': 1 tests ------
Name (time in ms)                  Min       Max      Mean  StdDev    Median      IQR
-------------------------------------------------------------------------------------
test_pillow_convert_small     853.6160  886.9666  866.4484  9.7187  866.5191  15.2944
-------------------------------------------------------------------------------------

Dev build for libheif 1.14.1 (currently unreleased, by will be soon)

---------------------------------------- benchmark 'large_image': 1 tests --
Name (time in s)                 Min     Max    Mean  StdDev  Median     IQR
----------------------------------------------------------------------------
test_pillow_convert_large     1.1414  1.1750  1.1569  0.0108  1.1567  0.0154
----------------------------------------------------------------------------

-------------------------------------------- benchmark 'small_image': 1 tests ------
Name (time in ms)                  Min       Max      Mean  StdDev    Median     IQR
------------------------------------------------------------------------------------
test_pillow_convert_small     424.9203  445.5481  430.4459  5.3578  428.2591  6.1504
------------------------------------------------------------------------------------

Approximate time for release of fix - release time of new version of libheif + 5-7 days.

@jtressle Thank you for reporting this.

P.S: i am not sure that macos or windows build are affected by this, cause I take version of libheif builded by brew or msys2, probably it is only Linux bug.

[jtressle]

@bigcat88 that's a big improvement, and I'm glad you were able to find it. Thanks again.

[bigcat88]

@jtressle you can take a look at benchmarks thread, new 0.9.0 version will be published in 4-6 days(still waiting a libheif release)

[jtressle]

@bigcat88 is this still on schedule? Looking forward to testing it.

Thanks.

[bigcat88]

@jtressle sorry, was waiting libheif to publish fix as a release(you can check, even created an issue in their repo) - but no luck, seems author is busy.
Will publish today 0.9.0 version, it will be without this bug, built on libheif 1.14.0 with commits that eliminates this problem.

[bigcat88]

done, updated benchmarks in docs with new results.

[jtressle]

@bigcat88 just checked out the benchmarks. Huge improvement! Also, I've been running 0.9 since yesterday, and it's running smoothly and faster.

Thanks!