Dependency version bumps ahead of v8.8.0

[kleisauke]

Hi John,

I had some spare-time left to make this repo more in line with the build-win64-mxe repo by updating all dependencies to the latest version (where possible), improving the existing patches and adding some new patches for dependencies that cause failures during building. I tested the binaries on the libvips test suite and did not encounter any problems.

This PR closes #22 because there's no need to switch to a new cross-environment. The only differences between this repo and build-win64-mxe are (after merge):

• librsvg isn't updated to 2.45.5 in this repo, because it needs the Rust toolchain and a lot of homemade patches that can break at any time.
• No support for OpenEXR because this also needs a lot of patches / build hacks.
• No support for heifsave because the libx265 DLL is 14.1 MiB.
• No support for MAT file version 7.3 because it requires the HDF5 library which is difficult to compile in a cross-compiler (see the cross-compile build and needed patches).
• No NIfTI support because I'm not sure if anyone's using this dependency on Windows. If you want, I could add this dependency (it's easy to cross-compile).
• No additional compiler flags for better performance and reduced binary size. These flags can easily be added here as well, but it will require some testing.

I also found some motives to cross-compile with JHBuild rather than with MXE:

• It takes less time to build with JHBuild because it will use Ubuntu's MinGW-w64 instead of compiling the entire MinGW-w64 from scratch.
• JHBuild seems to have better support for the Meson build system. Many GNOME dependencies move to this build system.

Improvements:

• Build with Meson for dependencies that no longer support autotools.
• Prefer building with CMake over autotools.
• Clean-up package-vipsdev.sh.
• Add peldd utility to copy only the relevant *.dll files.
• Switch to MinGW-w64 with posix threads to enable C++11/C11 multithreading features.
  (needed by Poppler, OpenJPEG and libheif)

Dependency changes:
-web variant:

• Update Fontconfig from 2.12.6 to 2.13.1 (+ remove libpng as dependency).
• Update FreeType from 2.9.1 to 2.10.0.
• Update HarfBuzz from 2.3.0 to 2.4.0 (+ use CMake).
• Update GDK-PixBuf from 2.36.0 to 2.38.0 (+ use Meson).
• Update GLib from 2.54.3 to 2.61.0 (+ use Meson).
• Update libcroco from 0.6.12 to 0.6.13.
• Update libjpeg-turbo from 1.5.3 to 2.0.2 (+ use CMake).
• Update libpng from 1.6.30 to 1.6.37.
• Update pango from 1.40.14 to 1.43.0 (+ use Meson, add fribidi as dependency).
• Update Pixman from 0.36.0 to 0.38.4.
• Enable Orc and update from 0.4.28 to 0.4.29 (+ use Meson).
• Enable FriBidi and update from 1.0.1 to 1.0.5.

-all variant:

• Update FFTW from 3.3.7 to 3.3.8.
• Update matio from 1.5.6 to 1.5.15.
• Update OpenJPEG from 2.1.1 to 2.3.1.
• Update Poppler from 0.46.0 to 0.76.1.
• Update SQLite from 3.22.0 to 3.28.0.
• Update ImageMagick from 6.8.9-10 to 6.9.10-42.
• Enable CFITSIO and update from 3.43 to 3.45 (+ use CMake).
• Enable libheif / libde265.

[kleisauke]

Removed, otherwise the build of poppler fails:
https://cgit.freedesktop.org/poppler/poppler/tree/CMakeLists.txt?id=248540fbe44886f7ede290c216822df21ad06d7d#n679

[kleisauke]

Removed, I've set cmakeargs and mesonargs in jhbuildrc instead:
https://github.com/libvips/build-win64/pull/26/files#diff-f876610988e7cf5e587ff77317650becR133

[jcupitt]

We should update 8.8/README.md to remove the ref to this. It'll need to note that you need a recent jhbuild for meson support as well.

[jcupitt]

Oh, there are loads of other things too, like pe-util. We should just point people to the Dockerfile for hints on setting up for a non-Docker build.

[kleisauke]

Changed, otherwise Harfbuzz fails:
https://github.com/harfbuzz/harfbuzz/blob/14e1fabc41a9a5ead3d91d560773050469982f54/CMakeLists.txt#L20

Matio seems now also to work with out of tree builds.

[kleisauke]

I've removed the gtk-doc and doc directories. Most people will look for the manuals online, I think.

[kleisauke]

Sorry for the auto-format changes. I should switch from this fancy IDE to VIM next time.

[kleisauke]

I haven't opened a PR at OpenSlide yet.

[kleisauke]

Version bumped a 4-year-old dependency, it'll reduce the attack surface (the latest versions of ImageMagick are continuous fuzzed by OSS-Fuzz).

The only disadvantage of this version bump is that it can suddenly break a build, see:
libvips/build-win64-mxe#7

(ImageMagick removes older versions on their website).

[jcupitt]

We were stuck on an old imagemagick because of intsafe. I guess this in mingw now?

[kleisauke]

I think this was fixed with ImageMagick 6.9.5-3 (released on 22 July 2016). They have excluded the intsafe.h header for MinGW-w64:
ImageMagick/ImageMagick6@23a6645#diff-6a582070a57b62127f59170511cae756R25

[kleisauke]

I couldn't reproduce the segv.

[kleisauke]

This is fixed with the cfitsio-3-fixes.patch.

[kleisauke]

I'm unsure if the g_stat is still buggy, perhaps it has been fixed with:
GNOME/glib@d3d6ef6

/cc @lovell

[lovell]

I hope so; the sharp unit tests will fail if not so there's one way to find out ;)

[kleisauke]

I downloaded the libvips-8.7.4-win32-x64.tar.gz tarball and replaced all lib/*.{dll,lib} and include/* files with the 8.8.0-rc1 (-web) binaries from this PR. Then I re-packed it and uploaded it to S3.

The following commands:

# Note: remove C:\Users\[user]\AppData\Roaming\npm-cache\_libvips before running these commands.
git clone https://github.com/lovell/sharp.git
cd sharp
npm config set sharp_dist_base_url "https://libvips-packaging.s3.amazonaws.com/"
npm install --build-from-source
npm test

Outputs:

...

info sharp Downloading https://libvips-packaging.s3.amazonaws.com/libvips-8.7.4-win32-x64.tar.gz
...

796 passing (48s)

Hooray! 🎉

If you want to test this as well, you can download the binaries of this PR here:
https://libvips-packaging.s3.amazonaws.com/vips-dev-w64-web-8.8.0.zip
https://libvips-packaging.s3.amazonaws.com/vips-dev-w64-all-8.8.0.zip

[jcupitt]

This is fantastic Kleis! I feel we all owe you pretty much a lake of beer. I'll try to read through this over the weekend.

[jcupitt]

-42 has already been deleted :-( perhaps downloading from the github repo will be safer.

[kleisauke]

Indeed downloading from the GitHub repo will be safer. I didn't try it because I'm not sure if they're doing any post-processing on the tarballs.

[jcupitt]

I assumed since they'd set up releases it was OK, and it seems to build. I'll see if I can see any differences.

[jcupitt]

I put the binaries I made with this on https://github.com/libvips/libvips/releases/tag/v8.8.0-rc1 ... I'll try to test them a bit on a win10 machine.

[kleisauke]

Is the transform.patch intentionally committed to version control? I could remember an old issue:
libvips/build-win32#2 (comment)

[jcupitt]

It's 25 years now and he's retired, so I think he's no longer worried about this.

[kleisauke]

Running the libvips test suite with the provided vips-dev-w64-transform-8.8.0-rc1b.zip binaries seems to work on my Windows 10 machine:

pip3 install --user --upgrade -e git+https://github.com/libvips/pyvips.git#egg=pyvips[test]

git clone https://github.com/libvips/libvips.git libvips-master
python -m pytest -v libvips-master/test/test-suite

Output

================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- C:\Python37\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\KleisAuke\Desktop
plugins: flake8-1.0.4
collected 190 items

libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_add PASSED                               [  0%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_sub PASSED                               [  1%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_mul PASSED                               [  1%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_div PASSED                               [  2%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_floordiv PASSED                          [  2%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_pow PASSED                               [  3%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_and PASSED                               [  3%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_or PASSED                                [  4%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_xor PASSED                               [  4%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_more PASSED                              [  5%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_moreeq PASSED                            [  5%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_less PASSED                              [  6%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_lesseq PASSED                            [  6%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_equal PASSED                             [  7%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_noteq PASSED                             [  7%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_abs PASSED                               [  8%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_lshift PASSED                            [  8%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_rshift PASSED                            [  9%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_mod PASSED                               [ 10%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_pos PASSED                               [ 10%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_neg PASSED                               [ 11%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_invert PASSED                            [ 11%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_avg PASSED                               [ 12%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_deviate PASSED                           [ 12%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_polar PASSED                             [ 13%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_rect PASSED                              [ 13%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_conjugate PASSED                         [ 14%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_histfind PASSED                          [ 14%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_histfind_indexed PASSED                  [ 15%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_histfind_ndim PASSED                     [ 15%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_hough_circle PASSED                      [ 16%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_hough_line PASSED                        [ 16%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_sin PASSED                               [ 17%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_cos PASSED                               [ 17%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_tan PASSED                               [ 18%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_asin PASSED                              [ 18%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_acos PASSED                              [ 19%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_atan PASSED                              [ 20%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_log PASSED                               [ 20%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_log10 PASSED                             [ 21%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_exp PASSED                               [ 21%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_exp10 PASSED                             [ 22%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_floor PASSED                             [ 22%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_ceil PASSED                              [ 23%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_rint PASSED                              [ 23%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_sign PASSED                              [ 24%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_max PASSED                               [ 24%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_min PASSED                               [ 25%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_measure PASSED                           [ 25%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_find_trim PASSED                         [ 26%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_profile PASSED                           [ 26%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_project PASSED                           [ 27%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_stats PASSED                             [ 27%]
libvips-master/test/test-suite/test_arithmetic.py::TestArithmetic::test_sum PASSED                               [ 28%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_colourspace PASSED                               [ 28%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_dE00 PASSED                                      [ 29%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_dE76 PASSED                                      [ 30%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_dECMC PASSED                                     [ 30%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_icc PASSED                                       [ 31%]
libvips-master/test/test-suite/test_colour.py::TestColour::test_cmyk PASSED                                      [ 31%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_band_and PASSED                          [ 32%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_band_or PASSED                           [ 32%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_band_eor PASSED                          [ 33%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_bandjoin PASSED                          [ 33%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_bandjoin_const PASSED                    [ 34%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_bandmean PASSED                          [ 34%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_bandrank PASSED                          [ 35%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_cache PASSED                             [ 35%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_copy PASSED                              [ 36%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_bandfold PASSED                          [ 36%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_byteswap PASSED                          [ 37%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_embed PASSED                             [ 37%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_gravity PASSED                           [ 38%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_extract PASSED                           [ 38%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_slice PASSED                             [ 39%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_crop PASSED                              [ 40%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_smartcrop PASSED                         [ 40%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_falsecolour PASSED                       [ 41%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_flatten PASSED                           [ 41%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_premultiply PASSED                       [ 42%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_composite PASSED                         [ 42%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_unpremultiply PASSED                     [ 43%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_flip PASSED                              [ 43%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_gamma PASSED                             [ 44%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_grid PASSED                              [ 44%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_ifthenelse PASSED                        [ 45%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_insert PASSED                            [ 45%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_arrayjoin PASSED                         [ 46%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_msb PASSED                               [ 46%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_recomb PASSED                            [ 47%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_replicate PASSED                         [ 47%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_rot45 PASSED                             [ 48%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_rot PASSED                               [ 48%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_scaleimage PASSED                        [ 49%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_subsample PASSED                         [ 50%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_zoom PASSED                              [ 50%]
libvips-master/test/test-suite/test_conversion.py::TestConversion::test_wrap PASSED                              [ 51%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_conv PASSED                            [ 51%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_compass PASSED                         [ 52%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_convsep PASSED                         [ 52%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_fastcor PASSED                         [ 53%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_spcor PASSED                           [ 53%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_gaussblur PASSED                       [ 54%]
libvips-master/test/test-suite/test_convolution.py::TestConvolution::test_sharpen PASSED                         [ 54%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_black PASSED                                     [ 55%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_buildlut PASSED                                  [ 55%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_eye PASSED                                       [ 56%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_fractsurf PASSED                                 [ 56%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_gaussmat PASSED                                  [ 57%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_gaussnoise PASSED                                [ 57%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_grey PASSED                                      [ 58%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_identity PASSED                                  [ 58%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_invertlut PASSED                                 [ 59%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_logmat PASSED                                    [ 60%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_butterworth_band PASSED                     [ 60%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_butterworth PASSED                          [ 61%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_butterworth_ring PASSED                     [ 61%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_fractal PASSED                              [ 62%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_gaussian_band PASSED                        [ 62%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_gaussian PASSED                             [ 63%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_gaussian_ring PASSED                        [ 63%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_ideal_band PASSED                           [ 64%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_ideal PASSED                                [ 64%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_mask_gaussian_ring_2 PASSED                      [ 65%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_sines PASSED                                     [ 65%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_text PASSED                                      [ 66%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_tonelut PASSED                                   [ 66%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_xyz PASSED                                       [ 67%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_zone PASSED                                      [ 67%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_worley PASSED                                    [ 68%]
libvips-master/test/test-suite/test_create.py::TestCreate::test_perlin PASSED                                    [ 68%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_circle PASSED                                   [ 69%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_flood PASSED                                    [ 70%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_image PASSED                                    [ 70%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_line PASSED                                     [ 71%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_mask PASSED                                     [ 71%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_rect PASSED                                     [ 72%]
libvips-master/test/test-suite/test_draw.py::TestDraw::test_draw_smudge PASSED                                   [ 72%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_vips PASSED                                    [ 73%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_jpeg PASSED                                    [ 73%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_png PASSED                                     [ 74%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_tiff PASSED                                    [ 74%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_magickload PASSED                              [ 75%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_magicksave PASSED                              [ 75%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_webp PASSED                                    [ 76%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_analyzeload PASSED                             [ 76%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_matload PASSED                                 [ 77%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_openexrload SKIPPED                            [ 77%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_fitsload PASSED                                [ 78%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_niftiload SKIPPED                              [ 78%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_openslideload PASSED                           [ 79%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_pdfload FAILED                                 [ 80%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_gifload PASSED                                 [ 80%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_svgload PASSED                                 [ 81%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_csv PASSED                                     [ 81%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_matrix PASSED                                  [ 82%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_ppm PASSED                                     [ 82%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_rad PASSED                                     [ 83%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_dzsave PASSED                                  [ 83%]
libvips-master/test/test-suite/test_foreign.py::TestForeign::test_heifload FAILED                                [ 84%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_cum PASSED                            [ 84%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_equal PASSED                          [ 85%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_ismonotonic PASSED                    [ 85%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_local PASSED                          [ 86%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_match PASSED                          [ 86%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_norm PASSED                           [ 87%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_plot PASSED                           [ 87%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_map PASSED                            [ 88%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_percent PASSED                             [ 88%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_hist_entropy PASSED                        [ 89%]
libvips-master/test/test-suite/test_histogram.py::TestHistogram::test_stdif PASSED                               [ 90%]
libvips-master/test/test-suite/test_iofuncs.py::TestIofuncs::test_split7 PASSED                                  [ 90%]
libvips-master/test/test-suite/test_iofuncs.py::TestIofuncs::test_new_from_image PASSED                          [ 91%]
libvips-master/test/test-suite/test_iofuncs.py::TestIofuncs::test_new_from_memory PASSED                         [ 91%]
libvips-master/test/test-suite/test_iofuncs.py::TestIofuncs::test_get_fields PASSED                              [ 92%]
libvips-master/test/test-suite/test_iofuncs.py::TestIofuncs::test_write_to_memory PASSED                         [ 92%]
libvips-master/test/test-suite/test_morphology.py::TestMorphology::test_countlines PASSED                        [ 93%]
libvips-master/test/test-suite/test_morphology.py::TestMorphology::test_labelregions PASSED                      [ 93%]
libvips-master/test/test-suite/test_morphology.py::TestMorphology::test_erode PASSED                             [ 94%]
libvips-master/test/test-suite/test_morphology.py::TestMorphology::test_dilate PASSED                            [ 94%]
libvips-master/test/test-suite/test_morphology.py::TestMorphology::test_rank PASSED                              [ 95%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_affine PASSED                                [ 95%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_reduce PASSED                                [ 96%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_resize PASSED                                [ 96%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_shrink PASSED                                [ 97%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_thumbnail PASSED                             [ 97%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_similarity PASSED                            [ 98%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_similarity_scale PASSED                      [ 98%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_rotate PASSED                                [ 99%]
libvips-master/test/test-suite/test_resample.py::TestResample::test_mapim PASSED                                 [100%]

====================================================== FAILURES =======================================================
______________________________________________ TestForeign.test_pdfload _______________________________________________

self = <test_foreign.TestForeign object at 0x0000028A5EBAE748>

    @skip_if_no("pdfload")
    def test_pdfload(self):
        def pdf_valid(im):
            a = im(10, 10)
            assert_almost_equal_objects(a, [35, 31, 32, 255])
            assert im.width == 1134
            assert im.height == 680
            assert im.bands == 4

>       self.file_loader("pdfload", PDF_FILE, pdf_valid)

libvips-master\test\test-suite\test_foreign.py:604:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
libvips-master\test\test-suite\test_foreign.py:45: in file_loader
    validate(im)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

im = <pyvips.Image 1133x680 uchar, 4 bands, srgb>

    def pdf_valid(im):
        a = im(10, 10)
        assert_almost_equal_objects(a, [35, 31, 32, 255])
>       assert im.width == 1134
E       assert 1133 == 1134
E         -1133
E         +1134

libvips-master\test\test-suite\test_foreign.py:600: AssertionError
______________________________________________ TestForeign.test_heifload ______________________________________________

self = <test_foreign.TestForeign object at 0x0000028A50032BA8>

    @skip_if_no("heifload")
    def test_heifload(self):
        def heif_valid(im):
            a = im(10, 10)
            assert_almost_equal_objects(a, [75.0, 86.0, 81.0])
            assert im.width == 4032
            assert im.height == 3024
            assert im.bands == 3

        self.file_loader("heifload", HEIC_FILE, heif_valid)
        self.buffer_loader("heifload_buffer", HEIC_FILE, heif_valid)
        self.save_load_buffer("heifsave_buffer", "heifload_buffer",
>                             self.colour, 80)

libvips-master\test\test-suite\test_foreign.py:847:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
libvips-master\test\test-suite\test_foreign.py:84: in save_load_buffer
    buf = pyvips.Operation.call(saver, im, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

operation_name = 'heifsave_buffer', args = (<pyvips.Image 1024x768 uchar, 3 bands, srgb>,), kwargs = {}
string_options = '', op = <pyvips.voperation.Operation object at 0x0000028A5EBAEC88>
arguments = [['in', 19], ['buffer', 35], ['page_height', 18], ['Q', 18], ['lossless', 18], ['strip', 18], ...]
flags_from_name = {'Q': 18, 'background': 18, 'buffer': 35, 'in': 19, ...}, n_required = 1, name = 'background'
flags = 18, match_image = <pyvips.Image 1024x768 uchar, 3 bands, srgb>, n = 1

    @staticmethod
    def call(operation_name, *args, **kwargs):
        """Call a libvips operation.

        Use this method to call any libvips operation. For example::

            black_image = pyvips.Operation.call('black', 10, 10)

        See the Introduction for notes on how this works.

        """

        logger.debug('VipsOperation.call: operation_name = %s', operation_name)
        # logger.debug('VipsOperation.call: args = %s, kwargs =%s',
        #              args, kwargs)

        # pull out the special string_options kwarg
        string_options = kwargs.pop('string_options', '')

        logger.debug('VipsOperation.call: string_options = %s', string_options)

        op = Operation.new_from_name(operation_name)

        arguments = op.get_args()
        # logger.debug('arguments = %s', arguments)

        # make a thing to quickly get flags from an arg name
        flags_from_name = {}

        # count required input args
        n_required = 0

        for name, flags in arguments:
            flags_from_name[name] = flags

            if ((flags & _INPUT) != 0 and
                    (flags & _REQUIRED) != 0 and
                    (flags & _DEPRECATED) == 0):
                n_required += 1

        if n_required != len(args):
            raise Error('unable to call {0}: {1} arguments given, '
                        'but {2} required'.format(operation_name, len(args),
                                                  n_required))

        # the first image argument is the thing we expand constants to
        # match ... look inside tables for images, since we may be passing
        # an array of image as a single param
        match_image = _find_inside(lambda x:
                                   isinstance(x, pyvips.Image),
                                   args)

        logger.debug('VipsOperation.call: match_image = %s', match_image)

        # set any string options before any args so they can't be
        # overridden
        if not op.set_string(string_options):
            raise Error('unable to call {0}'.format(operation_name))

        # set required and optional args
        n = 0
        for name, flags in arguments:
            if ((flags & _INPUT) != 0 and
                    (flags & _REQUIRED) != 0 and
                    (flags & _DEPRECATED) == 0):
                op.set(name, flags, match_image, args[n])
                n += 1

        for name, value in kwargs.items():
            if name not in flags_from_name:
                raise Error('{0} does not support argument '
                            '{1}'.format(operation_name, name))

            op.set(name, flags_from_name[name], match_image, value)

        # build operation
        vop = vips_lib.vips_cache_operation_build(op.pointer)
        if vop == ffi.NULL:
>           raise Error('unable to call {0}'.format(operation_name))
E           pyvips.error.Error: unable to call heifsave_buffer
E             heifload: Success

src\pyvips\pyvips\voperation.py:215: Error
================================== 2 failed, 186 passed, 2 skipped in 40.43 seconds ===================================

I think the pdfload and heifload test can be safely ignored because of the new sizing rules and due to the exclusion of the x265 library (for heifsave). Perhaps the heifload test should be split into a heifload and a heifsave test case.

By the way,
Can we check if libheif is compiled with the built-in h265 encoder during the compilation of libvips? If libheif is compiled without the h265 encoder it will still compile heifsave. The builtin_h265_encoder variable in the pkg-config file might help with this.

[jcupitt]

This all looks great, let's merge!

Nice job again Kleis.

Re heifload / heifsave, let's make a new issue on libvips for that.

[lovell]

I've finally got around to looking properly at the proposed improvements here and all is good. Thanks Kleis for this sterling work!

[lovell]

Ha, John must have merged whilst I was reviewing ;)

[jcupitt]

Sorry @lovell, I didn't realize you were planning to review too, I should have waited.

[lovell]

Oh no need to wait for me, especially as I've been relatively busy recently, it was merely an amusing coincidence that the few minutes I'd found to checkout, run and test this locally were the very same minutes you chose to merge.

[jcupitt]

@kleisauke git master (but not rc2) now spots libheif encode being disabled. It disables heifsave and the related tests.

[lovell]

Quick thought: this change means the libvips org will soon be hosting binaries that can de/encode HEVC, which may infringe patents. Are we OK with this?

[kleisauke]

Thanks for reviewing and merging!

@jcupitt The heifload and heifsave changes looks fine to me. The pkg-config variables were added in libheif v1.3.0 (see here) so this won't work with libheif v1.1.0 (I think the minimum version we support is v1.1.0?).

@lovell I had the same thought and added some notes here:

build-win64/8.8/vips.modules

Lines 913 to 915 in 9930ae7

	don't include libheif/libde265 since HEVC implementations (of which H.265 is one) have legal problems, see for e.g.:
	https://bugzilla.redhat.com/show_bug.cgi?id=1129691#c3
	https://lists.fedoraproject.org/archives/list/legal@lists.fedoraproject.org/thread/GDOFPFQKWJF5CRU7BNWNTJ756AIZOMYK/

Partly for this reason, I did not include the libx265 dependency in the -all variant (so only decoding HEIC files works). I'm not sure if decoding HEIC files will be a patent infringement (I'm not a lawyer). Maybe we should exclude libheif (and libde265) and follow the same path as Fedora.

I hope that in the future, royalty-free encoding formats such as AVIF will become more widely used.

[lovell]

One of the HEVC patent pools is no longer charging royalties for software decoding so the risk of distributing binaries to do this is definitely lower than it used to be. I guess the risk of patent claims will be lower than the risk of security vulnerabilities from these new, relatively large code bases.

I agree that AVIF will win this battle; there are still patents but they are royalty-free.

[jcupitt]

I made heifload/save default on, so it should now work with older libheif, good spot Kleis.

I found this Nov 2016 announcement:

https://www.prnewswire.com/news-releases/hevc-advance-announces-royalty-free-hevc-software-300367212.html

No licence or royalty on software-only encode/decode of h265. It's designed to allow web browsers etc. to include HEVC players, and I guess encoders too. Of course I've no idea if that's enough to make us safe.

[lovell]

Sadly "HEVC Advance" is only one of the HEVC patent pools/holders.

https://www.unifiedpatents.com/news/2019/1/9/independent-economic-study-suggests-hevc-royalties-should-be-comparable-to-or-less-than-rates-for-avc

The MPEG-LA pool starts to charge after 100000 "devices", which includes software, and would explain why the RedHat legal team refuse it in Fedora given the fixed US$0.20 cost per download even if that software/device is free.

I guess this has to be balanced with there being little reward for and a lot of bad press towards any patent holders chasing an open source project without corporate backing for royalties.

[jcupitt]

Argh!

Software patents don't really exist in Europe. Again, no idea if that gives us any protection.

It would be nice if we could move less-used loaders out to a plugin, though I think that's tricky on Windows.