revise jxlsave to reduce memory use

[tulzke]

Bug report

When using libvips to convert to jxl, there is a huge memory consumption.
Yes, the image is large. It has a size of 23205x15802. But this is no more than 1GB in raw format.
It eats up all the memory, after which it is killed by the OS via SIGKILL. (My system uses 32Gb RAM)
At the same time, cjxl (the libjxl utility) does this without problems.

To Reproduce
Download that image: https://disk.yandex.ru/d/7YnSsGURImI7Vg
Use cli command: vips copy to_jpg_example.jpg converted.jxl

Expected behavior
Successful conversion to jxl

Actual behavior
Killing a process after uncontrolled memory consumption

Screenshots

Environment

• OS: Zorin OS 16.2 (based on Ubuntu 20.04)
• Vips: vips-8.13.2
• libjxl: v0.7.0

[jcupitt]

Hi @tulzke,

libvips is still using the original one-shot encode mode for libjxl, so yes, memory consumption is high for large images. I tried with a largish image here:

$ vipsheader st-francis.jpg
st-francis.jpg: 30000x26319 uchar, 3 bands, srgb, jpegload
$ /usr/bin/time -f %M:%e vips copy st-francis.jpg st-francis.jxl
memory: high-water mark 2.46 GB
34650700:30.25

So 30gb and 30s for a 30k x 30k pixel image.

I think there's now a streaming encode mode, so yes, libvips should switch to that. As far as I know there's no streaming decode though :-( There's talk of adding a tiled mode to JXL, which would be even better, of course.

Let's tag this as an enhancement and try to revise this for 8.14.

[jcupitt]

I tried your image and saw:

$ /usr/bin/time -f %M:%e vips copy ~/to_jxl_example.jpg x.jxl 
memory: high-water mark 1.22 GB
48146648:66.13

So 48gb of memory and 66s of real time. I wonder why it's more difficult to encode? Perhaps it's related to the colour profile.

[tulzke]

Hello, @jcupitt.
Thanks for the reply. For my purposes (efficient storage of original images for the possibility of subsequent resizing) I went to the trick and reduced the too large images to 4k. In this case, I managed to achieve adequate resource consumption.

[atjn]

This is causing errors in production for me. The errors are pretty bad because they make the entire process hang instead of allowing error handling to run and finish the process. I know the integration is still very early, and I wasn't expecting it to be flawless, but just wanted to let you know :)

I ran some tests with wasm-vips, and it seems like any image that is roughly larger than 3500x3500 will crash the process. This is a problem because my pipeline deals with images for 4K screens, which are just in that ballpark of resolution. Here is a test case you can run in the browser: test case

Thank you for already having such good JPEG XL support!

[kleisauke]

It looks like libjxl is considering supporting a streaming/chunk-wise encode API, see:
libjxl/libjxl#2632

[sandstrom]

There were some updates in 0.9, with better support for streaming in libjxl:

https://github.com/libjxl/libjxl/releases/tag/v0.9.0

Also related is this issue in libjxl's repo, where there seem to be some interest from the libvips team to contribute to libvips directly. I think that would be ideal!

libjxl/libjxl#2640 (comment)

[JinEnMok]

Per Cloudinary's recent test, an 80 (?) MP image now requires only ~1GB of memory as opposed to 8GB.

Compressing OP's file also took only 1GB (per VIPS's 'highwater mark') with libvips 8.15.1 and libjxl 0.10.1, and although Gwenview refused to display the file, djxl was able to decode it to ppm and it displayed correctly.

[jcupitt]

Yes, it's a nice improvement! We've started building libvips binaries with 0.10.1, eg.:

https://github.com/jcupitt/vipsdisp/releases/tag/v3.0.4

[jcupitt]

per VIPS's 'highwater mark'

The libvips highwater mark just counts libvips pixel buffers, it won't include memory allocated by external libraries. I usually test memuse with eg.:

$ vipsheader nina.jpg
nina.jpg: 6048x4032 uchar, 3 bands, srgb, jpegload
$ /usr/bin/time -f %M:%e vips copy nina.jpg x.jxl
memory: high-water mark 74.22 MB
2303124:4.46

So 2.3gb of ram and 4.5s elapsed time. With lbjxl 0.10.1:

$ /usr/bin/time -f %M:%e vips copy nina.jpg x.jxl
memory: high-water mark 74.20 MB
463988:1.00

About a 5x improvement in encoding speed and memory use for this photo.