Recommendations for compression of satellite images that will be read by machines (not humans)

[JackKelly]

Thank you so much for developing JPEG-XL! In my tests it looks like it'll be an excellent compression algorithm for satellite imagery. Thank you!

I appreciate that lossy JPEG-XL compression (like most lossy image compression) is designed primarily for compressing images that will be consumed by humans. As such, the algorithm preferentially throws away information that the human visual system won't notice is gone.

My question is: Which settings should I use when using JPEG-XL compression for monochrome satellite images that will be consumed by machines, not by humans? We want the numerical values of each pixel in the compressed image to remain as close as possible to the original numerical values.

Or is that simply out-of-scope for JPEG-XL?

Somes specific questions might be:

1. Which colorspace should I use when what matters is getting the numerical values correct (and the images will never be displayed on a monitor). We're using single channel, monochrome images.
2. Should we set basic_info.uses_original_profile = JXL_TRUE when doing lossy compression, if we want the numerical values in the compressed image to be as close as possible to the original values?
3. The satellite actually records 12 spectral channels (infrared, water vapour, etc.). We're currently planning to compress each channel individually as a monochrome image. Is that the right thing to do? Or should we try compressing the images 3 channels at a time? (Which would mean fibbing to JPEG-XL: We'd be pretending that we've got RGB images, even though none of the satellite channels correspond at all closely to red, green, or blue).
4. In this comment, @jonsneyers says "One option for lossy is to use lossy Modular mode in non-XYB mode (e.g. cjxl -m -c 0 -q 90), which is a 'stupid' (non-perceptual) form of lossy compression that treats the input numbers as just numbers without any perceptual interpretation." Is that the setting I should be using for satellite imagery? If so, how do we enable those settings using the C++ API instead of cjxl?

Additional context
This is for work that I'm doing at my non-profit, Open Climate Fix, on using satellite images and machine learning to improve near-term predictions of solar electricity power generation (here's a good Wired article explaining our work). We use Zarr to save multiple TBytes of satellite imagery to disk. We're investigating using JPEG-XL as a "Zarr compressor" to compress each image chunk. We use the excellent imagecodecs library as a bridge between Python and JPEG-XL.

Related to issue #314

[jonsneyers]

At the moment, the VarDCT mode is really optimized for 3-channel images for human perception. I'm not sure how well it works if you basically want to optimize for PSNR instead of human perception - you'd indeed need to disable the XYB color transform (so yes, basic_info.uses_original_profile = JXL_TRUE), but I'm not sure how well the DCT-based lossy compression will work if you do that.

One alternative you can try is to use Modular mode in a lossy way. This mode doesn't use the DCT at all but a modified Haar transform, and at least at the moment it doesn't do perceptual optimization but just optimizes for PSNR. That is, if you set uses_original_profile to true; when using XYB it will optimize for PSNR in XYB.
Extra channels are always modular-encoded; to get the main (RGB or Grayscale) channel to be modular-encoded you have to set JXL_ENC_FRAME_SETTING_MODULAR to true.

You should consider compressing all 12 channels at a time, as extra channels. In the regular RGB or Grayscale channel you can either put 1 (or 3) of the channels if they make some kind of visual sense, or else you could put some kind of stub image there with some text or icon indicating "this is satellite imagery, you need a special viewer application for this!" so that people who accidentally open the file in a regular viewer know what to do.
That way, you can use a single jxl file to store all the data of a chunk; you probably have additional metadata about the spectral channels ; you can just define an ISOBMFF box for that and use the libjxl api to read and write it (even with general-purpose compression applied to it). If it's XML-based, you can use the existing xml box for it.

In principle, jxl can to some extent exploit correlations between the channels. In this case, you could set
JXL_ENC_FRAME_SETTING_MODULAR_NB_PREV_CHANNELS to 11 so the last channel can use all previous channels as context for the entropy coding. At the moment this will only help for lossless compression though. This way, it can compress 12 channels at a time better than what you would get by encoding them separately as monochrome channels.

[JackKelly]

Ooh, thank you so much for this speedy and very informative reply! This is super-useful!

Cool, I'll try the ideas you suggested, thank you. (I'm especially excited about using using lossless compression with all 12 channels in a single jxl file).

Please may I ask one more quick question: Our dataset is a timeseries: The satellite takes an image every five minutes. Consecutive frames for a given channel are pretty darn similar. Would I get even better lossless compression if I put multiple timesteps of a single channel into a single JXL file? (Although, if I go that route, would I be better off using a video compression algorithm like AV1 lossless?)

[jonsneyers]

Video codecs are designed for time series so I expect them to be better at it than JXL, at least for lossy.

For lossless, things are different, since inter-frame techniques tend to be less effective for lossless than for lossy. AV1 lossless is not very effective.

If you're dealing with sensor data, a lot of the entropy is in the noise, which is different in every frame. While inter-frame coding tools like motion estimation (or just subtracting the previous frame from the current one, which is something JXL can do) will be able to significantly reduce the average amplitudes of the residuals, most of the entropy is in the least significant bits. The problem is that you end up combining two or more entropy sources, which can actually end up being worse than just using intra frame only. Note that residuals either have twice the data range (e.g. if frame values are in 0..255, then subtracting the previous frame leads to frame values in -255..255), or you have to do something like modulo arithmetic or XORing the bits but that introduces discontinuities or weirdness that will mess with prediction.

If the data is not very noisy, like in synthetic data or data obtained with higher precision than what is actually stored, then it might help to subtract the previous frame from the current one. The libjxl encoder doesn't attempt to do anything other than just storing the frames as-is, though I think you can manually subtract the previous frame from the current one and tell the encoder to signal the kAdd blend mode in the frame header.

[JackKelly]

ooh, this is really useful stuff, thank you!

(The satellite's sensor is 10-bits-per-channel and we're experimenting with storing 8-bits-per-channel, so maybe there's some hope that our images will be fairly clean).

I hope to run some compression experiments tomorrow. I'll report back.

[JackKelly]

One alternative you can try is to use Modular mode in a lossy way... you have to set JXL_ENC_FRAME_SETTING_MODULAR to true

Is there a way to enable Modular mode in the libjxl API v0.6.1? If I've understood correctly, JXL_ENC_FRAME_SETTING_MODULAR doesn't exist in v0.6.1?

[JackKelly]

Actually, sorry, nevermind! I did a few quick experiments at the command line using cjxl v0.7.0 (compiled from source) and it looks like Modular-compressed satellite images (with distance of 0.5 or 1.0) are approximately the same size on disk as losslessly compressed images. Which is no worries! Actually, using JPEG-XL 0.6.1 out-of-the-box with lossy compression looks really great so far in my tests :)

[TheHardew]

@JackKelly if you pass -m -d 1 to cjxl it still encodes in lossless mode. You have to use the -q flag.

I'm intreseted to see how the satelite images would fare with lossy modular.

Edit:
https://docs.google.com/spreadsheets/d/1bTeraUXIl-nGM8c53IdURxmJbabX9eXqPZwVSynoH9U/edit#gid=196105105

[jonsneyers]

Closing the issue to reduce the pile of open issues — feel free to reopen if there are any additional questions or action items related to this.