JXL/bmff support

[arun54321]

Currently it doesn't recognize jxl images.
The file contains data of an unknown image type

[clanmills]

I've never heard of a jpeg-xl image. Can you provide a test sample, please. More importantly, what is the use case for this format and why should Team Exiv2 undertake the work to support this?

The engineering effort to support a new image format is considerable. There has been a huge effort in the last few months to add support for bmff (.CR3, .HEIC, .AVIF). I'm delighted to say that we released Exiv2 v0.27.4 RC1 earlier today. https://discuss.pixls.us/t/exiv2-v0-27-4-rc1-is-available/24139

[1div0]

JPEG XL is impressive, but it is still too early to focus on the support until it becomes available in the real, not virtual world.

[kmilos]

AFAICT, it is also BMFF based, no?

[1div0]

No. C https://cloud.reflexion.tv/nextcloud/index.php/s/tQpN2gp8eZtnrmT

[clanmills]

The file smells more of JPEG than bmff.

509 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl count=40
       0        0: .......'.#J....EQ._.i_QA.`28...T  ->  ff 0a fa bb e8 f3 e1 27 85 23 4a 01 03 0a 10 45 51 14 00 08 69 00 51 41 01 60 32 38 9e 09 a0 54
    0x20       32: E....y..                          ->  45 8f b6 a4 fb 79 1d 9d
510 rmills@rmillsm1:~/Downloads $ tvisitor -pR 20200717_221452.jxl 
unknown format  20200717_221452.jxl
511 rmills@rmillsm1:~/Downloads $ dmpf ~/Stonehenge.jpg count=40
       0        0: ....;.Exif__II*_.___._..._.___._  ->  ff d8 ff e1 3b a8 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0b 00 0f 01 02 00 12 00 00 00 92 00
    0x20       32: __..._._                          ->  00 00 10 01 02 00 0c 00
512 rmills@rmillsm1:~/Downloads $ 

If that file has Exif data, it's been compressed or something.

528 rmills@rmillsm1:~/Downloads $ dmpf ~/Stonehenge.jpg | grep II\\*_
       0        0: ....;.Exif__II*_.___._..._.___._  ->  ff d8 ff e1 3b a8 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0b 00 0f 01 02 00 12 00 00 00 92 00
   0x340      832: Nikon_..__II*_.___9_._._.___0211  ->  4e 69 6b 6f 6e 00 02 10 00 00 49 49 2a 00 08 00 00 00 39 00 01 00 07 00 04 00 00 00 30 32 31 31
  0x46e0    18144: _.3..M..E.w..)...X....MPF_II*_._  ->  00 10 33 9e ec 4d fa f7 45 c9 77 18 cd 29 d2 87 c2 58 ff e2 0f fe 4d 50 46 00 49 49 2a 00 08 00
0x5e8ee0  6196960: ________________......Exif__II*_  ->  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff d8 ff e1 03 fe 45 78 69 66 00 00 49 49 2a 00
0x5f22e0  6234848: ________________......Exif__II*_  ->  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff d8 ff e1 03 fe 45 78 69 66 00 00 49 49 2a 00
529 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl | grep II\\*_
530 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl | grep -i exi
 0xbf460   783456: ..p........t...q+SF.....f.,._exi  ->  b7 b7 70 9a a1 a7 da f9 b0 b1 7f 74 03 af a3 71 2b 53 46 87 19 c7 09 f5 66 1c 2c 89 5f 65 78 69
 0xd4100   868608: .|.....~.....XEXIK..y5...M......  ->  7f 7c 82 9d c0 da c0 7e eb ce bf c7 e2 58 45 58 49 4b bc 86 79 35 9b 91 9f 4d 07 09 03 e0 18 83
531 rmills@rmillsm1:~/Downloads $ 

[clanmills]

Without a request from a major Exiv2 user, such as darktable or The Gimp, this is unlikely to be undertaken. Our priorities in 2021:

1. v0.27.4 with bmff support
2. v0.28 'main'
3. transition to KDE

[1div0]

Encoding

peter.kovar@Pascal /1TB/usr/src/gitlab.com/wg1/jpeg-xl
€ time build/tools/cjxl ~/Obrázky/PNG/20200717_221452.png ~/Nextcloud/Photo/JPEG\ XL/20200717_221452.jxl
  J P E G   \/ |
            /\ |_   e n c o d e r    [v0.2.0 | SIMD supported: AVX2,SSE4,Scalar]Read 4000x6016 image, 13.8 MP/s
Encoding [VarDCT, d1.000, squirrel], 6 threads.
Compressed to 2643100 bytes (0.879 bpp).
4000 x 6016, 6.25 MP/s [6.25, 6.25], 1 reps, 6 threads.real	0m5,657s
user	0m7,788s
sys	0m1,237s

Decoding

peter.kovar@Pascal /1TB/usr/src/gitlab.com/wg1/jpeg-xl
€ time build/tools/djxl ~/Nextcloud/Photo/JPEG\ XL/20200717_221452.jxl ~/Obrázky/PNG/20200717_221452.jxl.png
Read 2643100 compressed bytes [v0.2.0 | SIMD supported: AVX2,SSE4,Scalar]
Done.
4000 x 6016, 36.31 MP/s [36.31, 36.31], 1 reps, 6 threads.
Allocations: 2276 (max bytes in use: 8.434688E+08)real	0m7,194s
user	0m7,398s
sys	0m0,590s

[jonsneyers]

JPEG XL, if it contains metadata, is using ISOBMFF as a container, using the same Exif box (and xml for XMP) as other formats. Additionally, we are planning to add an option to do Brotli-compressed versions of exif and xmp metadata, though maybe it's a bit early to add support for that since JPEG XL Part 2 (which defines these things) is not finalized yet.

[clanmills]

Thank you @jonsneyers. Do you have a sample image which contains Exif/XMP/ICC/IPTC metadata?

20200717_221452.jxl doesn't look much like bmff, because I don't see box names such as ftyp, meta etc as I see in this avif file. There's no II*\0 sequence from the embedded Tiff that holds the Exif data.

543 rmills@rmillsm1:~/gnu/exiv2/team/book $ dmpf files/avif.avif count=40
       0        0: ___ ftypavif____mif1avifmiafMA1B  ->  00 00 00 20 66 74 79 70 61 76 69 66 00 00 00 00 6d 69 66 31 61 76 69 66 6d 69 61 66 4d 41 31 42
    0x20       32: __.0meta                          ->  00 00 01 30 6d 65 74 61
544 rmills@rmillsm1:~/gnu/exiv2/team/book $ 

The reason I said 20200717_221452.jxl file "smells like JPEG" is because it starts with byte 0xff.

[jonsneyers]

There are two kinds of valid jxl files:

1. A naked codestream, which starts with the "start of jxl codestream" marker 0xFF0A (yes, that smells like JPEG, this is after all a codec standardized by JPEG).
2. A codestream wrapped in an ISOBMFF container (signature JXL , ftyp jxl ).

All render-impacting data is encoded in the codestream, including color space and orientation. So for web delivery, we anticipate option 1 will be most common. All non-render-impacting metadata is stored outside the codestream though, so if you want to have that, you need to use option 2. This includes Exif and XMP, but also JUMBF metadata and "JPEG bitstream reconstruction data", which is all non-image data needed to restore a losslessly recompressed JPEG file bit-exactly.

If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.

[clanmills]

Thank You, @jonsneyers That's very helpful. I've cloned and I'm building jpeg-xl on macOS as follows.

$ git clone --recurse-submodules -j8 https://gitlab.com/wg1/jpeg-xl.git
$ mkdir jpeg-xl/build 
$ cd jpeg-xml/build
$ cmake ..
$ make

The option --recurse-submodules manages the dependency complex which includes highway, Little-CMS and more:

633 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/.git $ cat config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	ignorecase = true
	precomposeunicode = true
[submodule]
	active = .
[remote "origin"]
	url = https://gitlab.com/wg1/jpeg-xl.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
	remote = origin
	merge = refs/heads/master
[submodule "third_party/IQA-optimization"]
	url = https://github.com/veluca93/IQA-optimization.git
[submodule "third_party/brotli"]
	url = https://github.com/google/brotli
[submodule "third_party/difftest_ng"]
	url = https://github.com/thorfdbg/difftest_ng.git
[submodule "third_party/googletest"]
	url = https://github.com/google/googletest
[submodule "third_party/highway"]
	url = https://github.com/google/highway
[submodule "third_party/lcms"]
	url = https://github.com/mm2/Little-CMS
[submodule "third_party/lodepng"]
	url = https://github.com/lvandeve/lodepng
[submodule "third_party/sjpeg"]
	url = https://github.com/webmproject/sjpeg.git
[submodule "third_party/skcms"]
	url = https://skia.googlesource.com/skcms
[submodule "third_party/vmaf"]
	url = https://github.com/Netflix/vmaf.git
634 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/.git $ 

I don't totally understand your comment:

If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.

However, I suspect when the build finishes, we'll be "on the same page". Thank You very much for your help with this.

[clanmills]

I can't build this on macOS.

/Users/rmills/gnu/github/jpeg-xl/lib/extras/codec_jpg.cc:289:5: error: no matching function for call to 'jpeg_mem_src'
    jpeg_mem_src(&cinfo, reinterpret_cast<const unsigned char*>(bytes.data()),
    ^~~~~~~~~~~~
/Library/Frameworks/Mono.framework/Headers/jpeglib.h:959:14: note: candidate function not viable: 2nd argument ('const unsigned char *') would lose const qualifier
EXTERN(void) jpeg_mem_src JPP((j_decompress_ptr cinfo,
             ^
1 error generated.
make[2]: *** [lib/CMakeFiles/jxl_extras-static.dir/extras/codec_jpg.cc.o] Error 1
make[1]: *** [lib/CMakeFiles/jxl_extras-static.dir/all] Error 2
make: *** [all] Error 2

I tried upgrading libjpeg.8.dylib to libjpeg.9.dylib. Same jpeg_mem_src error. So, I mutilated the code with a cast. A few seconds further into the build the compiler encountered issues with tell(), seek() and other I/O function.. So, I thought to enable/disable some of the undocumented options.

(venv) 1031 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/build $ grep -i enable ../CMakeLists.txt | grep -i true
  set(ENABLE_FUZZERS_DEFAULT true)
    set(ENABLE_TCMALLOC_DEFAULT true)
set(JPEGXL_ENABLE_BENCHMARK true CACHE BOOL
set(JPEGXL_ENABLE_EXAMPLES true CACHE BOOL
set(JPEGXL_ENABLE_SJPEG true CACHE BOOL
set(JPEGXL_ENABLE_OPENEXR true CACHE BOOL
set(JPEGXL_ENABLE_SKCMS true CACHE BOOL
(venv) 1032 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/build $ 

I don't know what ENABLE_SJPEG is, however I tried it and still hit the jpeg_mem_src issue.

$ rm -rf CM*
$ cmake .. -DJPEGXL_ENABLE_SJPEG=False
$ make 
.....  rattle rattle rattle .....
error: no matching function for call to 'jpeg_mem_src'
    jpeg_mem_src(&cinfo, reinterpret_cast<const unsigned char*>(bytes.data()),

[kmilos]

@clanmills FWIW, I managed to build under mingw easily:

1. Use system provided mingw-w64-x86_64-{asciidoc,brotli,clang,giflib,lcms2,libjpeg,libpng,libwebp,ninja,openexr}
2. Download and unpack the latest jpeg-xl tarball (0.3.4 for me)
3. Run deps.sh to get the rest of dependencies
4. Follow the rest of the instructions with some small mods:

export CC=clang CXX=clang++
mkdir build
cd build
cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF ..
cmake --build .

I was then able to convert

tools/cjxl ../../exiv2/test/data/Reagan.jpg Reagan.jxl

Reagan.zip

[clanmills]

Thanks, @kmilos for Reagan.jxl. For sure, that smells of bmff:

691 rmills@rmillsm1:~/Downloads $ dmpf Reagan.jxl count=100
       0        0: ___.JXL ....___.ftypjxl ____jxl   ->  00 00 00 0c 4a 58 4c 20 0d 0a 87 0a 00 00 00 14 66 74 79 70 6a 78 6c 20 00 00 00 00 6a 78 6c 20
    0x20       32: __.ZExif____MM_*___._..__.___._.  ->  00 00 16 5a 45 78 69 66 00 00 00 00 4d 4d 00 2a 00 00 00 08 00 13 01 00 00 03 00 00 00 01 00 c8
    0x40       64: __.._.___._.__.._.___.___..._.__  ->  00 00 01 01 00 03 00 00 00 01 00 82 00 00 01 02 00 03 00 00 00 04 00 00 00 f2 01 03 00 03 00 00
    0x60       96: _._.                              ->  00 01 00 01
692 rmills@rmillsm1:~/Downloads $ 

I expect we'll easily decode that. Support for this might go into Exiv2 v0.27.4 RC2.

The 'naked codestream' JXL (which starts with 0xff0a) will require more investigation. We might get lucky and src/jpgimage.cpp can be easily "tweaked" to deal with that.

@jonsneyers do you have a 'naked codestream' test image with metadata. Is there a specification for this format, or is it covered by a pre-existing spec?

[clanmills]

I made a little change to the tvisitor.cpp code in my book, and:

1055 rmills@rmillsmm-local:~/gnu/exiv2/team/book/build $ ./tvisitor -pR  ../files/Reagan.jxl 
STRUCTURE OF JP2 (jxl ) FILE (MM): ../files/Reagan.jxl
 address |   length | box  | uuid | data
      12 |       20 | ftyp |      | jxl ____jxl  106 120 108 32 0 0 0 0 106 120 108 32
      32 |     5722 | Exif |      | ____MM_*___._..__.__ 0 0 0 0 77 77 0 42 0 0 0 8 0 19 1 0 0 3 0 0
    5754 |     5306 | xml  |      | <?xpacket begin="... 60 63 120 112 97 99 107 101 116 32 98 101 103 105 110 61 34 239 187 191
   11060 |     1707 | jbrd |      | .6.....-........H_.. 194 54 20 221 13 232 8 45 147 149 5 222 11 142 166 8 72 0 7 128
   12767 |    20125 | jxlc |      | .......N..L_@_@.01.$ 255 10 8 4 142 129 16 78 25 6 76 0 64 0 64 128 48 49 15 36
END: ../files/Reagan.jxl
1056 rmills@rmillsmm-local:~/gnu/exiv2/team/book/build $ 

As you can see, we've found the Exif and xml boxes. The option -pR will require work because metadata in AVIF, HEIC and CR3 files is stored in substructures of the meta box. Adding box handlers for xml and Exif is straightforward. I will add this to tvisitor.cpp and add a paragraph to the book.

I am not committing to adding JXL support to Exiv2 at the moment. For sure, we need to know more about naked codestream JXL.

Index: ../tvisitor.cpp
===================================================================
--- ../tvisitor.cpp	(revision 5306)
+++ ../tvisitor.cpp	(working copy)
@@ -1334,6 +1334,7 @@
     const char*  kJp2Box_hdlr  = "hdlr";
     const char*  kJp2Box_iinf  = "iinf";
     const char*  kJp2Box_iloc  = "iloc";
+    const char*  kJp2Box_JXL   = "JXL ";
 
     const uint16_t kAppExt     = 0xff21;
     const uint16_t kComExt     = 0xfe21;
@@ -1951,6 +1952,12 @@
             io().read   (&box,4) ; // box
 
             valid_ = boxName(box) == kJp2Box_jP ;
+            if ( boxName(box) == kJp2Box_JXL ) {
+                start_ = 12;
+                io().seek(start_);
+                io().getLong(endian_); // length
+                io().read   (&box,4) ; // box
+            }
             if ( boxName(box) == kJp2Box_ftyp ) {
                 valid_  = true ;
                 io().read(&box,4);