Skip to content

JXL/bmff support #1503

Closed
Closed

Description

@arun54321

Currently it doesn't recognize jxl images.
The file contains data of an unknown image type

Activity

  1. clanmills commented on Mar 20, 2021

    @clanmills
    Collaborator

    I've never heard of a jpeg-xl image. Can you provide a test sample, please. More importantly, what is the use case for this format and why should Team Exiv2 undertake the work to support this?

    The engineering effort to support a new image format is considerable. There has been a huge effort in the last few months to add support for bmff (.CR3, .HEIC, .AVIF). I'm delighted to say that we released Exiv2 v0.27.4 RC1 earlier today. https://discuss.pixls.us/t/exiv2-v0-27-4-rc1-is-available/24139

  2. 1div0 commented on Mar 20, 2021

    @1div0
    Collaborator

    JPEG XL is impressive, but it is still too early to focus on the support until it becomes available in the real, not virtual world.

  3. kmilos commented on Mar 20, 2021

    @kmilos
    Collaborator

    AFAICT, it is also BMFF based, no?

  4. 1div0 commented on Mar 20, 2021

    @1div0
    Collaborator
  5. clanmills commented on Mar 20, 2021

    @clanmills
    Collaborator

    The file smells more of JPEG than bmff.

    509 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl count=40
           0        0: .......'.#J....EQ._.i_QA.`28...T  ->  ff 0a fa bb e8 f3 e1 27 85 23 4a 01 03 0a 10 45 51 14 00 08 69 00 51 41 01 60 32 38 9e 09 a0 54
        0x20       32: E....y..                          ->  45 8f b6 a4 fb 79 1d 9d
    510 rmills@rmillsm1:~/Downloads $ tvisitor -pR 20200717_221452.jxl 
    unknown format  20200717_221452.jxl
    511 rmills@rmillsm1:~/Downloads $ dmpf ~/Stonehenge.jpg count=40
           0        0: ....;.Exif__II*_.___._..._.___._  ->  ff d8 ff e1 3b a8 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0b 00 0f 01 02 00 12 00 00 00 92 00
        0x20       32: __..._._                          ->  00 00 10 01 02 00 0c 00
    512 rmills@rmillsm1:~/Downloads $ 

    If that file has Exif data, it's been compressed or something.

    528 rmills@rmillsm1:~/Downloads $ dmpf ~/Stonehenge.jpg | grep II\\*_
           0        0: ....;.Exif__II*_.___._..._.___._  ->  ff d8 ff e1 3b a8 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0b 00 0f 01 02 00 12 00 00 00 92 00
       0x340      832: Nikon_..__II*_.___9_._._.___0211  ->  4e 69 6b 6f 6e 00 02 10 00 00 49 49 2a 00 08 00 00 00 39 00 01 00 07 00 04 00 00 00 30 32 31 31
      0x46e0    18144: _.3..M..E.w..)...X....MPF_II*_._  ->  00 10 33 9e ec 4d fa f7 45 c9 77 18 cd 29 d2 87 c2 58 ff e2 0f fe 4d 50 46 00 49 49 2a 00 08 00
    0x5e8ee0  6196960: ________________......Exif__II*_  ->  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff d8 ff e1 03 fe 45 78 69 66 00 00 49 49 2a 00
    0x5f22e0  6234848: ________________......Exif__II*_  ->  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ff d8 ff e1 03 fe 45 78 69 66 00 00 49 49 2a 00
    529 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl | grep II\\*_
    530 rmills@rmillsm1:~/Downloads $ dmpf 20200717_221452.jxl | grep -i exi
     0xbf460   783456: ..p........t...q+SF.....f.,._exi  ->  b7 b7 70 9a a1 a7 da f9 b0 b1 7f 74 03 af a3 71 2b 53 46 87 19 c7 09 f5 66 1c 2c 89 5f 65 78 69
     0xd4100   868608: .|.....~.....XEXIK..y5...M......  ->  7f 7c 82 9d c0 da c0 7e eb ce bf c7 e2 58 45 58 49 4b bc 86 79 35 9b 91 9f 4d 07 09 03 e0 18 83
    531 rmills@rmillsm1:~/Downloads $ 
    

    jpeg

  6. clanmills commented on Mar 20, 2021

    @clanmills
    Collaborator

    Without a request from a major Exiv2 user, such as darktable or The Gimp, this is unlikely to be undertaken. Our priorities in 2021:

    1. v0.27.4 with bmff support
    2. v0.28 'main'
    3. transition to KDE
  7. 1div0 commented on Mar 20, 2021

    @1div0
    Collaborator

    Encoding

    peter.kovar@Pascal /1TB/usr/src/gitlab.com/wg1/jpeg-xl
    € time build/tools/cjxl ~/Obrázky/PNG/20200717_221452.png ~/Nextcloud/Photo/JPEG\ XL/20200717_221452.jxl
      J P E G   \/ |
                /\ |_   e n c o d e r    [v0.2.0 | SIMD supported: AVX2,SSE4,Scalar]Read 4000x6016 image, 13.8 MP/s
    Encoding [VarDCT, d1.000, squirrel], 6 threads.
    Compressed to 2643100 bytes (0.879 bpp).
    4000 x 6016, 6.25 MP/s [6.25, 6.25], 1 reps, 6 threads.real	0m5,657s
    user	0m7,788s
    sys	0m1,237s
    

    Decoding

    peter.kovar@Pascal /1TB/usr/src/gitlab.com/wg1/jpeg-xl
    € time build/tools/djxl ~/Nextcloud/Photo/JPEG\ XL/20200717_221452.jxl ~/Obrázky/PNG/20200717_221452.jxl.png
    Read 2643100 compressed bytes [v0.2.0 | SIMD supported: AVX2,SSE4,Scalar]
    Done.
    4000 x 6016, 36.31 MP/s [36.31, 36.31], 1 reps, 6 threads.
    Allocations: 2276 (max bytes in use: 8.434688E+08)real	0m7,194s
    user	0m7,398s
    sys	0m0,590s
    
  8. jonsneyers commented on Mar 20, 2021

    @jonsneyers

    JPEG XL, if it contains metadata, is using ISOBMFF as a container, using the same Exif box (and xml for XMP) as other formats. Additionally, we are planning to add an option to do Brotli-compressed versions of exif and xmp metadata, though maybe it's a bit early to add support for that since JPEG XL Part 2 (which defines these things) is not finalized yet.

  9. clanmills commented on Mar 20, 2021

    @clanmills
    Collaborator

    Thank you @jonsneyers. Do you have a sample image which contains Exif/XMP/ICC/IPTC metadata?

    20200717_221452.jxl doesn't look much like bmff, because I don't see box names such as ftyp, meta etc as I see in this avif file. There's no II*\0 sequence from the embedded Tiff that holds the Exif data.

    543 rmills@rmillsm1:~/gnu/exiv2/team/book $ dmpf files/avif.avif count=40
           0        0: ___ ftypavif____mif1avifmiafMA1B  ->  00 00 00 20 66 74 79 70 61 76 69 66 00 00 00 00 6d 69 66 31 61 76 69 66 6d 69 61 66 4d 41 31 42
        0x20       32: __.0meta                          ->  00 00 01 30 6d 65 74 61
    544 rmills@rmillsm1:~/gnu/exiv2/team/book $ 
    

    The reason I said 20200717_221452.jxl file "smells like JPEG" is because it starts with byte 0xff.

  10. jonsneyers commented on Mar 21, 2021

    @jonsneyers

    There are two kinds of valid jxl files:

    1. A naked codestream, which starts with the "start of jxl codestream" marker 0xFF0A (yes, that smells like JPEG, this is after all a codec standardized by JPEG).
    2. A codestream wrapped in an ISOBMFF container (signature JXL , ftyp jxl ).

    All render-impacting data is encoded in the codestream, including color space and orientation. So for web delivery, we anticipate option 1 will be most common. All non-render-impacting metadata is stored outside the codestream though, so if you want to have that, you need to use option 2. This includes Exif and XMP, but also JUMBF metadata and "JPEG bitstream reconstruction data", which is all non-image data needed to restore a losslessly recompressed JPEG file bit-exactly.

    If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.

  11. clanmills commented on Mar 22, 2021

    @clanmills
    Collaborator

    Thank You, @jonsneyers That's very helpful. I've cloned and I'm building jpeg-xl on macOS as follows.

    $ git clone --recurse-submodules -j8 https://gitlab.com/wg1/jpeg-xl.git
    $ mkdir jpeg-xl/build 
    $ cd jpeg-xml/build
    $ cmake ..
    $ make

    The option --recurse-submodules manages the dependency complex which includes highway, Little-CMS and more:

    633 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/.git $ cat config 
    [core]
    	repositoryformatversion = 0
    	filemode = true
    	bare = false
    	logallrefupdates = true
    	ignorecase = true
    	precomposeunicode = true
    [submodule]
    	active = .
    [remote "origin"]
    	url = https://gitlab.com/wg1/jpeg-xl.git
    	fetch = +refs/heads/*:refs/remotes/origin/*
    [branch "master"]
    	remote = origin
    	merge = refs/heads/master
    [submodule "third_party/IQA-optimization"]
    	url = https://github.com/veluca93/IQA-optimization.git
    [submodule "third_party/brotli"]
    	url = https://github.com/google/brotli
    [submodule "third_party/difftest_ng"]
    	url = https://github.com/thorfdbg/difftest_ng.git
    [submodule "third_party/googletest"]
    	url = https://github.com/google/googletest
    [submodule "third_party/highway"]
    	url = https://github.com/google/highway
    [submodule "third_party/lcms"]
    	url = https://github.com/mm2/Little-CMS
    [submodule "third_party/lodepng"]
    	url = https://github.com/lvandeve/lodepng
    [submodule "third_party/sjpeg"]
    	url = https://github.com/webmproject/sjpeg.git
    [submodule "third_party/skcms"]
    	url = https://skia.googlesource.com/skcms
    [submodule "third_party/vmaf"]
    	url = https://github.com/Netflix/vmaf.git
    634 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/.git $ 

    I don't totally understand your comment:

    If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.

    However, I suspect when the build finishes, we'll be "on the same page". Thank You very much for your help with this.

  12. clanmills commented on Mar 22, 2021

    @clanmills
    Collaborator

    I can't build this on macOS.

    /Users/rmills/gnu/github/jpeg-xl/lib/extras/codec_jpg.cc:289:5: error: no matching function for call to 'jpeg_mem_src'
        jpeg_mem_src(&cinfo, reinterpret_cast<const unsigned char*>(bytes.data()),
        ^~~~~~~~~~~~
    /Library/Frameworks/Mono.framework/Headers/jpeglib.h:959:14: note: candidate function not viable: 2nd argument ('const unsigned char *') would lose const qualifier
    EXTERN(void) jpeg_mem_src JPP((j_decompress_ptr cinfo,
                 ^
    1 error generated.
    make[2]: *** [lib/CMakeFiles/jxl_extras-static.dir/extras/codec_jpg.cc.o] Error 1
    make[1]: *** [lib/CMakeFiles/jxl_extras-static.dir/all] Error 2
    make: *** [all] Error 2

    I tried upgrading libjpeg.8.dylib to libjpeg.9.dylib. Same jpeg_mem_src error. So, I mutilated the code with a cast. A few seconds further into the build the compiler encountered issues with tell(), seek() and other I/O function.. So, I thought to enable/disable some of the undocumented options.

    (venv) 1031 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/build $ grep -i enable ../CMakeLists.txt | grep -i true
      set(ENABLE_FUZZERS_DEFAULT true)
        set(ENABLE_TCMALLOC_DEFAULT true)
    set(JPEGXL_ENABLE_BENCHMARK true CACHE BOOL
    set(JPEGXL_ENABLE_EXAMPLES true CACHE BOOL
    set(JPEGXL_ENABLE_SJPEG true CACHE BOOL
    set(JPEGXL_ENABLE_OPENEXR true CACHE BOOL
    set(JPEGXL_ENABLE_SKCMS true CACHE BOOL
    (venv) 1032 rmills@rmillsmm-local:~/gnu/github/jpeg-xl/build $ 
    

    I don't know what ENABLE_SJPEG is, however I tried it and still hit the jpeg_mem_src issue.

    $ rm -rf CM*
    $ cmake .. -DJPEGXL_ENABLE_SJPEG=False
    $ make 
    .....  rattle rattle rattle .....
    error: no matching function for call to 'jpeg_mem_src'
        jpeg_mem_src(&cinfo, reinterpret_cast<const unsigned char*>(bytes.data()),
  13. kmilos commented on Mar 22, 2021

    @kmilos
    Collaborator

    @clanmills FWIW, I managed to build under mingw easily:

    1. Use system provided mingw-w64-x86_64-{asciidoc,brotli,clang,giflib,lcms2,libjpeg,libpng,libwebp,ninja,openexr}
    2. Download and unpack the latest jpeg-xl tarball (0.3.4 for me)
    3. Run deps.sh to get the rest of dependencies
    4. Follow the rest of the instructions with some small mods:
    export CC=clang CXX=clang++
    mkdir build
    cd build
    cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF ..
    cmake --build .

    I was then able to convert

    tools/cjxl ../../exiv2/test/data/Reagan.jpg Reagan.jxl

    Reagan.zip

  14. clanmills commented on Mar 22, 2021

    @clanmills
    Collaborator

    Thanks, @kmilos for Reagan.jxl. For sure, that smells of bmff:

    691 rmills@rmillsm1:~/Downloads $ dmpf Reagan.jxl count=100
           0        0: ___.JXL ....___.ftypjxl ____jxl   ->  00 00 00 0c 4a 58 4c 20 0d 0a 87 0a 00 00 00 14 66 74 79 70 6a 78 6c 20 00 00 00 00 6a 78 6c 20
        0x20       32: __.ZExif____MM_*___._..__.___._.  ->  00 00 16 5a 45 78 69 66 00 00 00 00 4d 4d 00 2a 00 00 00 08 00 13 01 00 00 03 00 00 00 01 00 c8
        0x40       64: __.._.___._.__.._.___.___..._.__  ->  00 00 01 01 00 03 00 00 00 01 00 82 00 00 01 02 00 03 00 00 00 04 00 00 00 f2 01 03 00 03 00 00
        0x60       96: _._.                              ->  00 01 00 01
    692 rmills@rmillsm1:~/Downloads $ 

    I expect we'll easily decode that. Support for this might go into Exiv2 v0.27.4 RC2.

    The 'naked codestream' JXL (which starts with 0xff0a) will require more investigation. We might get lucky and src/jpgimage.cpp can be easily "tweaked" to deal with that.

    @jonsneyers do you have a 'naked codestream' test image with metadata. Is there a specification for this format, or is it covered by a pre-existing spec?

  15. clanmills commented on Mar 22, 2021

    @clanmills
    Collaborator

    I made a little change to the tvisitor.cpp code in my book, and:

    1055 rmills@rmillsmm-local:~/gnu/exiv2/team/book/build $ ./tvisitor -pR  ../files/Reagan.jxl 
    STRUCTURE OF JP2 (jxl ) FILE (MM): ../files/Reagan.jxl
     address |   length | box  | uuid | data
          12 |       20 | ftyp |      | jxl ____jxl  106 120 108 32 0 0 0 0 106 120 108 32
          32 |     5722 | Exif |      | ____MM_*___._..__.__ 0 0 0 0 77 77 0 42 0 0 0 8 0 19 1 0 0 3 0 0
        5754 |     5306 | xml  |      | <?xpacket begin="... 60 63 120 112 97 99 107 101 116 32 98 101 103 105 110 61 34 239 187 191
       11060 |     1707 | jbrd |      | .6.....-........H_.. 194 54 20 221 13 232 8 45 147 149 5 222 11 142 166 8 72 0 7 128
       12767 |    20125 | jxlc |      | .......N..L_@_@.01.$ 255 10 8 4 142 129 16 78 25 6 76 0 64 0 64 128 48 49 15 36
    END: ../files/Reagan.jxl
    1056 rmills@rmillsmm-local:~/gnu/exiv2/team/book/build $ 

    As you can see, we've found the Exif and xml boxes. The option -pR will require work because metadata in AVIF, HEIC and CR3 files is stored in substructures of the meta box. Adding box handlers for xml and Exif is straightforward. I will add this to tvisitor.cpp and add a paragraph to the book.

    I am not committing to adding JXL support to Exiv2 at the moment. For sure, we need to know more about naked codestream JXL.

    Index: ../tvisitor.cpp
    ===================================================================
    --- ../tvisitor.cpp	(revision 5306)
    +++ ../tvisitor.cpp	(working copy)
    @@ -1334,6 +1334,7 @@
         const char*  kJp2Box_hdlr  = "hdlr";
         const char*  kJp2Box_iinf  = "iinf";
         const char*  kJp2Box_iloc  = "iloc";
    +    const char*  kJp2Box_JXL   = "JXL ";
     
         const uint16_t kAppExt     = 0xff21;
         const uint16_t kComExt     = 0xfe21;
    @@ -1951,6 +1952,12 @@
                 io().read   (&box,4) ; // box
     
                 valid_ = boxName(box) == kJp2Box_jP ;
    +            if ( boxName(box) == kJp2Box_JXL ) {
    +                start_ = 12;
    +                io().seek(start_);
    +                io().getLong(endian_); // length
    +                io().read   (&box,4) ; // box
    +            }
                 if ( boxName(box) == kJp2Box_ftyp ) {
                     valid_  = true ;
                     io().read(&box,4);
  16. 18 remaining items

  17. self-assigned this
    on Mar 23, 2021
  18. modified the milestones: v0.28, v0.27.4 on Mar 23, 2021
  19. added
    requestfeature request or any other kind of wish
    on Mar 23, 2021
  20. kmilos commented on Mar 24, 2021

    @kmilos
    Collaborator

    Found some nice public info about the header metadata and ICC decoding here: https://arxiv.org/abs/1908.03565 (Annex A and Annex B are of relevance). Might be slightly out of date, but it's something to chew on for starters; one should verify against the reference jpeg-xl software finally of course.

    @clanmills How about clearly splitting and documenting this FR in 2 parts: Exif and xml box extraction from the BMFF container JXL variety first, then naked JXL codestream header and ICC decoding in the future?

  21. clanmills commented on Mar 24, 2021

    @clanmills
    Collaborator

    Thanks, @kmilos. This is very helpful.

    For sure, it's a two step process to fully support JXL. I think this issue should be renamed "Support JXL/bmff". It's likely to ship in v0.28 on 2021-09-15. You'll see I have set the milestone to v0.27.4, however that's just to keep it visible on my TODO list. I will probably defer it for v0.28.

    Can you open another issue (with no milestone) "Support JXL/codestream".

    We've had no feedback from @arun54321, so we don't know his expectations.

  22. veluca93 commented on Mar 24, 2021

    @veluca93

    IIRC the only part that is even remotely up to date in that document is the ICC predictor. The source code is probably the best place to get information from, i.e. https://gitlab.com/wg1/jpeg-xl/-/blob/master/lib/jxl/headers.h

  23. arun54321 commented on Mar 25, 2021

    @arun54321
    Author

    Should I rename the issue to 'Support JXL/bmff' ?

  24. changed the title [-]Add support for jpeg-xl images.[/-] [+]Please add Support for JXL/codestream.[/+] on Mar 25, 2021
  25. changed the title [-]Please add Support for JXL/codestream.[/-] [+]JXL/bmff support[/+] on Mar 25, 2021
  26. clanmills commented on Mar 25, 2021

    @clanmills
    Collaborator

    Thank You, @arun54321 for your feedback. I have opened #1506 for future work on JPX/codestream and renamed this issue to deal with JPX/bmff. I hope the direction we have taken on this issue meets your hopes and expectations.

  27. clanmills commented on Apr 3, 2021

    @clanmills
    Collaborator

    Fix for JXL/bmff added. PR: #1519. This fix will be in Exiv2 v0.27.4 RC2 expected 2021-04-08. Fix will be in Exiv2 v0.27.4 GM scheduled for 2021-04-30.

  28. linked a pull request that will close this issueFix 1503 jxl bmff #1519on Apr 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

requestfeature request or any other kind of wish

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions