You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've never heard of a jpeg-xl image. Can you provide a test sample, please. More importantly, what is the use case for this format and why should Team Exiv2 undertake the work to support this?
The engineering effort to support a new image format is considerable. There has been a huge effort in the last few months to add support for bmff (.CR3, .HEIC, .AVIF). I'm delighted to say that we released Exiv2 v0.27.4 RC1 earlier today. https://discuss.pixls.us/t/exiv2-v0-27-4-rc1-is-available/24139
peter.kovar@Pascal /1TB/usr/src/gitlab.com/wg1/jpeg-xl
€ time build/tools/cjxl ~/Obrázky/PNG/20200717_221452.png ~/Nextcloud/Photo/JPEG\ XL/20200717_221452.jxl
J P E G \/ |
/\ |_ e n c o d e r [v0.2.0 | SIMD supported: AVX2,SSE4,Scalar]Read 4000x6016 image, 13.8 MP/s
Encoding [VarDCT, d1.000, squirrel], 6 threads.
Compressed to 2643100 bytes (0.879 bpp).
4000 x 6016, 6.25 MP/s [6.25, 6.25], 1 reps, 6 threads.real 0m5,657s
user 0m7,788s
sys 0m1,237s
JPEG XL, if it contains metadata, is using ISOBMFF as a container, using the same Exif box (and xml for XMP) as other formats. Additionally, we are planning to add an option to do Brotli-compressed versions of exif and xmp metadata, though maybe it's a bit early to add support for that since JPEG XL Part 2 (which defines these things) is not finalized yet.
Thank you @jonsneyers. Do you have a sample image which contains Exif/XMP/ICC/IPTC metadata?
20200717_221452.jxl doesn't look much like bmff, because I don't see box names such as ftyp, meta etc as I see in this avif file. There's no II*\0 sequence from the embedded Tiff that holds the Exif data.
A naked codestream, which starts with the "start of jxl codestream" marker 0xFF0A (yes, that smells like JPEG, this is after all a codec standardized by JPEG).
A codestream wrapped in an ISOBMFF container (signature JXL , ftyp jxl ).
All render-impacting data is encoded in the codestream, including color space and orientation. So for web delivery, we anticipate option 1 will be most common. All non-render-impacting metadata is stored outside the codestream though, so if you want to have that, you need to use option 2. This includes Exif and XMP, but also JUMBF metadata and "JPEG bitstream reconstruction data", which is all non-image data needed to restore a losslessly recompressed JPEG file bit-exactly.
If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.
If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.
However, I suspect when the build finishes, we'll be "on the same page". Thank You very much for your help with this.
I tried upgrading libjpeg.8.dylib to libjpeg.9.dylib. Same jpeg_mem_src error. So, I mutilated the code with a cast. A few seconds further into the build the compiler encountered issues with tell(), seek() and other I/O function.. So, I thought to enable/disable some of the undocumented options.
I expect we'll easily decode that. Support for this might go into Exiv2 v0.27.4 RC2.
The 'naked codestream' JXL (which starts with 0xff0a) will require more investigation. We might get lucky and src/jpgimage.cpp can be easily "tweaked" to deal with that.
@jonsneyers do you have a 'naked codestream' test image with metadata. Is there a specification for this format, or is it covered by a pre-existing spec?
As you can see, we've found the Exif and xml boxes. The option -pR will require work because metadata in AVIF, HEIC and CR3 files is stored in substructures of the meta box. Adding box handlers for xml and Exif is straightforward. I will add this to tvisitor.cpp and add a paragraph to the book.
I am not committing to adding JXL support to Exiv2 at the moment. For sure, we need to know more about naked codestream JXL.
Found some nice public info about the header metadata and ICC decoding here: https://arxiv.org/abs/1908.03565 (Annex A and Annex B are of relevance). Might be slightly out of date, but it's something to chew on for starters; one should verify against the reference jpeg-xl software finally of course.
@clanmills How about clearly splitting and documenting this FR in 2 parts: Exif and xml box extraction from the BMFF container JXL variety first, then naked JXL codestream header and ICC decoding in the future?
For sure, it's a two step process to fully support JXL. I think this issue should be renamed "Support JXL/bmff". It's likely to ship in v0.28 on 2021-09-15. You'll see I have set the milestone to v0.27.4, however that's just to keep it visible on my TODO list. I will probably defer it for v0.28.
Can you open another issue (with no milestone) "Support JXL/codestream".
We've had no feedback from @arun54321, so we don't know his expectations.
Thank You, @arun54321 for your feedback. I have opened #1506 for future work on JPX/codestream and renamed this issue to deal with JPX/bmff. I hope the direction we have taken on this issue meets your hopes and expectations.
Fix for JXL/bmff added. PR: #1519. This fix will be in Exiv2 v0.27.4 RC2 expected 2021-04-08. Fix will be in Exiv2 v0.27.4 GM scheduled for 2021-04-30.
Activity
clanmills commentedon Mar 20, 2021
I've never heard of a jpeg-xl image. Can you provide a test sample, please. More importantly, what is the use case for this format and why should Team Exiv2 undertake the work to support this?
The engineering effort to support a new image format is considerable. There has been a huge effort in the last few months to add support for bmff (.CR3, .HEIC, .AVIF). I'm delighted to say that we released Exiv2 v0.27.4 RC1 earlier today. https://discuss.pixls.us/t/exiv2-v0-27-4-rc1-is-available/24139
1div0 commentedon Mar 20, 2021
JPEG XL is impressive, but it is still too early to focus on the support until it becomes available in the real, not virtual world.
kmilos commentedon Mar 20, 2021
AFAICT, it is also BMFF based, no?
1div0 commentedon Mar 20, 2021
No. C https://cloud.reflexion.tv/nextcloud/index.php/s/tQpN2gp8eZtnrmT
clanmills commentedon Mar 20, 2021
The file smells more of JPEG than bmff.
If that file has Exif data, it's been compressed or something.
clanmills commentedon Mar 20, 2021
Without a request from a major Exiv2 user, such as darktable or The Gimp, this is unlikely to be undertaken. Our priorities in 2021:
1div0 commentedon Mar 20, 2021
Encoding
Decoding
jonsneyers commentedon Mar 20, 2021
JPEG XL, if it contains metadata, is using ISOBMFF as a container, using the same
Exifbox (andxmlfor XMP) as other formats. Additionally, we are planning to add an option to do Brotli-compressed versions of exif and xmp metadata, though maybe it's a bit early to add support for that since JPEG XL Part 2 (which defines these things) is not finalized yet.clanmills commentedon Mar 20, 2021
Thank you @jonsneyers. Do you have a sample image which contains Exif/XMP/ICC/IPTC metadata?
20200717_221452.jxl doesn't look much like bmff, because I don't see box names such as ftyp, meta etc as I see in this avif file. There's no II*\0 sequence from the embedded Tiff that holds the Exif data.
The reason I said 20200717_221452.jxl file "smells like JPEG" is because it starts with byte 0xff.
jonsneyers commentedon Mar 21, 2021
There are two kinds of valid jxl files:
JXL, ftypjxl).All render-impacting data is encoded in the codestream, including color space and orientation. So for web delivery, we anticipate option 1 will be most common. All non-render-impacting metadata is stored outside the codestream though, so if you want to have that, you need to use option 2. This includes Exif and XMP, but also JUMBF metadata and "JPEG bitstream reconstruction data", which is all non-image data needed to restore a losslessly recompressed JPEG file bit-exactly.
If you take the current cjxl and encode a PNG or JPEG which has Exif metadata, it will produce an "option 2" file by default and preserve the Exif metadata.
clanmills commentedon Mar 22, 2021
Thank You, @jonsneyers That's very helpful. I've cloned and I'm building jpeg-xl on macOS as follows.
$ git clone --recurse-submodules -j8 https://gitlab.com/wg1/jpeg-xl.git $ mkdir jpeg-xl/build $ cd jpeg-xml/build $ cmake .. $ makeThe option
--recurse-submodulesmanages the dependency complex which includes highway, Little-CMS and more:I don't totally understand your comment:
However, I suspect when the build finishes, we'll be "on the same page". Thank You very much for your help with this.
clanmills commentedon Mar 22, 2021
I can't build this on macOS.
I tried upgrading libjpeg.8.dylib to libjpeg.9.dylib. Same jpeg_mem_src error. So, I mutilated the code with a cast. A few seconds further into the build the compiler encountered issues with tell(), seek() and other I/O function.. So, I thought to enable/disable some of the undocumented options.
I don't know what ENABLE_SJPEG is, however I tried it and still hit the jpeg_mem_src issue.
kmilos commentedon Mar 22, 2021
@clanmills FWIW, I managed to build under mingw easily:
I was then able to convert
Reagan.zip
clanmills commentedon Mar 22, 2021
Thanks, @kmilos for Reagan.jxl. For sure, that smells of bmff:
I expect we'll easily decode that. Support for this might go into Exiv2 v0.27.4 RC2.
The 'naked codestream' JXL (which starts with 0xff0a) will require more investigation. We might get lucky and src/jpgimage.cpp can be easily "tweaked" to deal with that.
@jonsneyers do you have a 'naked codestream' test image with metadata. Is there a specification for this format, or is it covered by a pre-existing spec?
clanmills commentedon Mar 22, 2021
I made a little change to the tvisitor.cpp code in my book, and:
As you can see, we've found the Exif and xml boxes. The option
-pRwill require work because metadata in AVIF, HEIC and CR3 files is stored in substructures of themetabox. Adding box handlers forxmlandExifis straightforward. I will add this to tvisitor.cpp and add a paragraph to the book.I am not committing to adding JXL support to Exiv2 at the moment. For sure, we need to know more about naked codestream JXL.
18 remaining items
kmilos commentedon Mar 24, 2021
Found some nice public info about the header metadata and ICC decoding here: https://arxiv.org/abs/1908.03565 (Annex A and Annex B are of relevance). Might be slightly out of date, but it's something to chew on for starters; one should verify against the reference jpeg-xl software finally of course.
@clanmills How about clearly splitting and documenting this FR in 2 parts:
Exifandxmlbox extraction from the BMFF container JXL variety first, then naked JXL codestream header and ICC decoding in the future?clanmills commentedon Mar 24, 2021
Thanks, @kmilos. This is very helpful.
For sure, it's a two step process to fully support JXL. I think this issue should be renamed "Support JXL/bmff". It's likely to ship in v0.28 on 2021-09-15. You'll see I have set the milestone to v0.27.4, however that's just to keep it visible on my TODO list. I will probably defer it for v0.28.
Can you open another issue (with no milestone) "Support JXL/codestream".
We've had no feedback from @arun54321, so we don't know his expectations.
veluca93 commentedon Mar 24, 2021
IIRC the only part that is even remotely up to date in that document is the ICC predictor. The source code is probably the best place to get information from, i.e. https://gitlab.com/wg1/jpeg-xl/-/blob/master/lib/jxl/headers.h
arun54321 commentedon Mar 25, 2021
Should I rename the issue to 'Support JXL/bmff' ?
[-]Add support for jpeg-xl images.[/-][+]Please add Support for JXL/codestream.[/+][-]Please add Support for JXL/codestream.[/-][+]JXL/bmff support[/+]clanmills commentedon Mar 25, 2021
Thank You, @arun54321 for your feedback. I have opened #1506 for future work on JPX/codestream and renamed this issue to deal with JPX/bmff. I hope the direction we have taken on this issue meets your hopes and expectations.
clanmills commentedon Apr 3, 2021
Fix for JXL/bmff added. PR: #1519. This fix will be in Exiv2 v0.27.4 RC2 expected 2021-04-08. Fix will be in Exiv2 v0.27.4 GM scheduled for 2021-04-30.
JXL. Updated book based on Exiv2/exiv2#1503 (comment)