Add support for ISOBMFF Files (AVIF, HEIF, CR3)

[johnny-bit]

Is your feature request related to a problem? Please describe.
Exiv2 does not support ISOBMFF format, therefore doesn't support AVIF, HEIF, CR3 and possibly other filetypes using this format

Describe the solution you'd like
Ideally - ISOBMFF format support, at least reading data from it :)

Describe alternatives you've considered
ExifTool and couple other open source projects already support ISOBMFF based files, however exiv2 still doesn't.

Additional context
Iam fully aware of #1066 - I asked my legal counsel and gethered much needed advice. According to all info I managed to gather, there's no legal issue in creating and using a tool capable of reading data and metadata stored in ISOBMFF containers. Worth noting is the fact that JPEG2000 IS an ISOBMFF based and support for it is present in the library without any issue.

Aditionally, already existing libraries like libavif, libheif, libraw and others already work with ISOBMFF files without legal problems.

The ONLY problematic issue would be not reading but decrypting encrypted (not encoded) data or using proprietary code, which I am sure is not the issue here.

[Filou83]

Please add support for CR3 and HEIF/HEIC image formats, at least to support reading the image/EXIF informations located in the container.

I've read #1066 , #978 and the comment here darktable-org/rawspeed#121 (comment). I tried to understand the concerns about legal issues when reading data from patented ISOBMFF, but I could not see any evidence in this concerns. As far as I've read, there already have been contributions submitted to support these file fomats. Please add them to your library.

Thank you in advance!

[clanmills]

This would now be in Exiv2 v0.27.3 without this show stopper from @D4N #1066 (comment)

It's too late to develop the code required to implement this feature for v0.27.3 which is now at RC2. #1018 (comment)

I will support and mentor/assist anybody willing to work on this issue.

[Filou83]

This would now be in Exiv2 v0.27.3 without this show stopper from @D4N #1066 (comment)

It's too late to develop the code required to implement this feature for v0.27.3 which is now at RC2. #1018 (comment)

I will support and mentor/assist anybody willing to work on this issue.

Thank you for your answer. But I still don't understand exactly what the goal is. There have been 2 contributions done to bring HEIF support to exiv2, maybe these could be reopened?
#978 and #1044
Then #1066 has been opened. So there are 3 proposals to get ISOBMFF support. Maybe one of them could be pushed again into master branch? Maybe it could be your last huge step forward for the project before retireing from exiv2? ;) (with a winking eye and full of hope that you're going to stay)
Or do you believe, the only possible option is to write independend code in exiv2 without using the libheif library?

[clanmills]

I'm working on a book "Image Metadata and Exiv2 Architecture". Here's today's version: https://clanmills.com/exiv2/book/ (and/or https://clanmills.com/exiv2/book/IMaEA.pdf).

The book is coming along quite steadily. The purpose behind the book is to help Exiv2 survive and be developed in future.

It was not only Dan's comment, there was another very negative comment: #1066 (comment) I didn't understand the comment because it refers to a case about which I know nothing.

I don't think we need to use another library to do this. We already have a CR2 and JP2000 parsers. We can probably modify that and we're done. And there's another brainless approach. Exif metadata is written as an embedded TIFF which has the signature II*\0 or MM\0*, XMP <x:xmpmeta x="adobe:ns:meta/" We could read the file byte-by-byte and find the metadata. What's the difference between reading a file byte-by-byte and using the command cp? None that I can think of. Does cp infringe image patents?

However, as Dan said "Patents don't work that way!", and I took that to mean "no matter what solution you propose, you may be in legal trouble!". A couple of years ago, we needed legal advice and I got free help from an expert in intellectual property licensing. I offered to consult with this person before publish any code for ISOBMFF. There was no response to my offer.

Exiv2 v0.27.3 will ship as scheduled on 2020-06-30. I'm going to finish the book and have offered to do "one final release" which will be 0.27.4 or 0.28. Again, no feedback from the team.

We could have a team meeting (on Zoom or something) to discuss this. I can't be bothered.

[pr0m1th3as]

@clanmills Thank you for all your work with exiv2 and please do everyone a big favor and go ahead with the initial plan of adding ISOBMFF support to your last "one final release".

A few words about software patents and multimedia formats

Software patents are put in place to ensure competition advantages of the companies who implement them. Companies like Canon do not want to see a rival bring a camera in the market using their own format and/or proprietary software to process their raw images. And this is the main obstacle for open sourcing their formats.

Nevertheless, what the end user may or may not do with the raw files generated with a legally obtained camera, is not Canon's concern and it can't be, because any files generated with a camera are always the property of the camera's end user and not the manufacturer's.

I am not sure to what Dan referred to when said "Patents don't work that way!"
but legal precedent has shown that software patents work this way: if I loose money from you using my patent, then I sue you and lawyers come into play, if I don't loose money or even benefit from your work, then I don't care, nevertheless I keep my patents in place.
So the question about foreseeable legal troubles boils down to: did exiv2 or darktable or anybody else face legal trouble for reading Canon's CR2 raw format? (yes, there patents for that too).

I am not a lawyer but following @johnny-bit and @clanmills initiative I discussed this issue with one. The above comments can be considered a short summary of this discussion.

[clanmills]

The exiv2 project has no money at all. None. Not a penny. So why would Canon sue somebody with no money? And if the project has no money, Canon would find it difficult to explain in court why reading metadata (written to a public standard) caused them to loose money. As I have pointed out, we could find the metadata by hunting for the markers.

After I started to work on v0.27.3, my wife said "Am I becoming an Exiv2 widow again? You're in your office 12 hours every day. When is this going to stop?". So when Dan and Jens started being so negative, I thought: "I don't need to spend 4 weeks developing and testing ISOBMFF.".

I'm not going to do anything at the moment. Exiv2 v0.27.3 RC2 was published as scheduled last Sunday on 2020-05-31. I hope there will be no code changes and it will be released as 0.27.3 on 2020-06-30.

If no-one else volunteers, I'll reopen #1066 for the next release (0.27.4 or 0.28). However, I hope somebody will undertake this project. I want to pursue other hobbies such as playing the Euphonium and Piano, running, the garden and spending time with my wife.

[pr0m1th3as]

Apparently they wouldn't, but I guess we will have to wait and see what's next. I only wish I had the necessary skills to get involved in a more productive manner other than expressing my support.

On the bright side, your music room makes a clear statement about pursuing other hobbies. Although euphonium, running and gardening are not my things, the piano is a serious temptation 👍 and reasonably deserves for some daily attention, not to mention the wife of course.

[clanmills]

There's another reason that I forgot to mention. I'd like to solve the "Lens Recognition" problem. I'm going to make a proposal to the community at LGM in Rennes in May 2021. #1202 (comment)

In order to work on M2Lscript (pronounced MillsScript), I have to escape from Exiv2 to give me the time. I hope the community will find somebody to take on the maintenance of Exiv2. I intend to run a work-shop at LGM based on my book. I'm doing all I can to ensure the long-term success of Exiv2.

[D4N]

Robin Mills <notifications@github.com> writes:

The exiv2 project has no money at all. None. Not a penny. So why would Canon sue somebody with no money?

It's not about us, it is about the consumers of the exiv2 library. Because they might not have the luxury of a lawyer being able to tell them that continuing to use libexiv2 is ok. Or their lawyer/legal advisor tells them that it is too risky. Given that a big chunk of the Linux Desktop stack depends on exiv2 makes me feel a little uneasy about supporting patent encumbered formats, as this is nothing that can be solved via compile time switches.

[johnny-bit]

@D4N - There is literally nothing legally speaking in the way of loading any file on your disk into memory, reading bytes from it and interpreting those. be it heif, mp3, jpg2k or whatever. Also if data is available within file in know standard, there's nothing legally barring you from interpreting that data according to that standard (meaning exif, xmp etc...).

As I mentioned - you are stomping on legal minefield if you try to decrypt encrypted data. You are violating law if you're using unlicenced proprietary decoder to read encoded data. You are threading near some borders if you do cleanroom-analysis of data within file and write your own decoder.

I assume than none of the above matches description of what's needed to read exif data from ISOBMFF container.

Legal-expertise wise: I asked my legal advisor for legal advice and it's cost. I'll setup a meeting with him and ask for cost of legal expertise. I can cover at least part of the cost associated. I also would be needing description of way of ow exiv2 reads files, does it use any proprietary software or any "problematic" parts I've mentioned above.

[pr0m1th3as]

@D4N Nikon's NEF is a proprietary format, Canon's CRW & CR2 are proprietary formats too, so could you clarify what's the difference with Canon's CR3? They all have patents filed to maintain advantage over their competitors. Even DNG has a patent license called "Digital Negative (DNG) Specification Patent License" although they 've stated about 5 years later (in 2009) that "there were no known intellectual property encumbrances or license requirements for DNG". (sourced from wikipedia and Adobe's website)

AFAIK if one runs into trouble for reading proprietary raw formats, then this trouble should be here onto all of us very long ago. If I remember correctly dcraw started reading such files by reverse engineering them almost two decades ago. To my knowledge, there has been no issue with any opensource software reading proprietary raw files all these years.

On the other hand, if your concern arise from the latest developments with EU legislation and the rumors/concerns that could potentially harm other open source projects like VLC for reading other media formats (audio or video), this is an entirely different story. Regarding VLC and the like, there are legal grounds because those files are (potentially) not legally acquired or/and they still have certain restrictions such as CDs, tapes, vinyl etc. for example you need extra permission to publicly play a music CD even if you bought lawfully. And there is a lot concern about online piracy and all that. Most importantly, when you buy some music on a CD and mp3 file, you own the medium not the intellectual content it is saved in there.

But this is never the case with our camera's raw files, any content saved in a raw format with a camera I take a picture with belongs to me including any copyrights I am willing to introduce to my photos. This the main reason why companies like Canon, Nikon etc never bothered coming after you or every other person like Robin who have devoted significant time into making these formats accessible to the open source community. The other reason is explained in my previous post (nothing to gain). They still charge (possibly) Adobe for some readily available API so they can include CR£ support in their latest LR version, which they charge end users for a lot of money, and they do this because they can (Adobe don't mind some premium for faster support for their software, they take for free from the opensource community a lot more, which is under LR's hood anyway). But you can add this to the reasons why Canon and the like are quite hesitant to provide a free API for reading their raw files.

[1div0]

• please read this https://www.linkedin.com/pulse/future-without-mpeg-leonardo-chiariglione/

[pr0m1th3as]

* please read this https://www.linkedin.com/pulse/future-without-mpeg-leonardo-chiariglione/

I am not sure how this directly relates to the matter of this thread. Quite interesting reading though.

As for the purposes of reverse engineering and reading a proprietary format for interoperability purposes, the following US Law makes it pretty clear IMO and also explains why StarOffice, OpenOffice, & LibreOffice had no legal troubles reading MS Word formats before Microsoft's Open Specification Promise over a decade ago. Not to mention about dcraw, exiftools, exiv2, darktable, etc., which are our concern in this thread.

"Sec. 103(f) of the DMCA (17 U.S.C. § 1201 (f)) says that a person who is in legal possession of a program, is permitted to reverse-engineer and circumvent its protection if this is necessary in order to achieve "interoperability" — a term broadly covering other devices and programs being able to interact with it, make use of it, and to use and transfer data to and from it, in useful ways. A limited exemption exists that allows the knowledge thus gained to be shared and used for interoperability purposes." $
$ text copied from Wikipedia
original source: https://www.law.cornell.edu/uscode/text/17/1201

[clanmills]

Topic: Exiv2 and ISOBMFF Support
Time: Jun 14, 2020 13:00 London

Join Zoom Meeting
https://us02web.zoom.us/j/82136730279?pwd=M3hCbll4cWN3ellJd2pCZkxjVEx3Zz09

Meeting ID: 821 3673 0279
Password: 1fDNUV