Support for FujiFilm IFD Tags

[FreddieWitherden]

I would like to add support into exiv2 for some missing FujiFilm tags. Specifically, those listed in:

https://exiftool.org/TagNames/FujiFilm.html#IFD

which exiftool refers to as the IFD tags. While I am reasonably happy adding support for new tags, it appears that exiv2 does not currently understand block they are contained in. I would therefore appreciate some pointers as to what blocks inside tags_int I would need to modify.

[clanmills]

Freddie. My book will be finished in 2020. I've documented about 20 file formats (TIFF, JPEG, PNG and many others). Two aren't finished (GIF and CRW). One isn't started RAF! Sounds as though that's what I'll be doing next week! RAF is Fuji RAW. So, the following files are implicated in the crime:

-rw-r--r--+ 1 rmills  staff   2.4K 11 Oct 11:36 fujimn_int.hpp
-rw-r--r--+ 1 rmills  staff    15K 11 Oct 11:36 fujimn_int.cpp
-rw-r--r--+ 1 rmills  staff    13K 11 Oct 11:36 rafimage.cpp

I've found some sample image here: https://filesamples.com/formats/raf and I'm jolly pleased that exiv2 is able to read the metadata:

867 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -g datetime/i ~/Downloads/sample1.raf 
Exif.Image.DateTime                          Ascii      20  2018:09:06 15:23:59
Exif.Photo.DateTimeOriginal                  Ascii      20  2018:09:06 15:23:59
Exif.Photo.DateTimeDigitized                 Ascii      20  2018:09:06 15:23:59
868 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

I'm also pleased (and surprised) that the debugging command -pS (print structure) also works. I say "surprised" because I don't remember working on that - however clearly I did.

868 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -pS ~/Downloads/sample1.raf 
STRUCTURE OF RAF FILE: /Users/rmills/Downloads/sample1.raf
    Length |   Offset | Payload
        16 |        0 | Magic number : FUJIFILMCCD-RAW 
         4 |       16 | data 1 : 0201
         8 |       20 | data 2 : FF159501
        32 |       28 | camera : X-T3
         4 |       60 | dir version : 0100
         4 |       84 | JPEG Image Offset : 148
         4 |       88 | JPEG Image Length : 2993767
         4 |       92 | CFA Header Offset : 2994088
         4 |       96 | CFA Header Length : 22104
         4 |      100 | CFA Offset : 3016192
         4 |      104 | CFA Length : 53397824
   2993767 |      148 | jpg image / exif : ......Exif..II*
     22104 |  2994088 | CFA Header: .........V......
  53397824 |  3016192 | CFA : II*............
869 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

You'll see that if it's located the Exiv2 metadata at 2993767. How? I can't remember. However, you don't need to know. You have to define that tags in the Fuji IFD and Phil has done that for you.

When you dump the metadata with the option --unknown, you'll discover lots of unidentified metadata such as:

$ exiv2 --unknown -pa ~/Downloads/sample1.raf 
...
Exif.Fujifilm.FocusMode                      Short       1  Auto
Exif.Fujifilm.0x1022                         Short       1  1
Exif.Fujifilm.0x1023                         Short       2  3598 2090
Exif.Fujifilm.0x1025                         Long        1  1900592
Exif.Fujifilm.0x1026                         Short       1  16
Exif.Fujifilm.0x102b                         Short       1  17
Exif.Fujifilm.0x102c                         Long        1  131073
Exif.Fujifilm.0x102d                         Long        1  16401
Exif.Fujifilm.0x102e                         Long        1  258
Exif.Fujifilm.SlowSync                       Short       1  Off

A tag such as FocusMode is listed in src/fujimn_int.cpp as follows:

Tags such as "Exif.Fujifilm.0x1022" have not been defined and by default are hidden by the exiv2 command-line utility. They have been seen by the Exiv2 engine, however he doesn't know their names!

If you look in code for a "known tag" such as FocusMode (0x1021), you'll find:

877 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ grep -A 3 0x1021 src/fujimn_int.cpp 
    //! FocusMode, tag 0x1021
    extern const TagDetails fujiFocusMode[] = {
        { 0, N_("Auto")   },
        { 1, N_("Manual") }
--
        TagInfo(0x1021, "FocusMode", N_("Focus Mode"),
                N_("Focusing mode setting"),
                fujiId, makerTags, unsignedShort, -1, EXV_PRINT_TAG(fujiFocusMode)),
878 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

You can bring 0x1022 and the others to life by listing them in src/fujimn_int.cpp. Code such as EXV_PRINT_TAG(fujiFocusMode) is the "pretty printer". The following two command will show you this in action:

878 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -pv -g FocusMode ~/Downloads/sample1.raf
0x1021 Fujifilm     FocusMode                   Short       1  0
879 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -pt -g FocusMode ~/Downloads/sample1.raf
Exif.Fujifilm.FocusMode                      Short       1  Auto
880 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

The option -pv says "print the value". It's a SHORT of 0. -pt says "print translated" (ie interpreted for humans). EXV_PRINT_TAG(fujiFocusMode) generates code to print the string at fujiFocusMode[0]. If you're not sure of the value for this field, printValue makes -pv and -pt identical.

You should be able to reconcile Phil's documentation with the Exiv2 implementation. Then decide which tags you want to support and you can probably make a good start on this. I'll be standing to help you. If you get this done, I'll give you a mention in my book. If you get this done (and update the test suite) without any help from me, I'll give you two mentions in the book.

There is a discussion about tag identification in the book here: https://clanmills.com/exiv2/book/#6-6 The code discussed in the book is tvisitor.cpp. It's a simplified version of Exiv2 (3000 lines of code instead of 60,000 + 40,000 Adobe XMPsdk).

You might find the discussion about Exif Metadata helpful: https://clanmills.com/exiv2/book/#Exif

Here's my TODO list (from Monday) I have 3 other matters to deal with tomorrow. RAF is in sight!

[FreddieWitherden]

Robin, thank you very much for taking the time out of your day to walk me through this; it is very much appreciated! I think I now have a reasonable handle on how to add additional tags/tag blocks inside Makernotes. However, there are still a couple of things which are perplexing me about these FujiFilm IFD tags. Please allow me to apologise up-front for my likely incorrect nomenclature.

The tags I am looking at do not appear to be in the unidentified list from exiv2. Additionally, in the excellent exiftool documentation these particular tags are presented without any type/length information (which makes them slightly different to the others I've worked on heretofore). This continues in the code:

https://github.com/exiftool/exiftool/blob/master/lib/Image/ExifTool/FujiFilm.pm#L1314

for example, where just the offset and name are given (but no type!?!). Looking at where the associated structure %Image::ExifTool::FujiFilm::IFD is used I find:

https://github.com/exiftool/exiftool/blob/master/lib/Image/ExifTool/FujiFilm.pm#L1611

in the ProcessRAF function, which gives me the impression (and I may be wrong!) that this structure is outside of the usual RAF gubbins. If I am reading the function correctly this IFD block starts at a location which is stored as a 4 byte integer at either 0x64 or 0x80.

[clanmills]

Well, this is the story of my life. I've documented 19/20 formats and I've explained how to deal with 'unidentified' tags. However we're discussing some secret and magical place that's unique to RAF and doesn't occur in any other format. For sure, RAF is going to be the "main event" next week.

Several suggestions about how to move forward:

1. What metadata are we discussing? Can you look at the files here: https://filesamples.com/formats/raf And give me an ExifTool command to reveal the data in which you are interested.

2. ExifTool has a very useful -v5 option which reveals a great deal of debugging information about how it found the metadata. You might find that helpful. As it produces huge amounts of output, you might find it confusing. Start with -v1 and advance from there.

885 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiftool -v1 ~/Downloads/sample1.raf | wc
    224    1444    8029
886 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiftool -v2 ~/Downloads/sample1.raf | wc
    408    2999   16424
887 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiftool -v3 ~/Downloads/sample1.raf | wc
  26087   57346  292433
888 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiftool -v4 ~/Downloads/sample1.raf | wc
  26356   62887  318520
889 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiftool -v5 ~/Downloads/sample1.raf | wc
  31704  160804  740483
890 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $

3. Get Adobe's DNG Convertor and see if it preserves (or looses) the metadata of interest.

4. What's in the embedded tiff of length 53m (53397824) starting at 3mb (3016192) into sample1.raf

881 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -pS ~/Downloads/sample1.raf 
STRUCTURE OF RAF FILE: /Users/rmills/Downloads/sample1.raf
    Length |   Offset | Payload
        16 |        0 | Magic number : FUJIFILMCCD-RAW 
         4 |       16 | data 1 : 0201
         8 |       20 | data 2 : FF159501
        32 |       28 | camera : X-T3
         4 |       60 | dir version : 0100
         4 |       84 | JPEG Image Offset : 148
         4 |       88 | JPEG Image Length : 2993767
         4 |       92 | CFA Header Offset : 2994088
         4 |       96 | CFA Header Length : 22104
         4 |      100 | CFA Offset : 3016192
         4 |      104 | CFA Length : 53397824
   2993767 |      148 | jpg image / exif : ......Exif..II*
     22104 |  2994088 | CFA Header: .........V......
  53397824 |  3016192 | CFA : II*............
882 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

894 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ dd bs=1 skip=3016192 count=53397824 if=~/Downloads/sample1.raf of=~/temp/Freddie.tiff
53397824+0 records in
53397824+0 records out
53397824 bytes transferred in 361.053512 secs (147894 bytes/sec)
895 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ exiv2 -pa ~/temp/Freddie.tiff 
896 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ tvisitor ~/temp/Freddie.tiff 
STRUCTURE OF TIFF FILE (II): /Users/rmills/temp/Freddie.tiff
 address |    tag                              |      type |    count |    offset | value
  STRUCTURE OF TIFF FILE (II): /Users/rmills/temp/Freddie.tiff
   address |    tag                              |      type |    count |    offset | value
  END: /Users/rmills/temp/Freddie.tiff
END: /Users/rmills/temp/Freddie.tiff
897 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ dmpf --count=40 ~/temp/Freddie.tiff 
       0        0: II*_.___.__.._.___._______._..._  ->  49 49 2a 00 08 00 00 00 01 00 00 f0 0d 00 01 00 00 00 1a 00 00 00 00 00 00 00 12 00 01 f0 04 00
    0x20       32: .___..__                          ->  01 00 00 00 f0 18 00 00
898 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ tvisitor -pUR ~/temp/Freddie.tiff 
STRUCTURE OF TIFF FILE (II): /Users/rmills/temp/Freddie.tiff
 address |    tag                              |      type |    count |    offset | value
      10 | 0xf000 Exif.Image.0xf000            |       IFD |        1 |           | 26
  STRUCTURE OF TIFF FILE (II): /Users/rmills/temp/Freddie.tiff
   address |    tag                              |      type |    count |    offset | value
        28 | 0xf001 Exif.Unknown.0xf001          |      LONG |        1 |           | 6384
        40 | 0xf002 Exif.Unknown.0xf002          |      LONG |        1 |           | 4182
        52 | 0xf003 Exif.Unknown.0xf003          |      LONG |        1 |           | 14
        64 | 0xf004 Exif.Unknown.0xf004          |      LONG |        1 |           | 0
        76 | 0xf005 Exif.Unknown.0xf005          |      LONG |        1 |           | 0
        88 | 0xf006 Exif.Unknown.0xf006          |      LONG |        1 |           | 0
       100 | 0xf007 Exif.Unknown.0xf007          |      LONG |        1 |           | 2048
       112 | 0xf008 Exif.Unknown.0xf008          |      LONG |        1 |           | 53395776
       124 | 0xf009 Exif.Unknown.0xf009          |      LONG |        1 |           | 142
       136 | 0xf00a Exif.Unknown.0xf00a          |      LONG |       36 |       248 | 1022 1022 1022 1022 1022 1022 1022 1 +++
       148 | 0xf00b Exif.Unknown.0xf00b          | SRATIONAL |       19 |       392 | 3749/9 502/1420 710/1420 870/1420 10 +++
       160 | 0xf00c Exif.Unknown.0xf00c          |      LONG |        8 |       544 | 302 371 826 17 302 626 485 21
       172 | 0xf00d Exif.Unknown.0xf00d          |      LONG |        3 |       576 | 302 561 552
       184 | 0xf00e Exif.Unknown.0xf00e          |      LONG |        3 |       588 | 302 561 552
       196 | 0xf00f Exif.Unknown.0xf00f          | SRATIONAL |       29 |       600 | 3749/9 502/1420 710/1420 870/1420 10 +++
       208 | 0xf010 Exif.Unknown.0xf010          | SRATIONAL |       19 |       824 | 3749/9 502/1420 710/1420 870/1420 10 +++
       220 | 0xf011 Exif.Unknown.0xf011          |     SLONG |        1 |           | 0
       232 | 0xf012 Exif.Unknown.0xf012          |     SLONG |        1 |           | 0
  END: /Users/rmills/temp/Freddie.tiff
END: /Users/rmills/temp/Freddie.tiff
899 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ 

That looks like what's in the ExifTool documentation!

[clanmills]

I think I've found it. Phil's documentation says:

---

FujiFilm RAF Tags

FujiFilm RAF images contain meta information stored in a proprietary FujiFilm RAF format, as well as EXIF information stored inside an embedded JPEG preview image. The table below lists tags currently decoded from the RAF-format information.

---

It's clear that Exiv2 does not decode any of this. And we'll need to discover the proprietary FujiFilm RAF format @boardhead is a good friend and will probably give us some pointers.

Today, I'd like to finish stuff I've promised for other Exiv2 users and RAF will be the "main event" next week. I'll get it working in tvisitor.cpp first before thinking about how to do this in Exiv2. Two questions:

1. Which items of metadata do you want to read?
2. Location of a file with this metadata?

[FreddieWitherden]

This is extremely useful. So the information I am interested in is what you pulled out in the previous comment:

  STRUCTURE OF TIFF FILE (II): /Users/rmills/temp/Freddie.tiff
   address |    tag                              |      type |    count |    offset | value
        28 | 0xf001 Exif.Unknown.0xf001          |      LONG |        1 |           | 6384
        40 | 0xf002 Exif.Unknown.0xf002          |      LONG |        1 |           | 4182
        52 | 0xf003 Exif.Unknown.0xf003          |      LONG |        1 |           | 14
        64 | 0xf004 Exif.Unknown.0xf004          |      LONG |        1 |           | 0
        76 | 0xf005 Exif.Unknown.0xf005          |      LONG |        1 |           | 0
        88 | 0xf006 Exif.Unknown.0xf006          |      LONG |        1 |           | 0
       100 | 0xf007 Exif.Unknown.0xf007          |      LONG |        1 |           | 2048
       112 | 0xf008 Exif.Unknown.0xf008          |      LONG |        1 |           | 53395776
       124 | 0xf009 Exif.Unknown.0xf009          |      LONG |        1 |           | 142
       136 | 0xf00a Exif.Unknown.0xf00a          |      LONG |       36 |       248 | 1022 1022 1022 1022 1022 1022 1022 1 +++
       148 | 0xf00b Exif.Unknown.0xf00b          | SRATIONAL |       19 |       392 | 3749/9 502/1420 710/1420 870/1420 10 +++
       160 | 0xf00c Exif.Unknown.0xf00c          |      LONG |        8 |       544 | 302 371 826 17 302 626 485 21
       172 | 0xf00d Exif.Unknown.0xf00d          |      LONG |        3 |       576 | 302 561 552
       184 | 0xf00e Exif.Unknown.0xf00e          |      LONG |        3 |       588 | 302 561 552
       196 | 0xf00f Exif.Unknown.0xf00f          | SRATIONAL |       29 |       600 | 3749/9 502/1420 710/1420 870/1420 10 +++
       208 | 0xf010 Exif.Unknown.0xf010          | SRATIONAL |       19 |       824 | 3749/9 502/1420 710/1420 870/1420 10 +++
       220 | 0xf011 Exif.Unknown.0xf011          |     SLONG |        1 |           | 0
       232 | 0xf012 Exif.Unknown.0xf012          |     SLONG |        1 |           | 0

which if I am understanding everything correctly is stored within an embedded TIFF file that exiv2 currently does not probe inside (as the regular EXIF data is stored elsewhere in the RAF file). Looking at the output from $ exiv2 -pS ~/Downloads/sample1.raf I think we have a clear idea how to find this embedded file. My question is where would be the appropriate place in exiv2 to add the tags, and what would be needed to get the RAF parser to have a go at also decoding this second TIFF?

I am happy to do the leg work here.

[clanmills]

It's Friday night in England and we're about to have 🥩 and 🍷 . You can have fun with the code. We have other plans.

Have a look in printStructure()/rafimage.cpp and you'll see how I discovered the location of the embedded TIFF. I think all you have to do is to tell TiffReader to decode that in readMetadata()/rafimage.cpp.

You don't need to define the MakerNote tags immediately as --unknown will reveal them. When you see them unknown, add the TagInfo definitions in fujimn_int.cpp and I'll hear you laughing and cheering all the way from Texas.

Please work in the 0.27-maintenance branch which I maintain.

By the way, I read your comments about rendering glyphs. I'm a retired old Adobe PostScript guy. Fonts are Amazing.

[clanmills]

I've had a look at the code in rafimage.cpp. I think it's going to be quite easy to implement this and I'll do it next week.

Before reading the code, I was thinking, "how do we write/update the metadata if it's sourced from different parts of the file?". Not a problem. RAF is a "read-only" format in Exiv2.

    void RafImage::writeMetadata()
    {
        //! Todo: implement me!
        throw(Error(kerWritingImageFormatUnsupported, "RAF"));
    } // RafImage::writeMetadata

Next week, I'll deal with RAF in the book and then submit a PR to the 0.27-maintenance branch to support these tags.

Now you see the power of the open-source development model at work. New capability can be added in days. Imagine how long it would take to get a tech giant to update their code.

[clanmills]

Good Progress with this. The book (and tvisitor.cpp) have been updated to deal with RAF and it reads the FujiIFD tags. https://clanmills.com/exiv2/book/#RAF

I've also done some maintenance on rafimage::printStructure(). Assuming no unexpected surprises, I remain confident about finishing this by the end of this week. (Thanksgiviing is a non-event in England).

[FreddieWitherden]

So I had a go at stubbing out some code in rafimage.cpp to read the FujiIFD tags:

        if (io_->seek(100, BasicIo::beg) != 0) throw Error(kerFailedToReadImageData);
        byte sub_img_offset [4];
        if (io_->read(sub_img_offset, 4) != 4) throw Error(kerFailedToReadImageData);
        uint32_t sub_img_off_u32 = Exiv2::getULong((const byte *) sub_img_offset, bigEndian);

        DataBuf sbuf(4096); // Read 4k
        if (io_->seek(sub_img_off_u32, BasicIo::beg) != 0) throw Error(kerFailedToReadImageData);
        io_->read(sbuf.pData_, sbuf.size_);

        if (!io_->error() && !io_->eof())
        {
            TiffParser::decode(exifData_,
                               iptcData_,
                               xmpData_,
                               sbuf.pData_,
                               sbuf.size_);
        }

However, TiffParser::decode does not seem to 'recurse' into the IFD given by tag 0xf000 (unlike your visitor which does). Additionally, it seems to clear anything which may already exist in exifData_ and friends. Is there a recommended means of accomplishing the merging?

[clanmills]

I've tackled this from tvisitor.cpp because I wrote that and totally understand how it works. One of the motivations to write the book was to reveal how the TiffVisitor code in Exiv2 works. It's 10,000 lines of abstract recursive code. Difficult to understand. I hope when somebody studies tvisitor.cpp they'll understand the beauty of how it works. Andreas is a very clever guy to have thought of parsing Tiff in this way. He's a very clever guy in many other ways.

The 0xf000 tag is of type tiffIfd and I think we'll have inject a little bit of magic into the TiffVisitor to cause him to recurse into that structure. And that's why I provided myself with the get out Assuming no unexpected surprises. Your DataBuf sbuff(4096) might be sufficient data to achieve success, however it looks optimistic. The Tiff Tags will be inside sbuff, however the offsets could reference values anywhere in the file. I'll investigate this tomorrow.

I believe TiffParser::decode(...) does clear() the metadata vectors in exifData and chums. Not a problem. We can use "new" vectors and copy the metadata from the "new" vectors into exifData and chums. The samples/metacopy.cpp does something like this. By "new" I only mean a temporary stack variable, nothing to do with C++ new operator.