description : The value of TIFFOffset is not approciate and setting D…

[jaehojung]

Hello, My Id is Jungho8033 in http://dev.exiv2.org/projects.
I would like to adjust code this for offset of TIFFImage Header is not approciate.
Please consider this code... Thank you.

[D4N]

Hi @jaehojung, thanks for your patch. I see that you have created a bug report on redmine: http://dev.exiv2.org/issues/1359. Could you please provide us with a test image that fails when being processed by exiv2, before applying your patch? You can drag-and-drop the image here, I'll then be able to take a look at the issue more closely.

[jaehojung]

It's an invalid file, but is this possible?

[D4N]

That file is definitely corrupted, I cannot open it with any image viewer. What happened to it?

Exiv2 build from master cannot find anything in that file, but with your patch it at least finds the resolution (like exiftool does). I'll have to take a look at the tiff file specifications to be able to decide whether we want to merge this (that might take a while).

[jaehojung]

As you can see, the uploaded image is not opened, but the tiffheader value is one of the files I want to test.
If you enter the exiv2 command, the modified part of the code will be output.(Please refer to the file below.)

If it is in the exif tool, it should be exiv2.

[clanmills]

@D4N. I'll investigate this as I am familiar with the TIFF Spec.

@jaehojung Where is the test file? Please explain what this is about. If it's not a valid file, why should we accept your patch?

[D4N]

@clanmills The test file can be found here: https://user-images.githubusercontent.com/18898538/42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg. It's linked in the third comment.

If I understand this patch correctly it just works around a wrong header offset. However, I can't really asses what it causes further in the parser.

[jaehojung]

@clanmills The test file is in the third command, you can right-click and save it under a different name.
There are two reasons why I might want to patch in the next version.

1. The exif tool works well, why not in exiv2?
2. For personal reasons, I would like exiv2 to process those files too, and it sounds like both reasons.

[clanmills]

This is very interesting. I've updated Redmine #1359 appropriately including the test file.

Thank You for the test file. I downloaded that earlier and was confused when it was a JPEG. Because the subject involves tiff, I was expecting a tiff file.

I've investigated your file and can extract some metadata.

Here's a document I made about the structure of Tiff files:
Tiff.pdf

$ exiv2 -pR ~/42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg
STRUCTURE OF JPEG FILE: /Users/rmills/42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe0 APP0  |      16 | JFIF.....H.H....
      20 | 0xffe1 APP1  |     804 | Exif..II*..G...................
  STRUCTURE OF TIFF FILE (II): MemIo
  END MemIo
     826 | 0xffdb DQT   |     132 
     960 | 0xffc4 DHT   |     418 
    1380 | 0xffc0 SOF0  |      17 
    1399 | 0xffed APP13 |     502 | Photoshop 3.0.8BIM..........<...
  Record | DataSet | Name                     | Length | Data
       1 |      60 | EnvelopePriority         |      0 | 
       1 |      70 | DateSent                 |      0 | 
       1 |      80 | TimeSent                 |      0 | 
       1 |      90 | CharacterSet             |      0 | 
       2 |       3 | ObjectType               |      0 | 
       2 |       4 | ObjectAttribute          |      0 | 
       2 |       5 | ObjectName               |     17 | 0....... 0000000.
       2 |       7 | EditStatus               |      0 | 
       2 |      10 | Urgency                  |      1 | 3
       2 |      12 | Subject                  |      0 | 
       2 |      15 | Category                 |      0 | 
       2 |      25 | Keywords                 |      0 | 
       2 |      30 | ReleaseDate              |      0 | 
       2 |      35 | ReleaseTime              |      0 | 
       2 |      40 | SpecialInstructions      |      0 | 
       2 |      55 | DateCreated              |      0 | 
       2 |      60 | TimeCreated              |      0 | 
       2 |      80 | Byline                   |      0 | 
       2 |      90 | City                     |      4 | 0...
       2 |      95 | ProvinceState            |      0 | 
       2 |     101 | CountryName              |      0 | 
       2 |     105 | Headline                 |      0 | 
       2 |     110 | Credit                   |      0 | 
       2 |     115 | Source                   |      3 | 000
       2 |     116 | Copyright                |  10524 | .z....U.......00000000.....0000000.........
    1906 | 0xff00      
...
  826843 | 0xff00      
  827035 | 0xffd9 EOI  

It's clear that the Exif/APP1 block is malformed. Let's extract that:

# 20+4+6 = Offset, 2 JPEG bytes, 2 byte length, 6 bytes for 'Exif\0\0'`
$ dd bs=1 skip=$((20+4+6)) count=$((804-4-6)) if=~/42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg > 42.tiff
$ od -h 42.tiff | head -1
0000000      4949    002a    470e    0000    000a    010f    0002    0006
$ od -a 42.tiff  | head -1
0000000    I   I   * nul  so   G nul nul  nl nul  si soh stx nul ack nul
$ 

The offset record 0x470e is incorrect. I've modified it with HexFiend to be 8

$ od -h 42.tiff.modified | head -1
0000000      4949    002a    0008    0000    000a    010f    0002    0006
629 rmills@rmillsmbp:~/gnu/github/exiv2/exiv2/build $ 

And now I can see metadata:

629 rmills@rmillsmbp:~/gnu/github/exiv2/exiv2/build $ exiv2 -pR 42.tiff.modified
STRUCTURE OF TIFF FILE (II): 42.tiff.modified
 address |    tag                              |      type |    count |    offset | value
      10 | 0x010f Make                         |     ASCII |        6 |       134 | ......
      22 | 0x0110 Model                        |     ASCII |       15 |       140 | ..............
      34 | 0x011a XResolution                  |  RATIONAL |        1 |       156 | 72/1
      46 | 0x011b YResolution                  |  RATIONAL |        1 |       164 | 72/1
      58 | 0x0128 ResolutionUnit               |     SHORT |        1 |           | 2
      70 | 0x0131 Software                     |     ASCII |       40 |       172 | ............................... ...
      82 | 0x0132 DateTime                     |     ASCII |       20 |       212 | 2017:11:14 17:28:01
      94 | 0x013b Artist                       |     ASCII |        8 |       232 | ........
     106 | 0x8298 Copyright                    |     ASCII |       10 |       240 | .........
     118 | 0x8769 ExifTag                      |      LONG |        1 |           | 250
  STRUCTURE OF TIFF FILE (II): 42.tiff.modified
   address |    tag                              |      type |    count |    offset | value
       252 | 0x829a ExposureTime                 |  RATIONAL |        1 |       616 | 1/400
       264 | 0x829d FNumber                      |  RATIONAL |        1 |       624 | 45/10
       276 | 0x8822 ExposureProgram              |     SHORT |        1 |           | 1
       288 | 0x8827 ISOSpeedRatings              |     SHORT |        1 |           | 400
       300 | 0x8830 SensitivityType              |     SHORT |        1 |           | 2
       312 | 0x8832 RecommendedExposureIndex     |      LONG |        1 |           | 400
       324 | 0x9000 ExifVersion                  | UNDEFINED |        4 |           | 0230
       336 | 0x9003 DateTimeOriginal             |     ASCII |       20 |       632 | 2017:11:14 15:25:38
       348 | 0x9004 DateTimeDigitized            |     ASCII |       20 |       652 | 2017:11:14 15:25:38
       360 | 0x9201 ShutterSpeedValue            | SRATIONAL |        1 |       672 | 8643856/1000000
       372 | 0x9202 ApertureValue                |  RATIONAL |        1 |       680 | 433985/100000
       384 | 0x9204 ExposureBiasValue            | SRATIONAL |        1 |       688 | 0/1
       396 | 0x9205 MaxApertureValue             |  RATIONAL |        1 |       696 | 3/1
       408 | 0x9207 MeteringMode                 |     SHORT |        1 |           | 5
       420 | 0x9209 Flash                        |     SHORT |        1 |           | 9
       432 | 0x920a FocalLength                  |  RATIONAL |        1 |       704 | 35/1
       444 | 0x9291 SubSecTimeOriginal           |     ASCII |        3 |           | 23
       456 | 0x9292 SubSecTimeDigitized          |     ASCII |        3 |           | 23
       468 | 0xa001 ColorSpace                   |     SHORT |        1 |           | 1
       480 | 0xa20e FocalPlaneXResolution        |  RATIONAL |        1 |       712 | 47185920/32768
       492 | 0xa20f FocalPlaneYResolution        |  RATIONAL |        1 |       720 | 47185920/32768
       504 | 0xa210 FocalPlaneResolutionUnit     |     SHORT |        1 |           | 3
       516 | 0xa401 CustomRendered               |     SHORT |        1 |           | 0
       528 | 0xa402 ExposureMode                 |     SHORT |        1 |           | 1
       540 | 0xa403 WhiteBalance                 |     SHORT |        1 |           | 0
       552 | 0xa406 SceneCaptureType             |     SHORT |        1 |           | 0
       564 | 0xa431 BodySerialNumber             |     ASCII |       13 |       728 | 098016001259
       576 | 0xa432 LensSpecification            |  RATIONAL |        4 |       742 | 24/1 70/1 0/0 0/0
       588 | 0xa434 LensModel                    |     ASCII |       24 |       774 | EF24-70mm f/2.8L II nsSp
       600 | 0xa435 LensSerialNumber             |     ASCII |       11 |       798 | ..........
  END 42.tiff.modified
Exiv2 exception in print action for file 42.tiff.modified:
tiff directory length is too large
630 rmills@rmillsmbp:~/gnu/github/exiv2/exiv2/build $ 

Ah, something interesting. For sure the dates smell OK. The lens looks plausable.

Your patch does not implement my manual edit. I will have to further correct the TIFF directory chain to avoid the error tiff directory length is too large

The output of exiftool is suspicious and contains no Exif data (warning: Bad IFD0 directory).

630 rmills@rmillsmbp:~/gnu/github/exiv2/exiv2/build $ exiftool ~/42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg 
ExifTool Version Number         : 10.82
File Name                       : 42606059-50399768-85b6-11e8-9acd-bad8e8690afb.jpg
Directory                       : /Users/rmills
File Size                       : 808 kB
File Modification Date/Time     : 2018:07:12 16:10:23+01:00
File Access Date/Time           : 2018:07:12 17:12:00+01:00
File Inode Change Date/Time     : 2018:07:12 16:12:10+01:00
File Permissions                : rw-r--r--
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
JFIF Version                    : 1.00
Resolution Unit                 : None
X Resolution                    : 72
Y Resolution                    : 72
Exif Byte Order                 : Little-endian (Intel, II)
Warning                         : Bad IFD0 directory
Image Width                     : 1600
Image Height                    : 1067
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Current IPTC Digest             : a9009b6d018e3a5390227888ce1f595f
Envelope Priority               : Unknown ()
Date Sent                       : 
Time Sent                       : 
Coded Character Set             : 
Object Type Reference           : 
Object Attribute Reference      : 
Object Name                     : 0ëÇѹα¹ 0000000ü
Edit Status                     : 
Urgency                         : 3
Subject Reference               : 
Category                        : 
Keywords                        : 
Release Date                    : 
Release Time                    : 
Special Instructions            : 
Date Created                    : 
Time Created                    : 
By-line                         : 
City                            : 0¸¹Ì
Province-State                  : 
Country-Primary Location Name   : 
Headline                        : 
Credit                          :
Source                          : 000
Date/Time Created               : 
Image Size                      : 1600x1067
Megapixels                      : 1.7
631 rmills@rmillsmbp:~/gnu/github/exiv2/exiv2/build $ 

There's something wrong with the IPTC data block of 502 bytes. In addition to have NUL data, the copyright length is 10524 bytes. Thats impossible and will almost certainly cause security issues in some applications.

Before we can make progress with this, you'll have to convince me that there is a "business case" for working on this. The file is clearly seriously malformed and cannot be opened in image processing applications. What is the origin of this file? For example: camera or a desktop application?

You're going to find it difficult to persuade me that we should support this file. Saying that exiftool operates on this file is not a strong argument as it clearly does not read the Exif metadata.

[jaehojung]

I can not explain the business case, just because it is personal.
It would be nice if you could upgrade this issue in the next version.
But even if it is not, I think I will ask again later for better exiv2.
Let's talk again, I'll erase the test file.
Thank you.

[clanmills]

I've studied your file and documented my analysis. I have proven that neither exiftool nor exiv2 reads the Exif metadata embedded in your file.

I think you've misunderstood my request for a 'business case'. I don't need to know about your private aims, however I need to know why Exiv2 should read metadata from a file which violates the standard.

A convincing 'business case' would be to something substantial such as:

1. I obtained this file was obtained from a camera 'abc'.
   or
   I obtained this this file which was written by application 'xyz' running on platform '123'
   and
2. I have an image library of 100,000 of these files and wish to post-process them using libexiv2
   and
3. These files can be opened in PhotoShop 'v CC06' (a recent version).

The patch you provided is not correct. You are correct to realise that the offset() is wrong, however the correct value is 8 and not size. Additionally, you should fix the directory chain. My analysis is sufficient to identify the chaining issue and insufficient to know the fix for that specification violation. There is another spec violation in the IPTC metadata. There are probably other specification violations to be identified and remedied.

My suggestion to you is to write a small utility program "fixjpeg" which would open a JPEG and carry out the modifications to the image and rewrite the file.

Once you have "fixjpeg" working, you can test your modified images with desktop applications such as GIMP, PhotoShop, Preview (Mac) or the major Browsers (Firefox, Edge, Chrome, Safari). When your image is acceptable to them, please open a new issue with your modified sample image and your request will be reconsidered.

I'm going to close this PR.

[D4N]

@jaehojung If you want exiv2 to be able to parse these kinds of files, please open a new issue where you provide us with more info concerning the origin of these files and your use case, so that we can estimate how common this issue is.

[jaehojung]

@D4N @clanmills , I would like to complete this issue. I am going to register again later.