Canon maker note corruption in v1.0.0 and v0.28.0

[jim-easterbrook]

A minor difference rather than a bug...
Jpeg files from my Canon EOS 100D have an Exif.Photo.MakerNote of 7898 bytes, as reported by the exiv2 command line tool. After modifying some metadata (not any Canon tags) in my own software this gets truncated to 7768 bytes when using libexiv2 v0.27.5, but when using v1.0.0 (nightly Linux64 build) it gets truncated to 7766 bytes. I noticed this because the tag Exif.CanonLe.0x0007 doesn't get saved.

[clanmills]

Great find, @jim-easterbrook. Keep 'em flowing.

[jim-easterbrook]

I've just discovered that the Exif.CanonLe.LensSerialNumber value is getting changed from 0 to 28 when the metadata is written. I treat all maker note values as read-only, so something's definitely awry. Here's an example original file anyway.

[jim-easterbrook]

Experimenting with the v1.0.0 exiv2 command line tool shows different changes to the Canon maker note values depending on which non-Canon tags I set.

[jim-easterbrook]

Here's a minimal example (in Python I'm afraid, but I hope the C++ equivalent is obvious) using libexiv2 compiled from the GitHub repo as it was this morning. The image file had not been modified since copying it from my EOS 100D camera.

Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> im = exiv2.ImageFactory.open('100D_IMG_7197.JPG')
>>> im.readMetadata()
>>> im.exifData()['Exif.Photo.MakerNote'].count()
7898
>>> im.writeMetadata()
>>> im.exifData()['Exif.Photo.MakerNote'].count()
7898
>>> im.readMetadata()
>>> im.exifData()['Exif.Photo.MakerNote'].count()
7872
>>> 

Doing the same with libexiv2 v0.27.5 reduces the MakerNote count to 7874 instead of 7872.

[jim-easterbrook]

Printing out all the exif values (with the exiv2 tool compiled today) shows no difference (other than the maker note binary values) when the file is modified by libexiv2 v0.27.5, but the Exif.CanonLe.0x0007 key is lost when it's modified by libexiv2 v1.0.0.

[jim-easterbrook]

Another test - setting some metadata:

>>> import exiv2
>>> im = exiv2.ImageFactory.open('100D_IMG_7197.JPG')
>>> im.readMetadata()                         
>>> ex = im.exifData()                        
>>> ex['Exif.Image.ImageDescription'] = 'fred'
>>> im.setExifData(ex)
>>> im.writeMetadata()

Comparing the exif values after running this with 0.27.5 and 1.0.0 I get this:

< Exif.CanonLe.LensSerialNumber                SLong       1  0
---
> Exif.CanonLe.LensSerialNumber                SLong       1  28
236d235
< Exif.CanonLe.0x0007                          SLong       0
316c315
< Exif.Photo.InteroperabilityTag               Long        1  9138
---
> Exif.Photo.InteroperabilityTag               Long        1  9136
331c330
< Exif.Image.GPSTag                            Long        1  9168
---
> Exif.Image.GPSTag                            Long        1  9166
337c336
< Exif.Thumbnail.JPEGInterchangeFormat         Long        1  9280
---
> Exif.Thumbnail.JPEGInterchangeFormat         Long        1  9278

I think these are mostly offset changes because of the difference in length of the maker note.

[jim-easterbrook]

I see this bug is present in the 0.28.0 release that's just come out.
(The 0.28.0 release appears to have all the stuff I was expecting to be in v1.0.0 - I'm confused.)

[postscript-dev]

@jim-easterbrook

(The 0.28.0 release appears to have all the stuff I was expecting to be in v1.0.0 - I'm confused.)

When creating a new version from the main branch, it was decided to call it 0.28.0 instead of 1.0.0.

[jim-easterbrook]

I assume this is related to, or even a duplicate of, bug #2746. I'll watch for developments there.

[jim-easterbrook]

Just trying exiv2 command line tool versions 0.27.5 versus 1.00.0.9 (current git main branch) and comparing the new Canon maker note tags with the old I find this inconsistency.

Old version:

Exif.Canon.LightingOpt                       Long        7  28 0 0 0 0 0 1
Exif.Canon.0x4019                            Undefined  30  0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Exif.Canon.AmbienceInfo                      Long        7  28 0 0 0 2147483647 0 1

New version:

Exif.CanonLiOp.0x0000                        SLong       1  28
Exif.CanonLiOp.PeripheralIlluminationCorr    SLong       1  Off
Exif.CanonLiOp.AutoLightingOptimizer         SLong       1  Standard
Exif.CanonLiOp.HighlightTonePriority         SLong       1  Off
Exif.CanonLiOp.LongExposureNoiseReduction    SLong       1  Off
Exif.CanonLiOp.HighISONoiseReduction         SLong       1  Standard
Exif.CanonLiOp.0x0006                        SLong       1  1
Exif.CanonLe.LensSerialNumber                SLong       1  0
Exif.CanonLe.0x0001                          SLong       1  0
Exif.CanonLe.0x0002                          SLong       1  0
Exif.CanonLe.0x0003                          SLong       1  0
Exif.CanonLe.0x0004                          SLong       1  0
Exif.CanonLe.0x0005                          SLong       1  0
Exif.CanonLe.0x0006                          SLong       1  0
Exif.CanonLe.0x0007                          SLong       0  
Exif.CanonAm.0x0000                          SLong       1  28
Exif.CanonAm.AmbienceSelection               SLong       1  Standard
Exif.CanonAm.0x0002                          SLong       1  0
Exif.CanonAm.0x0003                          SLong       1  0
Exif.CanonAm.0x0004                          SLong       1  2147483647
Exif.CanonAm.0x0005                          SLong       1  0
Exif.CanonAm.0x0006                          SLong       1  1

The old Exif.Canon.0x4019 value is 30 bytes of zero, but the corresponding Exif.CanonLe group is eight signed longs, i.e. 32 bytes. Hence the last one, Exif.CanonLe.0x0007 is not read and my maker note block ends up being two bytes shorter than it used to be.

[jim-easterbrook]

The above is with a non-Canon lens. Looking at a different file with a Canon 18-55mm kit lens I find these values of interest:

Exif.CanonLe.LensSerialNumber                SLong       1  422576128
Exif.CanonLe.0x0001                          SLong       1  129
Exif.Photo.LensSerialNumber                  Ascii      11  0000301981

422576128 in hex is 19300000 and 129 in hex is 81. Join those bytes together in the right order and you get 0000301981.

[jim-easterbrook]

I think I've worked out how to fix this bug, just need to add a print function to do the byte to hex conversion for display. Then I'll submit a PR.

[kmilos]

@jim-easterbrook Is the fix confirmed, can we close this now?

[jim-easterbrook]

Yes, I just tried latest 0.28.x branch and it's doing what I expect.

[kmilos]

Thanks, closing then.