Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metadata (Star Rating) not loaded correctly when updated with aperture #2005

Closed
Tom33St opened this issue Nov 28, 2021 · 5 comments
Closed

Metadata (Star Rating) not loaded correctly when updated with aperture #2005

Tom33St opened this issue Nov 28, 2021 · 5 comments
Labels
request feature request or any other kind of wish

Comments

@Tom33St
Copy link

Tom33St commented Nov 28, 2021

Is your feature request related to a problem?

Related to digikam Bug 446183 https://bugs.kde.org/show_bug.cgi?id=446183
I am currently migrating a large repository from Apple Aperture to digikam. In aperture I keep only the jpg, not the raw. Therefore I am writing all metadate in Aperture back into the jpg.
When I scan folders with these jpgs in digikam (which is using exiv2), then the star rating is not loaded (other metadata might be affected as well).
digikam:
digikam.metaengine: Exiv2 ( 3 ) : XMP Toolkit error 201: Error in XMLValidator
digikam.metaengine: Exiv2 ( 2 ) : Failed to decode XMP metadata.

After using exiftool D81_9520.jpg -xmp:all= -tagsfromfile @ -xmp:all -overwrite_original the correct star rating is shown in digikam. But after changing the star rating in Apple Aperture, then again it is not read correctly in digikam.
All other imaging software I tried (Apple Preview, exiftools, Excire, NX Studio)

Demo File:
https://www.dropbox.com/s/oxsmbbcby6w0zhp/D81_9520.jpg?dl=0

Describe the solution you would like

Interpret metadata "more tolerant" to match the behaviour of common sw packages.

Describe alternatives you have considered
Additional context

Add any other information about the feature request here (e.g. screenshots, URLs).
@Tom33St Tom33St added the request feature request or any other kind of wish label Nov 28, 2021
@mqualmann
Copy link

To avoid misunderstandings, only the toolkit error is relevant for Exiv2.
The error was probably caused (2018?) By editing with DxO PhotoLab.
All other problems are with the workflow with other programs.

@clanmills
Copy link
Collaborator

I don't know exactly what's wrong with the metadata. I get the same "validator" error as reported by digiKam. That message comes from the Adobe XMPsdk which is built into Exiv2.

511 rmills@rmillsm1:~/clanmills $ exiv2 -g star/i ~/Downloads/D81_9520.jpg 
Error: XMP Toolkit error 201: Error in XMLValidator
Warning: Failed to decode XMP metadata.
512 rmills@rmillsm1:~/clanmills $

I have extracted and printed the XMP. It is valid XML, however it has a <rdf:Bag/> that looks suspect.

509 rmills@rmillsm1:~/clanmills $ exiv2 -pX ~/Downloads/D81_9520.jpg  | xmllint --pretty 1 -
<?xml version="1.0"?>
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description 
       xmlns:dc="http://purl.org/dc/elements/1.1/" 
       xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
       xmlns:xmp="http://ns.adobe.com/xap/1.0/"
       xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0/"
       rdf:about=""
       photoshop:Country="Singapur"
       photoshop:City="Singapur"
       photoshop:DateCreated="2018-03-16T00:44:05"
       photoshop:State="Singapur"
       photoshop:Urgency="3" 
       xmp:Label="Gelb" 
       xmp:CreateDate="2018-03-16T00:44:05" 
       xmp:Rating="3"
       MicrosoftPhoto:Rating="50">
       <dc:creator>
         <rdf:Seq>
           <rdf:li>T. Franke                           </rdf:li>
         </rdf:Seq>
       </dc:creator>
       <dc:subject>
         <rdf:Bag/>
       </dc:subject>
       <dc:rights>
         <rdf:Alt>
           <rdf:li xml:lang="x-default">Tom Franke 2018                                       </rdf:li>
         </rdf:Alt>
       </dc:rights>
    </rdf:Description>
  </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>

You can write a script to use exiv2 -pX to extract the code, repair it, and re-insert it with exiv2 -iX-

I doubt if anybody in Team Exiv2 will help you with this matter as the damage has been caused by other applications. The good news is that exiv2 provides tools which you use to fix your images.

@clanmills
Copy link
Collaborator

I've done more investigation about this. I don't know what's wrong the the XMP, however I know how you can fix this. Look at the structure of your file:

610 rmills@rmillsm1:~/temp $ exiv2 -pS D81_9520.jpg 
STRUCTURE OF JPEG FILE: D81_9520.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe0 APP0  |      16 | JFIF.....,.,.
      20 | 0xffe1 APP1  |   27160 | Exif..MM.*......................
   27182 | 0xffe1 APP1  |    1233 | http://ns.adobe.com/xap/1.0/.<?x
   28417 | 0xffed APP13 |     258 | Photoshop 3.0.8BIM..........Z...
   28677 | 0xffdb DQT   |      67 
   28746 | 0xffdb DQT   |      67 
   28815 | 0xffc0 SOF0  |      17 
   28834 | 0xffc4 DHT   |      31 
   28867 | 0xffc4 DHT   |     181 
   29050 | 0xffc4 DHT   |      31 
   29083 | 0xffc4 DHT   |     181 
   29266 | 0xffda SOS  
611 rmills@rmillsm1:~/temp $

The XMP is stored in the JPEG APP1 segment that starts at 27182. Let's remove that.

$ dd bs=1 skip=0 count=27182 if=D81_9520.jpg         2>/dev/null  > foo.jpg
$ dd bs=1 skip=28415 count=1000000 if=D81_9520.jpg   2>/dev/null >> foo.jpg

The file now has no XMP data, however the IPTC and EXIF data are good:

615 rmills@rmillsm1:~/temp $ exiv2 -g date/i foo.jpg 
Exif.Image.DateTime                          Ascii      20  2018:03:29 09:20:07
Exif.Photo.DateTimeOriginal                  Ascii      20  2018:03:16 00:44:05
Exif.Photo.DateTimeDigitized                 Ascii      20  2018:03:16 00:44:05
Exif.NikonWt.DateDisplayFormat               Byte        1  D/M/Y
Iptc.Application2.DigitizationDate           Date        8  2018-03-16
Iptc.Application2.DateCreated                Date        8  2018-03-16
616 rmills@rmillsm1:~/temp $

We can add XMP to foo.jpg in the usual ways. For example, I can copy the XMP from Stonehenge.jpg:

628 rmills@rmillsm1:~/temp $ curl -LO --silent http://clanmills.com/Stonehenge.jpg 
629 rmills@rmillsm1:~/temp $ exiv2 -px -g description Stonehenge.jpg 
Xmp.dc.description                           LangAlt     1  lang="x-default" Classic View
630 rmills@rmillsm1:~/temp $ exiv2 -eX- Stonehenge.jpg  | exiv2 -iX- foo.jpg 
631 rmills@rmillsm1:~/temp $ exiv2 -px -g description foo.jpg 
Xmp.dc.description                           LangAlt     1  lang="x-default" Classic View
632 rmills@rmillsm1:~/temp $

As you can see, the JPEG/APP1 segment has been correctly inserted into foo.jpg:

632 rmills@rmillsm1:~/temp $ exiv2 -pS foo.jpg 
STRUCTURE OF JPEG FILE: foo.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe0 APP0  |      16 | JFIF.....,.,.
      20 | 0xffe1 APP1  |   27366 | Exif..MM.*......................
   27388 | 0xffe1 APP1  |    2804 | http://ns.adobe.com/xap/1.0/.<?x
   30194 | 0xffed APP13 |     276 | Photoshop 3.0.8BIM..........Z...
   30472 | 0xffdb DQT   |      67 
   30541 | 0xffdb DQT   |      67 
   30610 | 0xffc0 SOF0  |      17 
   30629 | 0xffc4 DHT   |      31 
   30662 | 0xffc4 DHT   |     181 
   30845 | 0xffc4 DHT   |      31 
   30878 | 0xffc4 DHT   |     181 
   31061 | 0xffda SOS  
633 rmills@rmillsm1:~/temp $

Although I copied the XMP from Stonehenge, you can copy your XML/XMP from anywhere. The exiv2 option -eX- says copy the XMP/XML to std-out:

634 rmills@rmillsm1:~/temp $ exiv2 -eX- Stonehenge.jpg  | xmllint -pretty 1 -
<?xml version="1.0"?>
<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmlns:tiff="http://ns.adobe.com/tiff/1.0/" xmlns:exif="http://ns.adobe.com/exif/1.0/" xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" rdf:about="" xmp:Rating="0" xmp:ModifyDate="2015-07-16T20:25:28+01:00" xmp:CreateDate="2015-07-16T15:38:54.00" tiff:Orientation="1" tiff:YCbCrPositioning="1" tiff:XResolution="300/1" tiff:YResolution="300/1" tiff:ResolutionUnit="2" tiff:Make="NIKON CORPORATION" tiff:Model="NIKON D5300" tiff:Software="Ver.1.00 " exif:ExifVersion="0230" exif:FlashpixVersion="0100" exif:ColorSpace="1" exif:CompressedBitsPerPixel="2/1" exif:PixelXDimension="6000" exif:PixelYDimension="4000" exif:ExposureTime="10/4000" exif:FNumber="100/10" exif:ExposureProgram="0" exif:ExposureBiasValue="0/6" exif:MaxApertureValue="43/10" exif:MeteringMode="5" exif:LightSource="0" exif:FocalLength="440/10" exif:SensingMethod="2" exif:FileSource="3" exif:SceneType="1" exif:CFAPattern="2 0 2 0 0 1 1 2" exif:CustomRendered="0" exif:ExposureMode="0" exif:WhiteBalance="0" exif:DigitalZoomRatio="1/1" exif:FocalLengthIn35mmFilm="66" exif:SceneCaptureType="0" exif:GainControl="0" exif:Contrast="0" exif:Saturation="0" exif:Sharpness="0" exif:SubjectDistanceRange="0" exif:ImageUniqueID="090caaf2c085f3e102513b24750041aa" exif:GPSVersionID="2.3.0.0" exif:GPSLatitude="51,10.6969000N" exif:GPSLongitude="1,49.5984000W" exif:GPSAltitudeRef="0" exif:GPSAltitude="97/1" exif:GPSTimeStamp="2015-07-16T14:38:55.880000000" exif:GPSSatellites="09" exif:GPSMapDatum="WGS-84          " photoshop:DateCreated="2015-07-16T15:38:54.00">
      <exif:ComponentsConfiguration>
        <rdf:Seq>
          <rdf:li>1</rdf:li>
          <rdf:li>2</rdf:li>
          <rdf:li>3</rdf:li>
          <rdf:li>0</rdf:li>
        </rdf:Seq>
      </exif:ComponentsConfiguration>
      <exif:UserComment>
        <rdf:Alt>
          <rdf:li xml:lang="x-default">                                    </rdf:li>
        </rdf:Alt>
      </exif:UserComment>
      <exif:ISOSpeedRatings>
        <rdf:Seq>
          <rdf:li>200</rdf:li>
        </rdf:Seq>
      </exif:ISOSpeedRatings>
      <exif:Flash exif:Fired="False" exif:Return="0" exif:Mode="2" exif:Function="False" exif:RedEyeMode="False"/>
      <dc:description>
        <rdf:Alt>
          <rdf:li xml:lang="x-default">Classic View</rdf:li>
        </rdf:Alt>
      </dc:description>
    </rdf:Description>
  </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>
635 rmills@rmillsm1:~/temp $

What's wrong with the XML/XMP in D81_9520.jpg that cause the Adobe/XMPsdk "validator" error? I don't know. I suggest you ask on the Adobe Forum.

I'm going to close this issue as there is no case for Exiv2 to answer. I wish you good luck in constructing the appropriate scripts to repair the damage inflicted on your images by non-Exiv2 software.

@Tom33St
Copy link
Author

Tom33St commented Nov 28, 2021 via email

@clanmills
Copy link
Collaborator

Not easy dealing with a lot of files. I'm currently re-engineering the whole of my web-site which has 80,000 photos that I've taken in the last 20 years. 2000 albums with stories. https://clanmills.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
request feature request or any other kind of wish
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@clanmills @mqualmann @Tom33St