Map SearchInfo, PersonInfo, labels etc. to EXIF

[ZarK]

OSXPhotos is fantastic! Love it and it already does everything I need... but seeing how powerful it is, I now want more. I see that if I use the command line to export all info for a photo, I get all the good bits that Apple Photos has added as well, like information about faces and persons, machine analysed content etc.

But how do I map that to some good Exif-structure so that I can export it with the photos? Is there a good standard to follow that at least some software will recognize and can utilise?

[RhetTbull]

I'm glad osxphotos is useful for you! With the exception of the SearchInfo data, all other metadata is written to the appropriate EXIF/IPTC/XMP metadata fields when you use --exiftool or --sidecar. The SearchInfo is not written for because 1) it is added by Photos and not the user so it may contain info the user doesn't want in the photos and 2) there's no standard mapping as you've surmised.

Labels map easily to keywords and you can set this with --keyword-template "{label}" or --keyword-template "{label_normalized}" if you want "normalized" (lowercase) label names.

It might be helpful to have season and some other search info in the keywords as well. You can also add this:

--keyword-template "{label_normalized}" --keyword-template "{photo.search_info.season}" --keyword-template "{photo.search_info.holidays}"

to include seasons and holidays. The names will be capitalized (first letter) so if you want all lower case you could do this:

--keyword-template "{photo.search_info.season|lower}" --keyword-template "{photo.search_info.holidays|lower}"

For other search info fields like venue, etc. I don't know there's a great mapping. You could use Description which is written with --description-template. There can be only one description so you need to think about how to construct the template so you don't overwrite anything already in the description.

For example:

--description-template "{descr?{descr}{newline},}{photo.search_info.season}{newline}{photo.search_info.activities,}{newline}{photo.search_info.venues,}"

You can use osxphotos template for experimenting with different templates.

Keep in mind that some of this metadata changes as Apple updates the machine learning models which would result in all or much of your library being re-exported each time the metadata changes. Sonoma will add many more labels.

The metadata fields currently written by osxphotos are:

EXIF:ImageDescription (may include template)
XMP:Description (may include template)
XMP:Title
IPTC:ObjectName
XMP:TagsList (may include album name, person name, or template)
IPTC:Keywords (may include album name, person name, or template)
IPTC:Caption-Abstract
XMP:Subject (set to keywords + persons)
XMP:PersonInImage
EXIF:GPSLatitudeRef, EXIF:GPSLongitudeRef
EXIF:GPSLatitude, EXIF:GPSLongitude
EXIF:GPSPosition
EXIF:DateTimeOriginal
EXIF:OffsetTimeOriginal
EXIF:ModifyDate
IPTC:DateCreated
IPTC:TimeCreated
QuickTime:CreationDate
QuickTime:ContentCreateDate
QuickTime:CreateDate (UTC)
QuickTime:ModifyDate (UTC)
QuickTime:GPSCoordinates
UserData:GPSCoordinates
XMP:Rating
XMP:RegionAppliedToDimensionsW
XMP:RegionAppliedToDimensionsH
XMP:RegionAppliedToDimensionsUnit
XMP:RegionName
XMP:RegionType
XMP:RegionAreaX
XMP:RegionAreaY
XMP:RegionAreaW
XMP:RegionAreaH
XMP:RegionAreaUnit
XMP:RegionPersonDisplayName

See also IPTC Specification and XMP Specification for what other fields are available. Note that osxphotos does not provide a way to access other fields when writing with --exiftool. I have an issue open (#243) to add a --exiftool-template option that would let you specify any field with any template value. For example, you could then do something like: --exiftool-template "IPTC:City" "{photo.search_info.city}" but not sure when I'll get to implementing that.

[ZarK]

Thanks for the answer! Looks great.

I should have mentioned that I am using the Python API. I see now there are documentation on ExifTool here: https://github.com/RhetTbull/osxphotos/blob/main/API_README.md#exiftool-1. However, it seems to mostly cover reading info. Do I have to explicitly, programatically set each field or is there some way to have it write everything but the SearchInfo through the API as well?

Yeah, I can see the issue with updates of labels. It will be an interesting conundrum. I have previously used cloudpd to export all my photos, which I can no longer do after I turned on Advanced Data Protection. Now with OSXPhotos and the additional power of more metadata I might just re-export / update everything anyway.

[RhetTbull]

Glad to hear you're using the API. I originally wrote OSXPhotos as a tool for other developers to write their own code. However, the CLI has grown to be so flexible I find it's rarely needed and most users just want to the use the CLI. The export command for example, with --post-function and --post-command, allows you to customize the export while still taking advantage of all the export logic which is non-trivial. So I'm curious why you're writing your own export code? Are you using PhotoExporter because if you are, you can have it write the EXIF data. See ExportOptions().exiftool.

I'm in the middle of a refactoring of some of the export code to implement a push-exif command (#160) and it will be much easier when that's done as I've added a new ExifWriter class that reads the metadata from the database and writes it out to the file with exiftool and this can be customized with templates.

Do I have to explicitly, programmatically set each field or is there some way to have it write everything but the SearchInfo through the API as well?

You can use PhotoExporter()._exiftool_dict() to get the dictionary of data with all the fields set correctly (including any templates rendered) then write it with PhotoExporter(). _write_exif_data(). (You should be able to use these even if not using the PhotoExporter to do the export as you can pass in the file to write). These are both private methods that are going to break (actually be removed) in the next release with the refactor I'm using but there will be similar methods in a new ExifWriter class. The branch in question is here if you want to take a look at the work in progress. The refactoring will separate the code that writes sidecars into a SidecarWriter class and the code that writes the EXIF data into ExifWriter class instead of having it all inside PhotoExporter now. That will be better for people implementing their own exporters because the export, the sidecars, and the EXIF writing can all be done independently.

osxphotos/osxphotos/photoexporter.py

Lines 1762 to 1804 in 1bdd993

	def _exiftool_dict(
	self,
	options: t.Optional[ExportOptions] = None,
	filename: t.Optional[str] = None,
	):
	"""Return dict of EXIF details for building exiftool JSON sidecar or sending commands to ExifTool.
	Does not include all the EXIF fields as those are likely already in the image.
	Args:
	options (ExportOptions): options for export
	filename (str): name of source image file (without path); if not None, exiftool JSON signature will be included; if None, signature will not be included
	Returns: dict with exiftool tags / values
	Exports the following:
	EXIF:ImageDescription (may include template)
	XMP:Description (may include template)
	XMP:Title
	IPTC:ObjectName
	XMP:TagsList (may include album name, person name, or template)
	IPTC:Keywords (may include album name, person name, or template)
	IPTC:Caption-Abstract
	XMP:Subject (set to keywords + persons)
	XMP:PersonInImage
	EXIF:GPSLatitudeRef, EXIF:GPSLongitudeRef
	EXIF:GPSLatitude, EXIF:GPSLongitude
	EXIF:GPSPosition
	EXIF:DateTimeOriginal
	EXIF:OffsetTimeOriginal
	EXIF:ModifyDate
	IPTC:DateCreated
	IPTC:TimeCreated
	QuickTime:CreationDate
	QuickTime:ContentCreateDate
	QuickTime:CreateDate (UTC)
	QuickTime:ModifyDate (UTC)
	QuickTime:GPSCoordinates
	UserData:GPSCoordinates
	XMP:Rating
	Reference:
	https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata-201610_1.pdf
	"""

Using ExifTool as a context manager avoids multiple writes to the file as exiftool doesn't run until the context exits:

osxphotos/osxphotos/photoexporter.py

Lines 1736 to 1760 in 1bdd993

	def _write_exif_data(self, filepath: str, options: ExportOptions):
	"""write exif data to image file at filepath
	Args:
	filepath: full path to the image file
	Returns:
	(warning, error) of warning and error strings if exiftool produces warnings or errors
	"""
	if not os.path.exists(filepath):
	raise FileNotFoundError(f"Could not find file {filepath}")
	exif_info = self._exiftool_dict(options=options)
	with ExifTool(
	filepath,
	flags=options.exiftool_flags,
	exiftool=self.photo._exiftool_path,
	) as exiftool:
	for exiftag, val in exif_info.items():
	if type(val) == list:
	for v in val:
	exiftool.setvalue(exiftag, v)
	else:
	exiftool.setvalue(exiftag, val)
	return exiftool.warning, exiftool.error

[ZarK]

So I'm curious why you're writing your own export code?

I mean, most likely just because I didn't read the CLI documentation well enough!

But there were a few cases I wanted to handle during export.

1. At some previous import to Apple Photos, I had added a lot of metadata to the filenames directly, so some files would now be named resolution_address_keyword_IMGxxxx.ext or something like that. I am now extracting the original IMGxxx if it exists for these cases and discards the rest of the custom name.
2. I want to keep the same folder structure and naming scheme for edits / Live Photos / originals that iCloudPD used.
3. I want to be able to just run it over the last 30 days or so and have it handle all duplicates. It might be I'm away on vacation or some system is down so that for some reason it has not run for a couple of weeks. Or it might run every night. In either case, I want it to just run and deal with it without creating duplicates.
   So when the filename I'm trying to export already exist in the location:
   • For supported image formats, I do a duplication check on the image content, ignoring metadata
   • For other formats and movies, I do a duplication check using file hash
4. Probably just because I didn't read the documentation properly, my export command using the API set the current timestamp as the last modified date, while I wanted the original last modified date to be retained, so I set that myself after the export (and handle this for exported Live Photos, edited photos, duplicate photos etc.)
5. I write a small report / log after each export where I can get a notification through automation if there were any issues
6. I needed a script anyway to deal with some other stuff, like making sure my NAS volume is properly mounted before the export

Thank you for your examples! They will for sure get me on the right track if I continue using the API.

[RhetTbull]

I'm always happy for feedback on the API design and documentation so feel free to let me know if something's not clear (or send a PR!).

You should be able to do everything you listed using the CLI (with exception of the NAS setup etc).

I am now extracting the original IMGxxx if it exists for these cases and discards the rest of the custom name.

--cleanup will delete any files that are not part of the export set (like these old naming scheme files)

See also --filename

I want to keep the same folder structure and naming scheme for edits / Live Photos / originals that iCloudPD used.

• See --directory

I want to be able to just run it over the last 30 days or so and have it handle all duplicates.

--update --added-in-last "30 days"

while I wanted the original last modified date to be retained

--touch-file

I write a small report / log after each export where I can get a notification through automation if there were any issues

--report : output can be CSV, JSON, or SQLite

You're of course welcome to write your own export code (that's why I expose the API)! However, the osxphotos export CLI handles these cases and many more and it handles lots of edge cases during export. For example, if the photo didn't change in Photos but the metadata did, you need to know that exiftool should be run again. The CLI keeps a copy of the metadata (as it would be rendered using any templates you have) so it can determine whether exiftool should be run or not (that's an expensive operation that shouldn't be done unless needed). The data is stored in a SQLite file called .osxphotos_export.db in the export directory. There's a CLI osxphotos exportdb for querying the database etc.

[ZarK]

From my brief testing, OSXPhotos would often create duplicates when ran over the same time period multiple times, but might just be because it is not handled by default and I should have added some update boolean.

--cleanup would be something dramatically different than what I'm doing. I'm just extracting the old, original filename from a longer filename, but still exports the file.

[RhetTbull]

From my brief testing, OSXPhotos would often create duplicates when ran over the same time period multiple times, but might just be because it is not handled by default and I should have added some update boolean.

You need to use --update to rerun an export if you don't want duplicates. Osxphotos will never overwrite files unless explicitly specified.