Add option to keep the sort order of an album

[gerrieg]

In all my albums I sort the photos manually. I have not found a way to export the photos and preserve the sort sequence. Maybe you can add a sequence number in the filename template, sidecar or in the photo metadata?
Or did I miss something?

[RhetTbull]

You haven't missed anything -- this isn't currently possible with osxphotos. osxphotos does know about the album sort order (see #184) and this information is accessible through the osxphotos python package but due to the way the export code is currently implemented it would be difficult to implement this. The main challenges are:

• the directory template (--directory) is evaluated separately from the filename template (--filename). To add sort order, a sequence number of some kind would need to be added to the filename thus the filename template would have to be aware of which directory it was being exported to. This would require a major rewrite to the export logic.
• A photo can be belong to multiple albums so in the Photos (and thus osxphotos) design, it's the album that knows about the sort order, not the photo. But because albums map to directories and not files, the first issue makes it difficult to take advantage of this information
• It would be straightforward to add a template like {album_sort_id} that could be used in --filename but this isn't what I expect you want...e.g your photos would be named album_name_01_img_name.jpg, album_name_02_img_name.jpg, etc. The album name would thus be part of the image name not the not folder name.
• This would further complicate the --update logic as the sort order could change between updates (e.g. you could change it or insert photos into an album). Thus previously exported images would possibly have to be renamed every time --update was run and osxphotos would need to keep a mapping of every image name to the last known sort order.

See also #154

I'll look at the code and think about possible ways to address these issues but no promises on implementing a fix as I do think this might require bigger code changes than I have time to do.

[gerrieg]

How about just writing the sort order index number into the sidecar file. Then I can create the correct order myself with an extra script. That would be enough for me. Is this easier to implement?

[RhetTbull]

@gerrieg I'll look at that but not sure it would be easier, also not sure which field in the sidecar I'd use to do this. The following is a bit hacky (I just threw it together) but it uses osxphotos --post-function to call a custom plug-in function that finds the sort order and writes it to a text file sidecar for each image. If you're comfortable with python, you could easily modify this example to likely do what you want. You could even rename the file inside the function as at that point it'll already be exported and you'll have access to the full path. A couple of caveats:

1. It's possible for multiple albums to have the same name and for a photo to be a member of these duplicate albums. This script makes no attempt to figure out which album is the album in question, it just finds the first album with the name that matches the export directory.
2. If you rename a file inside the script (or in post-processing), --update will not work nor will --cleanup, as osxphotos keeps a database of exported filenames and which photo they map to and this would thus break.
3. This example does no error checking -- if it can't find the right album or the right sort order, it simply does nothing for that particular photo.
4. This post-function script assumes you're using either the {album} or {folder_album} template for the --directory option.

Save the following as album_sort_order.py and run it like this (obviously inserting any other options you're using):

osxphotos export /path/to/export -V --directory {folder_album} --post-function album_sort_order.py::album_sort_order

The post-function will create a text file named like image_name.jpeg_sort_order.txt which will contain a single number (starting at 0) which is the album sort order for that image.

""" Example function for use with osxphotos export --post-function option showing how to record album sort order """

import pathlib
from typing import Optional

from osxphotos import ExportResults, PhotoInfo
from osxphotos.albuminfo import AlbumInfo


def _get_album_sort_order(album: AlbumInfo, photo: PhotoInfo) -> Optional[int]:
    """Get the sort order of photo in album
    
    Returns: sort order as int or None if photo not found in album
    """
    # get the album sort order from the album_info
    sort_order = 0  # change this to 1 if you want counting to start at 1
    for album_photo in album.photos:
        if album_photo.uuid == photo.uuid:
            # found the photo we're processing
            break
        sort_order += 1
    else:
        # didn't find the photo, so skip this file
        return None
    return sort_order


def album_sort_order(
    photo: PhotoInfo, results: ExportResults, verbose: callable, **kwargs
):
    """Call this with osxphotos export /path/to/export --post-function post_function.py::post_function
        This will get called immediately after the photo has been exported

    Args:
        photo: PhotoInfo instance for the photo that's just been exported
        results: ExportResults instance with information about the files associated with the exported photo
        verbose: A function to print verbose output if --verbose is set; if --verbose is not set, acts as a no-op (nothing gets printed)
        **kwargs: reserved for future use; recommend you include **kwargs so your function still works if additional arguments are added in future versions

    Notes:
        Use verbose(str) instead of print if you want your function to conditionally output text depending on --verbose flag
        Any string printed with verbose that contains "warning" or "error" (case-insensitive) will be printed with the appropriate warning or error color
        Will not be called if --dry-run flag is enabled
        Will be called immediately after export and before any --post-command commands are executed
    """

    # ExportResults has the following properties
    # fields with filenames contain the full path to the file
    # exported: list of all files exported
    # new: list of all new files exported (--update)
    # updated: list of all files updated (--update)
    # skipped: list of all files skipped (--update)
    # exif_updated: list of all files that were updated with --exiftool
    # touched: list of all files that had date updated with --touch-file
    # converted_to_jpeg: list of files converted to jpeg with --convert-to-jpeg
    # sidecar_json_written: list of all JSON sidecar files written
    # sidecar_json_skipped: list of all JSON sidecar files skipped (--update)
    # sidecar_exiftool_written: list of all exiftool sidecar files written
    # sidecar_exiftool_skipped: list of all exiftool sidecar files skipped (--update)
    # sidecar_xmp_written: list of all XMP sidecar files written
    # sidecar_xmp_skipped: list of all XMP sidecar files skipped (--update)
    # missing: list of all missing files
    # error: list tuples of (filename, error) for any errors generated during export
    # exiftool_warning: list of tuples of (filename, warning) for any warnings generated by exiftool with --exiftool
    # exiftool_error: list of tuples of (filename, error) for any errors generated by exiftool with --exiftool
    # xattr_written: list of files that had extended attributes written
    # xattr_skipped: list of files that where extended attributes were skipped (--update)
    # deleted_files: list of deleted files
    # deleted_directories: list of deleted directories
    # exported_album: list of tuples of (filename, album_name) for exported files added to album with --add-exported-to-album
    # skipped_album: list of tuples of (filename, album_name) for skipped files added to album with --add-skipped-to-album
    # missing_album: list of tuples of (filename, album_name) for missing files added to album with --add-missing-to-album

    for filepath in results.exported:
        # do your processing here
        filepath = pathlib.Path(filepath)
        album_dir = filepath.parent.name
        if album_dir not in photo.albums:
            return

        # get the first album that matches this name of which the photo is a member
        album_info = None
        for album in photo.album_info:
            if album.title == album_dir:
                album_info = album
                break
        else:
            # didn't find the album, so skip this file
            return

        sort_order = _get_album_sort_order(album_info, photo)
        if sort_order is None:
            # didn't find the photo, so skip this file
            return
        
        verbose(f"Sort order for {filepath} in album {album_dir} is {sort_order}")
        with open(str(filepath) + "_sort_order.txt", "w") as f:
            f.write(str(sort_order))

[RhetTbull]

This --post-function template got me thinking...it would be possible to pass the the destination path to the render engine in RenderOptions just as {export_dir} is passed and then pass this to a {function} template and the same logic used in this --post-function example could be used as a custom template to name the file accordingly at the time of export thus --update and --cleanup would work (assuming album sort order hadn't changed). This would be subject to the same caveats and limitations as the --post-function example but would get you pretty close to what you want. This is something I can implement fairly easy.

This is where the dest_path would need to passed:

osxphotos/osxphotos/cli.py

Line 2844 in e40ecc4

render_options = RenderOptions(export_dir=export_dir)

Once implemented you could then do something like this:

osxphotos export /path/to/export --directory "{folder_album} --filename "{function:/add_album_sort_order.py::album_sort_order}"

[gerrieg]

Thanks for your efforts, I will try it out soon.

[RhetTbull]

@gerrieg I think I have a better solution implemented now. In v0.42.64, I've changed the code so that template functions (custom user-defined plug-ins that work with the template system) can get access to the export path when used with the --filename template. This means you can define a custom template function that looks to see if the photo is being exported into a folder that matches its album name and if so, return the sequence (index) of the photo in the album for use in the --filename template. If you save the code below (also in the examples folder) as album_sort_order.py, you can call the custom template function like this:

osxphotos export /path/to/export -V --directory "{folder_album}" --filename "{album?{function:examples/album_sort_order.py::album_sequence}_,}{original_name}"

This only works when the directory ends with the {folder_album} template which outputs the full folder/album path but you could easily modify it to work with the {album} template (album title only, not folder path) or any other custom --directory template you are using.

The --filename template above reads:

• {album? : is the folder in an album?
• {function:examples/album_sort_order.py::album_sequence}_ : if so, call our custom album_sequence function and prepend the value returned (the index of the photo in the album) and _ to the filename.
• ,} : if not in an album, use no value (prepend nothing)
• {original_name} : finally, append the original filename.

Of course, this suffers the limitation that if you have two albums of the same name in the same folder, they'll be treated as a single folder upon export (but this is true anyway for osxphotos as the filesystem does not allow items with duplicate names to be created). Let me know if you have any questions -- I believe this will do what you originally asked for in this issue. I'll take a look at adding an {album_sequence} template now that these changes are made.

Implementation note: the code originally evaluated the filename template then the directory template as I'd assumed filename wouldn't change but the same file could be exported to multiple directories based on value of --directory template. In this fix, I switched the order to evaluate directory first then filename -- this means that in the normal use case, filename template is evaluated repeatedly even though the value won't change. This is a small performance penalty but it's the only way to enable the behavior requested in this issue.

Note: this example contains both the album_sequence plug-in for template function as well as the original album_sort_order post function for writing the sidecar. Only the function(s) specified in the osxphotos command line will be called.

The example album_sequence function returns sequence starting at 0 so your photos in this example will be named

album_name/0_img_123.jpg
album_name/1_img_234.jpg
album_name/2_img_345.jpg
...and so on

If you want to start sequence at 1 (or any other number), you can do so by setting the OSXPHOTOS_ALBUM_SEQUENCE_START environment variable to the start index. For example:

OSXPHOTOS_ALBUM_SEQUENCE_START=1 osxphotos export /path/to/export -V --directory "{folder_album}" --filename "{album?{function:examples/album_sort_order.py::album_sequence}_,}{original_name}"

and your files will be named:

album_name/1_img_123.jpg
album_name/2_img_234.jpg
album_name/3_img_345.jpg
...and so on

""" Example function for use with osxphotos export --post-function option showing how to record album sort order """

import os
import pathlib
from typing import Optional

from osxphotos import ExportResults, PhotoInfo
from osxphotos.albuminfo import AlbumInfo
from osxphotos.path_utils import sanitize_dirname
from osxphotos.phototemplate import RenderOptions


def _get_album_sort_order(album: AlbumInfo, photo: PhotoInfo) -> Optional[int]:
    """Get the sort order of photo in album

    Returns: sort order as int or None if photo not found in album
    """
    # get the album sort order from the album_info
    sort_order = 0  # change this to 1 if you want counting to start at 1
    for album_photo in album.photos:
        if album_photo.uuid == photo.uuid:
            # found the photo we're processing
            break
        sort_order += 1
    else:
        # didn't find the photo, so skip this file
        return None
    return sort_order


def album_sequence(photo: PhotoInfo, options: RenderOptions, **kwargs) -> str:
    """Call this with {function} template to get album sequence (sort order) when exporting with {folder_album} template

    For example, calling this template function like the following prepends sequence#_ to each exported file if the file is in an album:

    osxphotos export /path/to/export -V --directory "{folder_album}" --filename "{album?{function:examples/album_sort_order.py::album_sequence}_,}{original_name}"

    The sequence will start at 0.  To change the sequence to start at a different offset (e.g. 1), set the environment variable OSXPHOTOS_ALBUM_SEQUENCE_START=1 (or whatever offset you want)
    """
    dest_path = options.dest_path
    if not dest_path:
        return ""

    album_info = None
    for album in photo.album_info:
        # following code is how {folder_album} builds the folder path
        folder = "/".join(sanitize_dirname(f) for f in album.folder_names)
        folder += "/" + sanitize_dirname(album.title)
        if dest_path.endswith(folder):
            album_info = album
            break
    else:
        # didn't find the album, so skip this file
        return ""
    start_index = int(os.getenv("OSXPHOTOS_ALBUM_SEQUENCE_START", 0))
    return str(album_info.photo_index(photo) + start_index)


def album_sort_order(
    photo: PhotoInfo, results: ExportResults, verbose: callable, **kwargs
):
    """Call this with osxphotos export /path/to/export --post-function post_function.py::post_function
        This will get called immediately after the photo has been exported

    Args:
        photo: PhotoInfo instance for the photo that's just been exported
        results: ExportResults instance with information about the files associated with the exported photo
        verbose: A function to print verbose output if --verbose is set; if --verbose is not set, acts as a no-op (nothing gets printed)
        **kwargs: reserved for future use; recommend you include **kwargs so your function still works if additional arguments are added in future versions

    Notes:
        Use verbose(str) instead of print if you want your function to conditionally output text depending on --verbose flag
        Any string printed with verbose that contains "warning" or "error" (case-insensitive) will be printed with the appropriate warning or error color
        Will not be called if --dry-run flag is enabled
        Will be called immediately after export and before any --post-command commands are executed
    """

    # ExportResults has the following properties
    # fields with filenames contain the full path to the file
    # exported: list of all files exported
    # new: list of all new files exported (--update)
    # updated: list of all files updated (--update)
    # skipped: list of all files skipped (--update)
    # exif_updated: list of all files that were updated with --exiftool
    # touched: list of all files that had date updated with --touch-file
    # converted_to_jpeg: list of files converted to jpeg with --convert-to-jpeg
    # sidecar_json_written: list of all JSON sidecar files written
    # sidecar_json_skipped: list of all JSON sidecar files skipped (--update)
    # sidecar_exiftool_written: list of all exiftool sidecar files written
    # sidecar_exiftool_skipped: list of all exiftool sidecar files skipped (--update)
    # sidecar_xmp_written: list of all XMP sidecar files written
    # sidecar_xmp_skipped: list of all XMP sidecar files skipped (--update)
    # missing: list of all missing files
    # error: list tuples of (filename, error) for any errors generated during export
    # exiftool_warning: list of tuples of (filename, warning) for any warnings generated by exiftool with --exiftool
    # exiftool_error: list of tuples of (filename, error) for any errors generated by exiftool with --exiftool
    # xattr_written: list of files that had extended attributes written
    # xattr_skipped: list of files that where extended attributes were skipped (--update)
    # deleted_files: list of deleted files
    # deleted_directories: list of deleted directories
    # exported_album: list of tuples of (filename, album_name) for exported files added to album with --add-exported-to-album
    # skipped_album: list of tuples of (filename, album_name) for skipped files added to album with --add-skipped-to-album
    # missing_album: list of tuples of (filename, album_name) for missing files added to album with --add-missing-to-album

    for filepath in results.exported:
        # do your processing here
        filepath = pathlib.Path(filepath)
        album_dir = filepath.parent.name
        if album_dir not in photo.albums:
            return

        # get the first album that matches this name of which the photo is a member
        album_info = None
        for album in photo.album_info:
            if album.title == album_dir:
                album_info = album
                break
        else:
            # didn't find the album, so skip this file
            return

        sort_order = _get_album_sort_order(album_info, photo)
        if sort_order is None:
            # didn't find the photo, so skip this file
            return

        verbose(f"Sort order for {filepath} in album {album_dir} is {sort_order}")
        with open(str(filepath) + "_sort_order.txt", "w") as f:
            f.write(str(sort_order))

[RhetTbull]

Also, this is only guaranteed to work if the album has custom sort order (e.g. you manually arranged the order). If you use the View | Sort | Keep Sorted by Oldest First etc, osxphotos doesn't track the sort order. (See #497)

[RhetTbull]

The album sort order is now correct for albums sorted with "View | Sort" as of v0.42.65

[RhetTbull]

In version 0.42.67, I've added {album_seq} and {folder_album_seq} templates which render to the sequence (index) of a photo in an album. These only work if {album} or {folder_album} is used in the --directory.

For example, the following command exports photos in the folder/album structure and names photos in albums in format "1_filename.jpg", "2_filename.jpg", etc. If a photo is not in an album, it is named with the original filename as normal:

osxphotos export ~/Desktop/export -V --filename "{album?{folder_album_seq.1}_,}{original_name}" --directory "{folder_album}"

[gerrieg]

Many thanks for the very fast implementation! 👍