Store metadata in the filesystem

[lastzero]

In order to be able to copy the archive or restore database contents after a problem, it would be very convenient to store a copy of the metadata in the filesystem right next to the photos they belong to.

Contents need to be time stamped and fingerprinted in order to avoid sync/merge issues.

YAML, JSON or XML? Are there existing standard formats?

This way we could also store the original filename during import (simply write it to this file).

[eliasp]

What about writing those kind of metadata in a way where they aren't detached from the actual file, e.g. FDO CommonExtendedAttributes on Linux/OS X (see also: https://gist.github.com/gwpl/a00e18bc150f1e3518335d13c5e46ad7), NTFS Alternate Data Streams on Windows?

[lastzero]

You should be able to create a long-term backup and edit all file using regular tools (in this case: text editor). Wouldn't work with filesystem specific metadata, right?

[eliasp]

Unfortunately yes :(

[lastzero]

That's actually an issue with most modern software out there... primary storage is a database, ideally somewhere in the cloud, some distributed NoSQL bleeding edge technology... Works great in terms of scalability (when running Web apps with millions of users), but how can regular users create a reliable backup from that? That they can use/read in 20 or 40 years? Photos are not just for the moment.

[husayt]

This is very important. I have stumbled on this issue so many times with Picasa. Had all my metadata overwritten if by mistake forgot to have a harddrive with photos connected. Metadata needs to be next to images and in some readable/editable format like json/xml.

[tendant]

This is a great idea. Text file has a lot of benefits for backup purpose. Format like YAML can be easily tracked in version control system like git, most of editing conflict can be resolved automatically. It can be edited using text editor when needed.

Storing metadata in original image file is nice to have, it serves different purpose though. For backup purpose, it is better not to modify original file, so that original file can be de-duplicated using hash.

[lastzero]

Darktable docs for sidecar files: https://www.darktable.org/usermanual/en/sidecar_files.html

[lastzero]

Created a wiki page for XMP sidecar files: https://github.com/photoprism/photoprism/wiki/XMP

[splitbrain]

Personally I'd prefer to have the meta data written back to the image file (as XMP/IPTC/Exif tags). That way the info is not lost when the image is moved or copied somewhere else (and reimported in a different instance).

[lastzero]

@splitbrain That way, the md5/sha fingerprint would change, which makes it much more difficult to detect duplicate imports. Also XMP files are human readable, other than binary data inside image files.

[lastzero]

Still think we should do this, maybe soon. Using XMP seems difficult and we can't write data back to XMP files yet: https://github.com/photoprism/photoprism/wiki/XMP

[jagot]

I would vote for TOML

[lastzero]

Implemented. Please open a new issue if you find bugs!