Import: Extract metadata with Exiftool so that video filepath uses the actual creation date

[nevado]

tl;dr - importing mp4 videos from import/ -> originals/ seems to be ignoring the exif data for determining the new filename in originals/, it's using the file modification time only causing new filename to be incorrect.

I'm importing mp4 video files from import directory to originals directory. Import is being triggered via web interface with the 'move' option selected.

The photos were all correctly moved to year/month/filename structure using exif data to determine when they were taken, but the mp4 videos were all moved into that structure using the file modification times only (ignoring exif). Looking in the logs, I noticed all the mp4 files have messages similar to this one, showing file mod time was used:

photoprism          | time="2021-01-04T00:47:20Z" level=info msg="media: VID_20150414_134951.mp4 was taken at 2021-01-03 22:47:43.712865869 +0000 UTC (file mod time)"
photoprism          | time="2021-01-04T00:47:20Z" level=info msg="import: moving main mp4 file VID_20150414_134951.mp4 to 2021/01/20210103_224743_DB0DA828.mp4"

Note, despite this, the photoprism web interface correctly shows the file as being taken in 2015, but the file has been renamed to a 2021 filename based on the file modification time which is a bit confusing if looking directly at the 'originals' directory.

The exif data for one of the renamed files is as below:

$ exiftool -j 20210103_224743_DB0DA828.mp4
[{
  "SourceFile": "20210103_224743_DB0DA828.mp4",
  "ExifToolVersion": 10.80,
  "FileName": "20210103_224743_DB0DA828.mp4",
  "Directory": ".",
  "FileSize": "93 MB",
  "FileModifyDate": "2021:01:04 00:47:20+00:00",
  "FileAccessDate": "2021:01:04 00:47:20+00:00",
  "FileInodeChangeDate": "2021:01:04 00:47:20+00:00",
  "FilePermissions": "rwxr-xr-x",
  "FileType": "MP4",
  "FileTypeExtension": "mp4",
  "MIMEType": "video/mp4",
  "MajorBrand": "MP4 v2 [ISO 14496-14]",
  "MinorVersion": "0.0.0",
  "CompatibleBrands": ["isom","mp42"],
  "MovieHeaderVersion": 0,
  "CreateDate": "2015:04:14 12:50:57",
  "ModifyDate": "2015:04:14 12:50:57",
  "TimeScale": 1000,
  "Duration": "0:01:03",
  "PreferredRate": 1,
  "PreferredVolume": "100.00%",
  "PreviewTime": "0 s",
  "PreviewDuration": "0 s",
  "PosterTime": "0 s",
  "SelectionTime": "0 s",
  "SelectionDuration": "0 s",
  "CurrentTime": "0 s",
  "NextTrackID": 3,
  "TrackHeaderVersion": 0,
  "TrackCreateDate": "2015:04:14 12:50:57",
  "TrackModifyDate": "2015:04:14 12:50:57",
  "TrackID": 1,
  "TrackDuration": "0:01:03",
  "TrackLayer": 0,
  "TrackVolume": "0.00%",
  "ImageWidth": 1920,
  "ImageHeight": 1080,
  "GraphicsMode": "srcCopy",
  "OpColor": "0 0 0",
  "CompressorID": "avc1",
  "SourceImageWidth": 1920,
  "SourceImageHeight": 1080,
  "XResolution": 72,
  "YResolution": 72,
  "BitDepth": 24,
  "PixelAspectRatio": "65536:65536",
  "VideoFrameRate": 30.32,
  "MatrixStructure": "1 0 0 0 1 0 0 0 1",
  "MediaHeaderVersion": 0,
  "MediaCreateDate": "2015:04:14 12:50:57",
  "MediaModifyDate": "2015:04:14 12:50:57",
  "MediaTimeScale": 48000,
  "MediaDuration": "0:01:03",
  "HandlerType": "Audio Track",
  "HandlerDescription": "SoundHandle",
  "Balance": 0,
  "AudioFormat": "mp4a",
  "AudioChannels": 1,
  "AudioBitsPerSample": 16,
  "AudioSampleRate": 48000,
  "MovieDataSize": 97150822,
  "MovieDataOffset": 405032,
  "AvgBitrate": "12.2 Mbps",
  "ImageSize": "1920x1080",
  "Megapixels": 2.1,
  "Rotation": 90
}]

Using docker version 201231-8e22fbf8-Linux-x86_64

[lastzero]

This is because files are first moved, then indexed. However, PhotoPrism can't natively read video metadata - so the "real" date is unknown and the file system date is used. The indexer later won't move any files, so the "mistake" is not corrected.

For now, it seems best for you to manually copy files to originals if the correct folder with the right date is critical for you. Mobile sync apps often already structure uploads in sub-folders using YEAR/MONTH, so you don't need to use import in this case.

[lastzero]

Note that videos don't use Exif. Exiftool also reads other metadata models & formats, other than its name suggests.

[nevado]

Ah - so I'm curious then, PhotoPrism is correctly identifying the date in the UI, and the sidecar .yml file has picked up the 'TakenAt' date:

$ cat 20210103_224743_DB0DA828.yml
TakenAt: 2015-04-14T12:50:57Z
TakenSrc: meta
UID: pqmdwuz1mf3rr57b
Type: video
Title: Unknown / 2015
Year: 2015
Month: 4
Day: 14
Details:
  Keywords: grey
CreatedAt: 2021-01-04T00:47:23Z
UpdatedAt: 2021-01-04T00:47:25.079137832Z

Shouldn't the 'TakenAt' date be used for the file naming too?

[lastzero]

That's what I mean: The INDEXER uses Exiftool, but the IMPORTER doesn't as importing happens before indexing.

[nevado]

Thank you for your help - one quick question if I may, for the files that are already imported, as there's no delete function - if I simply remove the files from the originals/ directory will PhotoPrism automatically clean up all associated sidecar files and so on? If so, do I need to re-index for that clean up to happen?

[lastzero]

Yes. If you don't delete via WebDAV, you have to manually trigger indexing as there is no way for PhotoPrism to know when you're done.

[lastzero]

You may also move files, PhotoPrism should automatically update the index (again, when you trigger indexing or use WebDAV).

[lastzero]

It's part of our latest release, see https://github.com/photoprism/photoprism/releases/tag/210104-7f9e806a

Let us know if importing works for you as expected!

[nevado]

This works perfectly now, thank you - very impressed with the quick response here and PhotoPrism overall!

[lastzero]

Thank you ❤️ Glad we have such an amazing community!