Exiftool: Large File Support

[BetaAthe]

Hi. Doing some testing, I am unable to play large videos on my Photoprism instance. With large videos, I mean close to 3GB .mp4 avc1 encoded videos.

I have a folder with more than 10 videos, all encoded the same way. All the videos with size < 2.2 GB are playable, and all of them with size > 2.9GB are not. I don't have any video in the 2.2-2.9 range.

When I try to play that videos, ffmpeg kicks in consuming all the cpu of my server. Disabling ffmpeg transcoding makes the video unplayable.

Also, I saw that those video's metadata was wrong, so digging in a little found out that it was a problem with exiftools, solved by writting

%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);

into the .ExifTool_config file in the /root directory.

[rickysarraf]

Speaking for myself, I have preferred to re-encode my video files, carefully choosing a setting, which keeps the video file's size substantially low. The reason for doing so is simply because those files are rarely to be viewed on a large display. My expected clients of PhotoPrism, in all probability, will mostly be PC, Tablet and Mobile. For these device form factors, I don't really need video files with very high bit rate. But this is purely my personal choice.

File sizes greater than 2 GiB are ignored by PP. I think it logs that information in its system logs.

[BetaAthe]

Thanks for the response! These videos are more than 3 hours long, as they are VHS digitalizations, and I think they are already compressed to an acceptable threshold. I understand that one possibility is to split them into two or more files, but I would prefer to not resort to that.

To be able to upload them to PhotoPrism, I had to add the PHOTOPRISM_ORIGINALS_LIMIT: 5000 variable.

As I saw that 2.2 GiB videos are working flawlessly, I thought it was a matter of changing one setting or removing one hard limit from the app.

[lastzero]

As the name suggests, we've designed PhotoPrism for still pictures and short clips like live photos. It MAY not be the best tool for managing full-length movies, especially if you have a lot of them.

Increasing PHOTOPRISM_ORIGINALS_LIMIT to the max file size in MB and then restarting your instance should work though. If your videos are not encoded with MP4 / AVC1, we recommend you transcode them upfront by running photoprism convert in a terminal to avoid high server load and long delays when playing them in the UI.

[lastzero]

Please close this issue if your question is answered with that :)

[BetaAthe]

Hi! I've added that variable and restarted the instance to be able to index them in the library. All the videos are encoded using the same preset, so they are basically the same. Also, I checked them using exiftools and all of them reported the same codec, AVC1. In the PhotoPrism GUI, large playable videos report "AVC1" and the very large unplayable ones report "MP4", for some reason.

I know that PhotoPrism was not made for large videos in mind, and can understand that this issue could be marked as wontfix because of that, but as I saw that this software is able to reproduce some large videos without trouble, I considered this case a bug as, I think, nothing prevents these very large videos for playing properly.

[lastzero]

PhotoPrism uses exiftool to detect the codec, so it should show AVC1 if exiftool does. Did you try a full rescan of the affected folders (don't need to rescan your complete library)?

[BetaAthe]

Exiftools wasn't able to open this video files at first. I had to follow this forum post to being able to read them [1]. Once done, exiftools read the codec correctly. I've done the same inside the PhotoPrism Docker container but the reported codec is still MP4 in the PhotoPrism web interface after the rescan. Thumbnails are created though.

[lastzero]

Then it's probably because we didn't enable large file support in exiftool. Imagine this could lead to memory issues or DOS attacks.

[BetaAthe]

Ok, got it. I understand your concerns about memory or DOS but, is there a way to tell PhotoPrism to use exiftool with large files support?

[lastzero]

Need to change this in our code as the parameters can not be configured, there are already a ton of config options nobody understands.