Live Photos: Add support for Samsung Motion Photos

[ark-]

Initial support for Samsung Motion Photos is now available for testing on our public demo at https://demo.photoprism.app/ and/or with the development preview build:

• https://docs.photoprism.app/release-notes/#development-preview
• https://docs.photoprism.app/getting-started/updates/#development-preview

Please do not upload any private pictures to our demo, as everyone can see them there. Thank you very much! 🙏

---

Support for Samsung Motion Photos could be implemented similar to Apple Live Photos:

• Support for Apple Live Photos #177

The videos are stored in the exif data under the following tags:

EmbeddedVideoType: MotionPhoto_Data
EmbeddedVideoFile: <Binary File>

Where EmbeddedVideoFile is an MP4 file with its own exif data and is of type video/mp4. Version MP4 v2 [ISO 14496-14], compressor avc1.

I'm sure EmbeddedVideoType could be something other than MotionPhoto_Data for other Android phone manufacturers as I imagine "MotionPhoto" is a trademark name. Unfortunately I do not know the details of the other devices but some research could make this improvement cover other manufacturer's phones too.

I found discussion and exiftool commands to extract this information here: https://forum.xda-developers.com/android/software/samsung-motion-photo-extractor-t3339997

Example commands:

exiftool -EmbeddedVideoFile -b -w _MotionPhoto.mp4 *.jpg

exiftool -trailer:all= -if "$EmbeddedVideoType eq 'MotionPhoto_Data'" *.jpg

[lastzero]

Samsung embeds video files in JPEGs, is that correct?

Extracting makes sense if this is a common file type and Samsung still uses it... seems to be a hack on the first sight, so I wouldn't be surprised if they start(ed) using a different solution like Apple e.g. storing multiple files.

As a workaround, you can probably write a script to "manually" extract videos before indexing (or maybe you already did so)?

[lastzero]

Anyhow, it would be good to get example files for testing! If we don't have time now, then maybe later.

[ark-]

Hmmm, the script would be useful for a one time import but if I'm auto uploading from my phone that's not ideal. BUT it's a workable solution. I may do this to see how it looks in the UI.

Let me ask around some people I know with newer Samsung phones and get sample photos from them. Also if others realise they could use this feature maybe some thumbs up on this suggestion would help show a need.

I agree that if this format is only used for just the one phone generation it's definitely a low priority. I'll update my top post with a sample image in a few days, I don't have any to hand right now.

[libeanim]

@lastzero Here an example picture.
Not sure if Samsung considers this a hack, as they seem to store the motion photo in the metadata for 2+ years now. (The example picture was taken with a Galaxy S10 and I have pics from an S8 where it is stored in the same way.)

I'd suggest, instead of extracting and storing the video at import, I'd rather read it from the metadata every time when it's needed. Don't think that has a big performance impact as it will be loaded through the image anyway. I guess it makes a big difference in terms of storage space needed if the video is in metadata and on disc simultaneously. (And ideally, we don't want to delete it from the metadata automatically).

Metadata:

ExifTool Version Number         : 11.88
File Size                       : 6.9 MB
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Make                            : samsung
Camera Model Name               : SM-G973F
[...]
Micro Video                     : 1
Micro Video Version             : 1
Micro Video Offset              : 4535831
Micro Video Presentation Timestamp Us: -1
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Embedded Video Type             : MotionPhoto_Data
Embedded Video File             : (Binary data 4535775 bytes, use -b option to extract)
Aperture                        : 2.4
Image Size                      : 4032x3024
Megapixels                      : 12.2
Light Value                     : 13.9

[lastzero]

Thanks for sharing an example! I was able to extract the embedded MP4 / AVC1 video using the following command:

exiftool -b -EmbeddedVideoFile samsung-motion.jpg > samsung-motion.mp4

After running Exiftool to generate JSON sidecar files, we may extract the video file next if EmbeddedVideoFile exists:

  "TimeStamp": "2020:04:23 19:33:41+02:00",
  "EmbeddedVideoType": "MotionPhoto_Data",
  "EmbeddedVideoFile": "(Binary data 4535775 bytes, use -b option to extract)",
  "Aperture": 2.4,
  "ImageSize": "4032x3024",

Extracting it on the fly, every time the user hovers over a motion photo, doesn't seem to be a great solution in terms of complexity, disk I/O, and performance... although it would save disk storage in favor of reducing CPU / RAM usage & better performance.

Most browsers won't be able to play the original motion photo directly, right? It is ~50% larger than just the video, so a significant difference. Especially for mobile users who pay for traffic.

PhotoPrism also needs the actual video file for extracting & indexing properties like width, height & duration. Note that video (1440x1080) and photo resolution (4032x3024) are not identical. Rotation might also differ.

How many motion photos do users typically have? Is this more like an exception or a default, so that storage becomes a real issue?

[libeanim]

Ace, glad it worked!

Hmm yea, I guess the only way where it makes sense to leave it as one file is if the video can be extracted in the browser (as the browser would download the entire file anyway). Didn't get to testing if it works but I do see that it is probably a more hacky way of doing it and probably not the best solution...

Ideally, we want to do it in a way that is compatible with the Apple Live Photo integration and also get the correct metadata from the video (like resolution). So I guess you are right and extracting it might be the best option after all.

Usually, the motion photo setting is activated once in the camera and then stays active, so all pictures will be motion photos from then onwards. I assume not many users activate/deactivate it on a photo-by-photo basis.

Maybe having two options in the settings would do the trick?

• Extract Motion Photo at import
• Auto-delete Motion Photo video embedding after extraction

Edit: I'm fairly new to photoprism and just saw that there are different ways to import pictures. So a normal import (not indexing) could just automatically extract the video and delete the embedding, no user-editable options required. But I'd leave this decision up to you! Thanks for your work!

[lastzero]

We avoid modifying originals for now, see FAQ.

When deciding for a strategy, keep in mind that it should also work on a Raspberry Pi without causing the server to become unresponsive. Happens easily:

• Memory usage spiked making photoprism unusable on RPI #291
• Photoprism maxing out swap space, making system unresponsive. #294

Users with existing photo collections typically index directly and don't use import as they want to keep file and folder names. The extracted video can be stored outside the originals folder in the storage / sidecar folder, similar to generated JSON and JPEG sidecar files.

[CraftJJGamer308]

Hello, I am just wandering if there's a way to make a motion photo from a photo and a video with exiftool. So it's the other way around. I'd love to hear from you...

[alex-phillips]

Not sure if a separate issue should be open, but would love to see support for Google Motion photos. These have been in the pixel phones for quite a while now. They also embed an MP4 inside of the JPEG image.

[lastzero]

Sure, once we're done with high priority issues on our roadmap :)

[graciousgrey]

@alex-phillips could you send us example google motion files to hello@photoprism.app? :)