TIKA-4777: Expose presentation start of delayed QuickTime timed metadata tracks#2936
Open
dschmidt wants to merge 1 commit into
Open
TIKA-4777: Expose presentation start of delayed QuickTime timed metadata tracks#2936dschmidt wants to merge 1 commit into
dschmidt wants to merge 1 commit into
Conversation
…data tracks QuickTime timed metadata tracks (mebx sample descriptions, ISO 14496-12 boxed metadata) declare their key names in the file, and the track's presentation start is defined by its edit list: a leading empty edit delays the media by its duration. Parse the mebx key declarations (keys/keyd) and emit, for every declared key of a delayed track, the presentation start in microseconds as <key name>.track-start-us. Undelayed tracks (start 0) are not reported, their start carries no information. This is deliberately generic, no key names are hardcoded. The main use case are Apple Live Photos, which mark the moment the paired still image was captured as the single one-tick sample of the still-image-time track, shifted to that moment by an empty edit. The sample value itself is a constant -1 marker, so the moov boxes alone are sufficient and mdat is never touched. The handler stays active for timed metadata tracks (handler type 'meta') instead of switching to the extraction-free Mp4MetaHandler, which would keep it from seeing the track's stsd. Verified against a real Live Photo video (iPhone 15 Pro, iOS 18.5): com.apple.quicktime.still-image-time.track-start-us = 1233333 (1.2333s, matching the empty edit of 740/600s), likewise the transform keys of the same track; the undelayed video-orientation and live-photo-info tracks are not reported.
dschmidt
force-pushed
the
mp4-still-image-time
branch
from
9ae502f to
3820b30
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: https://issues.apache.org/jira/browse/TIKA-4777
Builds on the QuickTime metadata support from #2935 (TIKA-2861, now
merged); rebased onto main accordingly, a single commit.
QuickTime timed metadata tracks (
mebxsample descriptions, ISO 14496-12boxed metadata) declare their key names in the file, and a track's
presentation start is defined by its edit list: a leading empty edit
delays the media by its duration. This parses the mebx key declarations
(keys/keyd) and emits, for every declared key of a delayed track, the
presentation start in microseconds as
<key name>.track-start-us.Undelayed tracks are not reported, their start carries no information.
No key names are hardcoded. The main use case are Apple Live Photos,
which mark the moment the paired still image was captured as the single
one-tick sample of the
com.apple.quicktime.still-image-timetrack,shifted there by an empty edit. The sample value itself is a constant -1
marker, so the moov boxes alone are sufficient and mdat is never read.
Apple's own LivePhotosKit JS player derives its
photoTimethe same way(it parses trak/elst/mebx client-side).
Verified against a real Live Photo video (iPhone 15 Pro, iOS 18.5):
com.apple.quicktime.still-image-time.track-start-us = 1233333(1.2333s,matching the empty edit of 740/600s), likewise the transform keys
declared by the same track; the undelayed video-orientation and
live-photo-info tracks are not reported.
Testing: the crafted
testMP4_QuickTimeMetadata.movfixture gains atimed metadata track with a proper mebx keys/keyd declaration and an
empty edit;
MP4ParserTestasserts the emitted track start.