TIKA-2861: Extract QuickTime GPS location and item-list metadata in MP4Parser#2935
Conversation
The MP4 handler descends into the moov/meta container but skips the keys and ilst boxes, so the QuickTime metadata stored there (the com.apple.quicktime.* keys, e.g. the Apple Live Photo content identifier and the ISO 6709 location) was dropped for QuickTime .mov files and any .mp4 carrying it. Parse the keys/ilst boxes in TikaMp4BoxHandler and expose the UTF-8 text values under their key names. This is additive: the udta path and all existing fields are untouched.
In addition to the raw com.apple.quicktime.location.ISO6709 value, parse it
into the standard geo:lat, geo:long and geo:alt properties. This matches the
geo:* output the udta ("(c)xyz") path already produces for other files, and
additionally exposes the altitude, which the udta path drops.
The udta "(c)xyz" box carries an ISO 6709 string whose optional third component is the altitude (ISO 6709 Annex H). Older iOS versions and Google Photos write it with altitude, while Android writes lat/long only. TikaUserDataBox only matched lat/long, so the altitude was dropped. Capture the optional altitude and set geo:alt directly on the Tika metadata: the drewnoakes Mp4Directory only defines latitude/longitude tags, so it cannot carry the altitude to MP4Parser. Lat/long continue to flow through the directory as before. The new crafted fixture is the first test coverage for the udta location path (no existing fixture contained a "(c)xyz" box).
dschmidt
force-pushed
the
mp4-quicktime-metadata
branch
from
29f3646 to
4b06331
Compare
|
Tested with a real Live Photo pair from the osxphotos test libraries (iPhone 15 Pro, iOS 18.5): https://github.com/RhetTbull/osxphotos/blob/main/tests/Test-Live-15.7.2.photoslibrary/originals/C/C3090F66-942C-41D3-BEC7-4F2B4876A109.heic With a server built from this branch, Also checked the .MOV attached to the JIRA issue, its location |
The item-list parser only emitted UTF-8 text values (well-known type 1) and silently dropped the numeric types. Real iPhone Live Photo videos carry several of those, e.g. com.apple.quicktime.live-photo.vitality-score (float32), live-photo.auto (uint8) and camera.focal_length.35mm_equivalent (int32). Decode the QTFF well-known types 21 (signed int BE), 22 (unsigned int BE), 23 (float32) and 24 (float64) as well. Integers may be 1 to 8 bytes wide. Other value types (images, binary plists) are still skipped.
|
When comparing output for live photos I've noticed that some output was missing and that I should have not been handling only strings 🙃 |
There was a problem hiding this comment.
Pull request overview
This PR addresses missing QuickTime metadata extraction in the MP4 parser path by teaching TikaMp4BoxHandler to parse moov/meta/keys + moov/meta/ilst (including ISO 6709 GPS and numeric “well-known” value types), and by extending the existing udta (c)xyz location parsing to include optional altitude.
Changes:
- Parse QuickTime
keys/ilstitem-list metadata and emitcom.apple.quicktime.*fields (including numeric well-known types). - Map
com.apple.quicktime.location.ISO6709togeo:lat,geo:long, andgeo:alt(altitude optional). - Extend
(c)xyzudtaISO 6709 parsing to capture optional altitude and expose it viageo:alt, with new test fixtures + assertions.
Reviewed changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
.../src/test/java/org/apache/tika/parser/mp4/MP4ParserTest.java |
Adds regression tests for QuickTime keys/ilst extraction (including numeric types) and for udta altitude extraction. |
.../src/main/java/org/apache/tika/parser/mp4/TikaMp4BoxHandler.java |
Adds parsing of QuickTime keys/ilst, decoding of well-known value types, and ISO 6709 → geo:* mapping. |
.../src/main/java/org/apache/tika/parser/mp4/boxes/TikaUserDataBox.java |
Extends (c)xyz coordinate parsing to include optional altitude and sets geo:alt directly. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Sorry, should have confirmed license before merging. The two photos added were from the link above? If so, looks like MIT so we're good? https://github.com/RhetTbull/osxphotos/blob/main/LICENSE Otherwise, you can confirm they are ASL 2.0? |
|
No license concern there: the two committed test files are not photos at The osxphotos files linked above (MIT) were only used locally for the |
JIRA: https://issues.apache.org/jira/browse/TIKA-2861
MP4Parser drops GPS location and other QuickTime metadata from videos:
the box handler skips the
keys/ilstboxes insidemoov/meta, losing allcom.apple.quicktime.*metadata written by modern iOS devices, and the olderudta"(c)xyz" path only matched latitude/longitude, dropping the altitude.Changes (one commit each):
keys/ilstinTikaMp4BoxHandlerand emit UTF-8 text values undertheir key names (e.g.
com.apple.quicktime.content.identifier). Additive,the existing fields are untouched.
com.apple.quicktime.location.ISO6709value togeo:lat,geo:longandgeo:altin addition to the raw value.as
geo:alt. Older iOS versions and Google Photos write altitude there;Android writes lat/long only, which is unaffected. ExifTool parses the
same optional component.
big endian integers, float32 and float64. Real Live Photo videos carry several of
those, e.g.
live-photo.vitality-score(float32) andlive-photo.auto(uint8),which were silently dropped before.
Alternatives considered:
QuickTime/MOV reader, which does handle this metadata): rejected as much
higher risk, since it would change the parser's existing output wholesale
instead of adding the missing fields.
Mp4BoxHandler: awkward,since it would duplicate functionality of their QuickTime reader, and
upstream would likely (and reasonably) answer that MOV files should just
be read with that reader instead. Tika already extends the box handler
with Tika-specific boxes, so this follows the established pattern.
Testing: two new minimal fixtures (
testMP4_QuickTimeMetadata.movfor thekeys/ilst path,
testMP4_udtaLocation.mp4for the previously untested udtapath) with
MP4ParserTestassertions for the raw values and geo properties.