Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request: Add support to extract telemetry data (stored as subtitles) from recordings #961

Open
cfsmp3 opened this issue Mar 9, 2018 · 14 comments

Comments

Projects
None yet
4 participants
@cfsmp3
Copy link
Contributor

commented Mar 9, 2018

This file is a MP4 recorded from a drone (specifically: "Zenmuse camera on a DJI Inspire 2 Drone"). The telemetry data is saved as subtitles and in fact VLC can display it.

CCExtractor doesn't find it - possible it's stored in a MP4 box we still don't handle, or something like that. We do show a number of "unknown box type", even though we're using a fairly new libGPAC.

GSoC qualification: 6 points

@cfsmp3 cfsmp3 added the GSoC-related label Mar 9, 2018

@cfsmp3

This comment has been minimized.

Copy link
Contributor Author

commented Mar 12, 2018

After some digging - this is just EXIF data. Not sure CCExtractor is the right tool to actually extract it (since it has nothing to do with subtitles after all), since exiftool does it perfectly already. We could however detect its presence and display some notification.

Anyway this tool processes the sample file just fine and reports everything:
https://www.sno.phy.queensu.ca/~phil/exiftool/

@thetransformerr

This comment has been minimized.

Copy link
Contributor

commented Mar 13, 2018

This was the dump I got from atom "moov/udta" present in above said file where the data was stored,

udta(?xyz?+26.483061-80.124142+38.000?xsp?-0.40?ysp?+0.10?zsp?+0.00?fpt?+2.80?fyw?+82.90?frl?-9.50?gpt?-27.70?gyw?+83.50?grl?+0.00?rec ?repPmeta hdlrmdir?ilst̩cmt?dataDE=TrueColor, Mode=P, DSW=0001Xtra?
           WM/Categorv01.07.2091.1.v1.0.0?uid??QPd ?mdlFC65100?csndb5c42c8e7d3048193df558fbdc77dfb(?aud??mux????free 
@thetransformerr

This comment has been minimized.

Copy link
Contributor

commented Mar 13, 2018

They might be the location from where we can make some changes:

while (atomStart < sample->dataLength) {

or

// TODO: support extension & style

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2018

@cfsmp3 What changes are we looking forward to? I might be able to finish this with some lead and more information.

@cfsmp3

This comment has been minimized.

Copy link
Contributor Author

commented Mar 19, 2018

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2018

@cfsmp3 When ccextractor is installed with -FFMPEG, MP4 files are also processed in general mode. Is that desirable? Or it should go with GPAC only?

This

case CCX_SM_FFMPEG:

comes before
case CCX_SM_MP4:

which leads to the problem I mentioned.

The sample provided in this discussion gives no output nonetheless but the box information is also not provided (which is very obvious).

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2018

@cfsmp3 What I think we can do is when we detect the first error, we stop the loop and tell the user that the file doesn't contain subtitles but EXIF data and point it to some tool that can extract it.

What do you say?

@cfsmp3

This comment has been minimized.

Copy link
Contributor Author

commented Mar 20, 2018

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2018

Digging a little deeper into the problem, I found that using the latest gpac's MP4Box, I'm unable to extract the information from the mp4 file and it is throwing the same errors as CCExtractor.

screenshot from 2018-04-09 01-19-34

This could mean that the issue lies with GPAC and not our program. I'm thinking about raising this issue in the gpac repository, what do you say @cfsmp3 ?

@cfsmp3

This comment has been minimized.

Copy link
Contributor Author

commented Apr 9, 2018

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Apr 16, 2018

@cfsmp3 libGPAC has been updated to handle the above file (thanks to @jeanlf), and I think we should update our version also in order to correct the issue with this sample. Can you please guide me how we update an entire library in our source? I can change the files manually but if there's a better way I would like to learn :)

Below is the updated MP4Box output for the video.

screenshot from 2018-04-16 23-41-49

@cfsmp3

This comment has been minimized.

Copy link
Contributor Author

commented Apr 16, 2018

@thealphadollar thealphadollar referenced this issue Apr 19, 2018

Merged

[IMPROVEMENT] Update libGPAC #974

6 of 6 tasks complete
@animesh-chouhan

This comment has been minimized.

Copy link

commented Jan 21, 2019

I want to work on this issue,
Can you brief me about the current state of this issue
as I can see there have been few merges in past

@thealphadollar

This comment has been minimized.

Copy link
Contributor

commented Jan 23, 2019

@cfsmp3 I think we should close this as we had concluded these are not subtitles and information is provided by GPAC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.