Updated support for annotating video frames

[alexbw]

Using the actively maintained project moviepy (https://github.com/Zulko/moviepy), I have added support for annotating video frames, with a simple example of how to write the JSON specification for including frames.
I also included moviepy as a pip-installable dependency, so users should automatically have it installed, and not need to fish for dependencies. A caveat is it requires an installed version of ffmpeg, but all modern Linux systems include this, and it is easily installable on OSX. If the maintainers of sloth felt it was required, I could write a small bit of documentation highlighting how to troubleshoot the installation of ffmpeg.

I also did not use the _video_cache object at all in my code, instead opting for a seek-and-read strategy for single frames. It performs decently well on the videos (all tens of minutes long) that I have tried. However, caching might be desirable in the future.

Please let me know any comments or tips for improvement. Thanks for the great tool, would be excellent to contribute to it.

You should also advertise! It took me forever to find out this tool exists, and it's very useful for my community.

[baeuml]

Thanks for the PR. I think it would be good for now for our internal users that the okapy way still worked, therefore I am leaving this unmerged for now.

Do you have any experience on how accurate and repeatable the seeking to a certain timestamp is with moviepy? We found that ffmpeg (which I understand moviepy uses under the hood) has some problems to repeatably seek to the exact same frame if the video does not have an index (i.e. if you seek to the same timestamp/frame from different locations in the video you actually end up at different frames, and sometimes with more than just a few frames offset). This would obviously be bad for labeling since you can't be sure that you actually labeled the correct frame :(

Did you notice anything in this direction so far?

[alexbw]

I had not heard of this bug, and have not found it in my own work.

In my code, I always seek from the beginning to the desired frame, so this
should mitigate any potential problem.

Definitely understand re: migration of your users. Would be good for
external folks if there was a clear way to install okapy, which I couldn't
find.
On Wed, Nov 12, 2014 at 3:51 AM baeuml notifications@github.com wrote:

Thanks for the PR. I think it would be good for now for our internal users
that the okapy way still worked, therefore I am leaving this unmerged for
now.

Do you have any experience on how accurate and repeatable the seeking to a
certain timestamp is with moviepy? We found that ffmpeg (which I understand
moviepy uses under the hood) has some problems to repeatably seek to the
exact same frame if the video does not have an index (i.e. if you seek to
the same timestamp/frame from different locations in the video you actually
end up at different frames, and sometimes with more than just a few frames
offset). This would obviously be bad for labeling since you can't be sure
that you actually labeled the correct frame :(

Did you notice anything in this direction so far?

—
Reply to this email directly or view it on GitHub
#87 (comment).

[Zulko]

Hey there,

Just found this discussion through Github.

I believe MoviePy is very accurate now. See this script for details on how I checked time accuracy.

For the technical details: fetching a given timestamp with FFMPEG can be very fast and precise if you use combined seeking. Basically, you first make an unprecise seek to the nearest keyframe, then you crawl to the wanted frame.
See also this blog post for the basic python implementation with Python/FFMPEG.

[alexbw]

@Zulko, while you're here, is there functionality through moviepy to automatically install a decent version of ffmpeg? Just maybe as an included binary? A small headache I've run into while deploying my code to end-users is they either don't have ffmpeg, or it's so old as to be unusable. Would be very useful in the context of sloth, for my use cases.

[Zulko]

I guess it would be possible but I am very bad at these things (maybe it will be platform-specific, too). @srid proposed to include a Docker file which I believe answers this problem, but I didn't have time to really understand how that works. If you find a nice "standard" way of solving this, I'll happily merge it.

Maybe the solution would be system-specific packaging. I have seen that there is a package for Arch-Linux, maybe we can make a package for Ubuntu/Debian, Windows, MacOS. I have no Idea how to do these.

Can you open an issue so we can continue the discussion on the MoviePy page and keep track of this request ?

[nilsonholger]

@alexbw Is there still interest in this to be merged. I'm actually on @baeuml's side to not break the existing functionality with OKAPI which is used internally. I'm really sorry OKAPI is not widely available. Without looking too detailed at your commits, is there any way to alternate between OKAPI and moviepy?

[alexbw]

I no longer use this tool, so this request can be closed if it's too much
trouble.

On Fri, Aug 5, 2016 at 6:58 AM nilsonholger notifications@github.com
wrote:

@alexbw https://github.com/alexbw Is there still interest in this to be
merged. I'm actually on @baeuml https://github.com/baeuml's side to not
break the existing functionality with OKAPI which is used internally. I'm
really sorry OKAPI is not widely available. Without looking too detailed at
your commits, is there any way to alternate between OKAPI and moviepy?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#87 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAJ4jwWzHmWAs98cFGIrf-Q1sbirCW0uks5qcxc8gaJpZM4C2cS3
.

[nilsonholger]

Ok, then I'm closing this, may someday somebody will pick this up as a base for improvement, so thank you for your effort. Are you using any other tool?

[alexbw]

No, I don't use frame-by-frame video annotation anymore for research, that
phase is finished.

On Fri, Aug 5, 2016 at 7:23 AM nilsonholger notifications@github.com
wrote:

Closed #87 #87.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#87 (comment), or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAJ4j71VHmTu7VkGxCYlSQzfPoesO22Qks5qcx0cgaJpZM4C2cS3
.