Skip to content

avfilter: New scene classification filter#9

Merged
jailuthra merged 7 commits intojai/ffmpeg-v4.4-workingfrom
jai/ffmpeg-v4.4
Aug 5, 2021
Merged

avfilter: New scene classification filter#9
jailuthra merged 7 commits intojai/ffmpeg-v4.4-workingfrom
jai/ffmpeg-v4.4

Conversation

@jailuthra
Copy link
Copy Markdown

@jailuthra jailuthra commented Aug 2, 2021

Why?
livepeer/lpms#248

What?

  • Replace our old vf_lvpdnn filter with vf_livepeer_dnn which is simpler, and uses the newer APIs in ffmpeg v4.4

TODO

  • Update documentation
  • Remove old pre-loading API from avfilter
  • Use newer session config API in v4.4 instead of our custom logic

Copy link
Copy Markdown
Author

@jailuthra jailuthra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some self-review comments to help other reviewers:

Comment thread libavfilter/avfilter.h Outdated
Comment thread doc/filters.texi Outdated
Comment thread libavfilter/dnn/dnn_backend_tf.c Outdated
Comment thread libavfilter/vf_livepeer_dnn.c Outdated
Comment thread libavfilter/vf_lvpdnn.c Outdated
Copy link
Copy Markdown
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see how much less code we need for the livepeer_dnn filter since we can take advantage of the recent updates to the DNN backend APIs in ffmpeg!

Comment thread libavfilter/vf_livepeer_dnn.c Outdated
@jailuthra jailuthra force-pushed the jai/ffmpeg-v4.4 branch 2 times, most recently from 72868f4 to 8d3e09a Compare August 4, 2021 12:31
This is required, as without it Tensorflow doesn't initialize
input/output tensors - and we end up spending that precious time at the
start of the first incoming frame.
Copy link
Copy Markdown

@oscar-davids oscar-davids left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Member

@yondonfu yondonfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Just one comment about commit structure - feel free to consider and address during merge process.

Comment thread libavfilter/dnn/dnn_backend_tf.c
Comment thread libavfilter/dnn/dnn_backend_tf.c Outdated
@jailuthra jailuthra merged commit b947074 into jai/ffmpeg-v4.4-working Aug 5, 2021
jailuthra added a commit to livepeer/lpms that referenced this pull request Aug 6, 2021
On top of v4.4, the new branch contains some critical fixes needed for
our transcoding pipeline - like flushing buffers of NVENC.

It also contains a refactored scene classification DNN filter:
livepeer/FFmpeg#9
ad-astra-video pushed a commit to ad-astra-video/FFmpeg that referenced this pull request Feb 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants