Implement an FfmpegVideoRenderer

[haohaozaici]

I implemented a simple ffmpeg video extension that can currently be used to decode H.264 format videos.

Briefly explain what I did.

1. create FfmpegVideoRenderer same as Libgav1VideoRenderer
2. create FfmpegVideoDecoder like Gav1Decoder but decode method like FfmpegDecoder, then provide getExtraData() method for h264 type.
3. Modify the ffmpegDecode jni method from the ffmpeg_jni.cc file. In decode method, i separated the send packet method and passed inputBuffer.timeUs as AVPacket.pts. Use the receive frame method to receive the decoded data and copy it to the outputBuffer.
4. create FFmpegRenderersFactory to build custom VideoRenderers.

If anyone wants to use it or help with this extension, please refer to the README.

If I have any mistakes or features that need to be added, please feel free to raise them and I will try my best to modify them.

The features currently supported are very limited, including:

• video supported codec: only H.264
• audio supported codec: same as original extension
• supported surface type: video_decoder_gl_surface_view

There are many things to do next, including:

• Support other surface types
• Fix possible issues
• Organize the code
• Video Decoder support Format.rotationDegrees
• Support other codecs

[tonihei]

Thanks for filing the pull request!

@ojw28 Do you want to take a look? Feel free to reassign if not.

[ojw28]

Hi @haohaozaici - Thanks for sending this! I would like to merge this, but in order to do so I think we need it to modify the existing FFmpeg extension, rather than create a new one. To help, I've submitted a number of preliminary changes that help to get things in place. See the commits referenced from #2159 for details.

It would be great if you could have a go at reworking this pull request to slot into the existing FFmpeg extension now that these changes are in place. Roughly, I think you need to:

1. Merge your JNI changes into the existing ffmpeg_jni.cc file.
2. Rename FfmpegDecoder to FfmpegAudioDecoder.
3. Add your new FfmpegVideoDecoder class.
4. Merge your FfmpegVideoRenderer changes into the FfmpegVideoRenderer class, which is currently empty.

Notes:

1. 818925d makes it so that FfmpegVideoRenderer will be automatically picked up and used by DefaultRenderersFactory if using EXTENSION_RENDERER_MODE_PREFER or EXTENSION_RENDERER_MODE_ON. This means your FFmpegRenderersFactory class is no longer required.
2. The necessary change in FfmpegLibrary has already been made, in b7a5ace.
3. You no longer need separate FfmpegAudioDecoderException and FfmpegVideoDecoderException classes. You can use the common FfmpegDecoderException for both.

Please let us know if you plan to take a look at this! Thanks.

[haohaozaici]

Thank you very much for your suggestion, I will modify it as soon as possible.

[haohaozaici]

Thank you for making a lot of changes. I completed the changes based on your suggestions, but I accidentally performed the wrong operation when I entered the git command. Then I deleted the repository of my fork and re-fork a copy. Now I don't know how to proceed. I'm really sorry, do I need to resubmit a pull request, or is there any other way to fix this?

[ojw28]

Sending a new pull request is fine, if that's easiest for you!

[haohaozaici]

Thank you very much!
I send a new pull request #7132

[ojw28]

Superseded by #7132.