How to process each video frame when using Exoplayer play a RTSP stream?

[gscholer]

Here is a similar question google/ExoPlayer#7167, I am using exoplayer to pull and decode a RTSP stream, it can display the video now, but I want apply a deep learning model to detect something appeared on each frame.

From the answer of andrewlewis, I think using a ImageReader is the easiest solution, but I am new to android development and couldn't find any examples trying my best. I hope someone can help me, and other solution to this problem can also be accepted.

[droid-girl]

Hi @gscholer ,
This issue seems to be similar to #1422. Are you also looking at using TFLite?
Another suggestion is to check out MediaPipe integration with Transformer in the demo app to get an idea how to integrate with MediaPipe library.
If you want to use MediaPipe with ExoPlayer, you can:

• create your custom effect using MediaPipe (check out transformer demo for more info)
• use .setVideoEffects in ExoPlayer to set your custom MediaPipe effect

For TFLite, I will add more details on integration shortly to #1422.

[gscholer]

Hi @droid-girl, Thanks for your reply. I am looking forward your TFLite integration.

[droid-girl]

More info is now added to #1422. Closing this issue