ExoPlayer re-render current frame when pause

[xiaos]

I am using exoplayer with surfaceTexture.

    SimpleExoPlayer exoPlayer = ExoPlayerFactory.newSimpleInstance(this, new DefaultTrackSelector());
    Surface surface = new Surface(surfaceTexture);

For each frame decoded, i will receive a callback in
public void onFrameAvailable(SurfaceTexture surfaceTexture) {
}

When i pause the player, i want to refresh my surfaceView with the current frame, by calling seekTo(exoPlayer.getPosition()); this is working when i am using MediaPlayer, but after I changed to exoplayer, i cannot seekTo the current position.

I just want to let exoplayer send the current decoded frame to surface. any suggestion?

Any suggestion?

[andrewlewis]

MediaCodec doesn't provide a way to re-render the last frame (and even if ExoPlayer didn't treat seeking to the current position as a no-op, re-rendering the last frame can take a significant amount of time as the decoder will need to decode from the last keyframe to the current position).

If you have a setup for getting decoder output from the SurfaceTexture to a SurfaceView, I guess you might be copying output using a GL shader. But then I'm confused why you need to re-render the frame as it should still be present in the surface texture. If you could provide some more details about why you need to re-render the last frame we might be able to make some suggestions. Giving detailed guidance on this is outside the scope of this issue tracker, however.

[xiaos]

Thanks for your reply.
I am using exoplayer to output video stream to surfacetexture, rendered by openGL on surfaceView.
I am developing a video editor, allow user to add sticker on the video. user can move around the stiker on video.

When the video is playing, there is no problem, because every frame, the surfaceview will be redraw.
but when user pause the video, and want to move around the sticker, i got problem.

When i am using MediaPlayer, when user move sticker, i will call mediaPlayers' seekTo() , then cause the screen redraw with the current video frame. But when switch to exoplayer, the seekTo() takes much more time, so cannot work.

Thanks.

[andrewlewis]

I think it's possible to composite your stickers as part of the operation that copies the video frame to the surface view. The decoder's output frame remains unmodified in the SurfaceTexture if you do this. (As a side effect this also lets you make changes to the sticker position more responsive on lower frame rate videos.)

[xiaos]

I am not sure if the frame still in SurfaceTexture or not. When pause the video and touch/move the sticker, i will call the below method in custom GlSurfaceView.

public void update(){
queueEvent(new Runnable() {
@OverRide
public void run() {
surfaceTexture.updateTexImage();

            requestRender();
        }
    });

}

But the screen is black, only the sticker is redraw.

[xiaos]

Another way i can think of is using FBO, for each frame, render to fbo then to GlSurfaceView.

So when player is paused, i just using the FBO texture to render. This should be working but just need an additional fbo texture.

[andrewlewis]

I'd probably try not calling updateTexImage each time you need to update the output. I think it should only be called once per onFrameAvailable callback from the SurfaceTexture. Having called updateTexImage once you should be able to copy from the texture multiple times.