[Android] Low frame rate when using OES Texture Format on Android

[yukikkawa]

Unity version

2021.3.57f2

Unity editor platform

macOS

AVPro Video edition

Ultra

AVPro Video version

3.3.3

Device hardware

Galaxy S9

Which Android OS version are you using?

10

Unity Graphics API

OpenGLES 3

Video API

media3/ExoPlayer

Texture format

Y Cb Cr 420_OES

Audio output

System Direct

Any other Media Player component configuration required to reproduce the issue.

No response

Which output component(s) are you using?

Apply to Mesh

Any other component configuration required to reproduce the issue.

No response

The issue

I am experiencing a significant drop in frame rate when playing videos using the OES texture format.

Using the demo scene Assets/AVProVideo/Demos/Scenes/Demo_360Stereo.unity, I achieve approximately 30 fps with the default settings (without OES). However, when I switch to using OES, the frame rate drops to around 15 fps.

Is there any way to achieve better performance when using the OES path?

Configuration & Results

I compared the performance using the following settings:

Setting	Default	OES
Texture Format (Android Platform)	RGBA	Y Cb Cr 420_OES
Shader	AVProVideo/VR/InsideSphere Unlit (stereo+fog)	AVProVideo/VR/InsideSphere Unlit (stereo+color) - Android OES ONLY
Observed FrameRate	~30fps	~15fps

Media information

No response

Logcat output

[yukikkawa]

Default Demo_360Stereo Scene

Demo_360Stereo Scene using OES

The Screenshots taken with the following InfoOnGui.cs attached to the MediaPlayer object:

// InfoOnGui.cs

using UnityEngine;
using UnityEngine.UI;
using UnityEngine.EventSystems;
using RenderHeads.Media.AVProVideo;
using RenderHeads.Media.AVProVideo.Demos.UI;

public class InfoOnGui : MonoBehaviour
{
	[SerializeField] MediaPlayer _mediaPlayer = null;

	void OnGUI()
	{
		if (!_mediaPlayer || _mediaPlayer.Control == null)
		{
			return;
		}

		GUI.matrix = Matrix4x4.TRS(Vector3.zero, Quaternion.identity, new Vector3(3f, 3f, 1f));

		GUI.backgroundColor = Color.red;
		GUILayout.BeginVertical(GUI.skin.box);
		GUI.backgroundColor = Color.white;

		GUILayout.Label($"FrameRate: {_mediaPlayer.Info.GetVideoDisplayRate():F2}FPS");
		GUILayout.Label($"GraphicsDeviceType: {SystemInfo.graphicsDeviceType}");
		GUILayout.Label($"TextureFormat: {_mediaPlayer.PlatformOptionsAndroid.textureFormat}");

		GUILayout.BeginHorizontal();
		GUILayout.Label("States: ");
		GUILayout.Toggle(_mediaPlayer.Control.HasMetaData(), "HasMetaData", GUI.skin.button);
		GUILayout.Toggle(_mediaPlayer.Control.IsPaused(), "Paused", GUI.skin.button);
		GUILayout.Toggle(_mediaPlayer.Control.IsPlaying(), "Playing", GUI.skin.button);
		GUILayout.Toggle(_mediaPlayer.Control.IsBuffering(), "Buffering", GUI.skin.button);
		GUILayout.Toggle(_mediaPlayer.Control.IsSeeking(), "Seeking", GUI.skin.button);
		GUILayout.Toggle(_mediaPlayer.Control.IsFinished(), "Finished", GUI.skin.button);
		GUILayout.EndHorizontal();

		{
			TimeRanges times = _mediaPlayer.Control.GetBufferedTimes();
			if (times != null)
			{
				GUILayout.Label("Buffered Range " + times.MinTime + " - " + times.MaxTime);
			}
		}
		{
			TimeRanges times = _mediaPlayer.Control.GetSeekableTimes();
			if (times != null)
			{
				GUILayout.Label("Seek Range " + times.MinTime + " - " + times.MaxTime);
			}
		}

		GUILayout.EndVertical();
	}
}

[Chris-RH]

We've been able to reproduce this and are a bit stumped as to the cause of the issue at the moment. We'll look into this.

[MorrisRH]

A fix for this issue will make it into the next release

[yukikkawa]

@Chris-RH, Thank you for the update!

[yukikkawa]

@Chris-RH, When do you plan to release the next version?

[Chris-RH]

We are hoping to release the next version this week.

[yukikkawa]

@Chris-RH, thanks for the quick response.

[Chris-RH]

AVPro Video version 3.3.4 has been released. Please let us know if it has not fixed this issue.