SphericalGLSurfaceView appears to break on v2.16.1; same code works on v2.15.1

[w3woody]

When attempting to play a 360° video using the version 2.16.1 of the ExoPlayer, only a gray screen is displayed. Reverting to 2.15.1 of the ExoPlayer successfully plays the same video using the same code.

---

The source code for our view activity is largely cobbled together from the sample source kit, and is included below, along with the activity layout XML.

When debugging the v2.16.1 version of the SphericalGLSurfaceView, within the ProjectionRenderer.init method (line 123), the following code:

program = new GlUtil.Program(VERTEX_SHADER_CODE, FRAGMENT_SHADER_CODE);
mvpMatrixHandle = program.getUniformLocation("uMvpMatrix");
uTexMatrixHandle = program.getUniformLocation("uTexMatrix");
positionHandle = program.getAttribLocation("aPosition");
texCoordsHandle = program.getAttribLocation("aTexCoords");
textureHandle = program.getUniformLocation("uTexture");

This fails; all calls to getUniformLocation and getAttribLocation all fail with -1. This means the spherical renderer is unable to render anything--and only a gray screen is displayed.

Rolling back to v2.15.1 fixes the problem, though I did not investigate the code to see what changed between the two versions.

---

Note that upgrading to v2.16.1 and migrating to the non-deprecated API calls do not resolve this issue.

Tests were run on a Google Pixel 5 running Android version 12.

---

The 360° video we're using is linked above.

The activity source code:

Video360ViewerActivity.java

public class Video360ViewerActivity extends BaseNetworkActivity
{
    private PlayerView playerView;
    private SimpleExoPlayer player;
    private String playURL;

    // Player state (from example code) at https://codelabs.developers.google.com/codelabs/exoplayer-intro/#2
    private boolean playWhenReady = true;
    private int currentWindow = 0;
    private long playbackPosition = 0;

    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_360viewer);

        /*
         *  This receives an ID which refers to a ChatMedia record containing
         *  an image or video.
         */

        Bundle bundle = getIntent().getExtras();
        playURL = bundle.getString("url");

        playerView = (PlayerView)findViewById(R.id.video);
    }

    // From https://codelabs.developers.google.com/codelabs/exoplayer-intro/#2
    @SuppressLint ("InlinedApi")
    private void hideSystemUi()
    {
        playerView.setSystemUiVisibility(
                View.SYSTEM_UI_FLAG_LOW_PROFILE
                | View.SYSTEM_UI_FLAG_FULLSCREEN
                | View.SYSTEM_UI_FLAG_LAYOUT_STABLE
                | View.SYSTEM_UI_FLAG_IMMERSIVE_STICKY
//              | View.SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION
//              | View.SYSTEM_UI_FLAG_HIDE_NAVIGATION
        );
    }

    @Override
    protected Properties getScreenProperties()
    {
        Properties p = new Properties();
        p.put("URL",playURL);
        return p;
    }



    @Override
    protected void onStart()
    {
        super.onStart();
        if (Util.SDK_INT >= 24) {
            initializePlayer(playURL);
        }
    }

    @Override
    protected void onResume()
    {
        super.onResume();
        hideSystemUi();
        if ((Util.SDK_INT < 24 || player == null)) {
            initializePlayer(playURL);
        }
    }

    @Override
    protected void onPause()
    {
        super.onPause();
        if (Util.SDK_INT < 24) {
            releasePlayer();
        }
    }

    @Override
    protected void onStop()
    {
        super.onStop();
        if (Util.SDK_INT >= 24) {
            releasePlayer();
        }
    }

    private void initializePlayer(String url)
    {
        SimpleExoPlayer.Builder builder = new SimpleExoPlayer.Builder(this);
        player = builder.build();
//      player = ExoPlayerFactory.newSimpleInstance(this);
        playerView.setPlayer(player);

        MediaSource mediaSource = buildMediaSource(url);

        player.setPlayWhenReady(playWhenReady);
        player.seekTo(currentWindow, playbackPosition);
        player.prepare(mediaSource, false, false);
    }

    private void releasePlayer() {
        if (player != null) {
            playWhenReady = player.getPlayWhenReady();
            playbackPosition = player.getCurrentPosition();
            currentWindow = player.getCurrentWindowIndex();
            player.release();
            player = null;
        }
    }

    private MediaSource buildMediaSource(String url)
    {
        Uri uri = Uri.parse(url);
        DefaultDataSourceFactory factory = new DefaultDataSourceFactory(this,"Relocity");

        if (url.endsWith("m3u8")) {
            return new HlsMediaSource.Factory(factory).createMediaSource(uri);
        } else {
            return new ProgressiveMediaSource.Factory(factory).createMediaSource(uri);
        }
    }

    @Override
    protected void onDestroy()
    {
        super.onDestroy();
    }
}

---

XML layout activity_360viewer.xml

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:animateLayoutChanges="true"
    android:background="@color/black"
    android:orientation="vertical">

    <com.google.android.exoplayer2.ui.PlayerView
        android:id="@+id/video"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:visibility="visible"
        app:surface_type="spherical_gl_surface_view"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

[christosts]

@andrewlewis can you please take a look?

[tomgnapster]

You need to add:
program.use();
to ProjectionRenderer.java l125 (so after program = new GlUtil.Program(VERTEX_SHADER_CODE, FRAGMENT_SHADER_CODE);)

This change sorts everything out for me on a branch created from tag r2.16.1.
I tried submitting a PR against dev-v2, but that branch contains a ton more changes (and also doesn't work for me).

[andrewlewis]

On the dev branch it appears spherical rendering is broken since 14b42a4 because that change removed the call glEnableVertexAttribArray per draw call, yet there is still a call to glDisableVertexAttribArray at the end of each draw call, and (also on the dev branch) from what I can tell there is no need to add back the use call because the program is used when it's created. I will prepare a fix accordingly. Thanks!