How to prevent PIP mode from controlling the MediaLibraryService

[omkarquest]

How to prevent PIP mode from controlling the MediaLibraryService

I have a audio player which uses a service and a VideoPlayer which is in a activity.

The codes used are from the DemoPlaybackService and DemoMediaLibrarySessionCallback from the sessions demo app for the audio service and PlayerActivity from the main app for the video player.

These codes work perfectly when used Separately.

I am trying to implement the Picture-in-picture mode, when I minimize the app to PIP mode the video player looses its control and the audio player takes over. i.e if in PIP mode we click the play/pause /next/previous buttons they play the songs which are from the service instead of communicating with the videoplayer.
Also even though we have override fun onPictureInPictureModeChanged the playerview doesnot show its controller when they are back in full mode.

I understand this is an issue related to mediacontroller and how it is connected int he DemoMediaLibrarySessionCallback

For Pip mode I have used this code in the PlayerActivity

 private fun startPictureInPicture(): Boolean {
        
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
            enterPictureInPictureMode(
                PictureInPictureParams.Builder()
                    .setAspectRatio(
                        Rational(
                            16,
                            9
                        )
                    ) 
                    .build()

            )

           
            return true
        }
        else {
            Toast.makeText(
                this,
                getString(R.string.not_supported),
                Toast.LENGTH_SHORT
            ).show()
             
        }
        return false
    }


override fun onPictureInPictureModeChanged(
        isInPictureInPictureMode: Boolean,
        newConfig: Configuration,
    ) {
        super.onPictureInPictureModeChanged(isInPictureInPictureMode,
            newConfig
        )
        isCrossChecked = isInPictureInPictureMode
        if (isInPictureInPictureMode) {
           

            playerView!!.hideController()
        }
        else {
            
            playerView!!.showController()
        }
    }

and the mediacontroller connection is handled using the same code from DemoMediaLibrarySessionCallback

@OptIn(UnstableApi::class) // ConnectionResult.AcceptedResultBuilder
  override fun onConnect(
    session: MediaSession,
    controller: MediaSession.ControllerInfo,
  ): MediaSession.ConnectionResult {
    if (controller.isTrusted) {
      // Provide full information and specify media button preferences.
      val customButton = commandButtons[if (session.player.shuffleModeEnabled) 1 else 0]
      return MediaSession.ConnectionResult.AcceptedResultBuilder(session)
        .setAvailableSessionCommands(sessionCommandsWithCustomCommands)
        .setMediaButtonPreferences(ImmutableList.of(customButton))
        .build()
    } else {
      // Restricted read-only view on currently playing item only.
      return MediaSession.ConnectionResult.accept(
        SessionCommands.EMPTY,
        restrictedAccessPlayerCommands,
      )
    }
  }

The question, is how to handle the controller to ensure videoplayer doesnot loose its control when it is in PIP mode and starts communicating with the audioPlayer.

[omkarquest]

This is in the same app not separately.

[marcbaechinger]

Thanks for your question.

I understand you have a MediaLibraryService that hosts an ExoPlayer instance to play audio. This is runing in the service and is started as soon as you create a controller.

Besides this, there is an ExoPlayer instance that lives in the Activity and that is set to the PlayerView in that activity. This is the PlayerView that has it's UI controls hidden/show when the activivty transitions into PIP mode and back in your code above.

am trying to implement the Picture-in-picture mode, when I minimize the app to PIP mode the video player looses
its control and the audio player takes over. i.e if in PIP mode we click the play/pause /next/previous buttons they
play the songs which are from the service instead of communicating with the videoplayer.

For this to work you would need to change the player that is set to the PlayerView in some ways. That's the missing piece in your code that I miss for this to work. The code above only hides/shows the UI controls of the PlayerView while there is still the video player attached to the PlayerView.

I think I would expect the code that is executed upon transitioning into or out of the PIP mode to have something like this:

if (isInPictureInPictureMode) {           
    playerView.setPlayer(mediaControllerConnectedToTheSeviceWithAnAudioPlayer);
} else {
    playerView.setPlayer(exoPlayerInstanceForVideoPlayback)
}

With this code the player of the PlayerView is swapped out. The controls of the PlayerView then would call the MediaController instance that is connected with the service when in PIP mode. When in normal mode, the controls of the PlayerView would call the ExoPlayer instance that is doing video playback.

I think this looks like how it could work, in case I have understood the question correctly. Lemme know in case I didn't got the message correctly.

[omkarquest]

Thanks for your question.

I understand you have a MediaLibraryService that hosts an ExoPlayer instance to play audio. This is runing in the service and is started as soon as you create a controller.

Besides this, there is an ExoPlayer instance that lives in the Activity and that is set to the PlayerView in that activity. This is the PlayerView that has it's UI controls hidden/show when the activivty transitions into PIP mode and back in your code above.

This part is correct both playeractvity and service have separate players

this is the reference

for player activity

media/demos/main/src/main/java/androidx/media3/demo/main/PlayerActivity.java

Line 285 in b7748f2

playerView.setPlayer(player);

and for service this

media/demos/session_service/src/main/java/androidx/media3/demo/session/DemoPlaybackService.kt

Line 152 in b7748f2

mediaLibrarySession =

I have used the same code from the demo apps

I think you got this part wrong. I dont want to service player to be used in pip mode, I want to maintain the videoplayer to the pip mode

I dont want to swap the players

the problem is in pip mode we loose control of the already set videoplayer and it somehow gains control of the service

For this to work you would need to change the player that is set to the PlayerView in some ways. That's the missing piece in your code that I miss for this to work. The code above only hides/shows the UI controls of the PlayerView while there is still the video player attached to the PlayerView.

With this code the player of the PlayerView is swapped out. The controls of the PlayerView then would call the MediaController instance that is connected with the service when in PIP mode. When in normal mode, the controls of the PlayerView would call the ExoPlayer instance that is doing video playback.

I think this looks like how it could work, in case I have understood the question correctly. Lemme know in case I didn't got the message correctly.

[omkarquest]

To recreate the issue you simply merge the main demo and the sessions demo and use the PIP mode in the video player activity in main demo

[marcbaechinger]

According to the PIP documentation [1] PIP is using a media controller to send playback controls when in PIP mode. So I think the current approach you are having would not work because the video player is not using a media session?

[1] https://developer.android.com/develop/ui/views/picture-in-picture#add_controls

[omkarquest]

There is a way to ensure mediacontroller gets commands from our own app, Similarly is there a way to ensure that media controller is not used for the service as long as the pip mode is on for the video, and get back the control when we are not using pip mode.

[marcbaechinger]

I'm not super familiar with what the PIP implementation is doing. I think it's using a legacy controller. So you could disallow access for the notification controller or like removing all commands. However, if you do that SystemUI (media notification) wouldn't work anymore I think. The issue is that all legacy controller are using the platform session. There isn't a way to grant different permissions to different legacy controllers I'm afraid. So setting these for one would mean to set these for all.

[omkarquest]

I was thinking of the same issue, Alternatively I was thinking if we can remove the SessionCallback from the Demoservice while in PIP mode. Will it work?,

Or can we have different mediasessions and manage that with communicating with service or player activity.

Also this will be a problem when users decide to play the songs from notification I am confused about that.

I think we should be able to handle multiple Players and their communication with mediacontorller. will this be considered as an enhancement.

[marcbaechinger]

I'm not sure I understand what you mean.

In case a client is using the Media3 MediaController you can configure available commands for each controller specifically. If a client is using the legacy controller then this isn't possible.

So if I understand correctly the enhancement would be that the PIP UI would migrate to Media3. Then you could disallow some commands for that very controller only.

I think we should be able to handle multiple Players and their communication with mediacontorller.

This is either already possible with multiple sessions from the Media3 library side. Here again, the documentation of the PIP implementation leaves it open how it would work when an app has multiple sessions.

From how I understand you rissue so far, I don't see how we could help with an enhancement. You would need to clarify what you expect.

[omkarquest]

Yes this seems to be a problem on the PIP implementation side, how does it communicate .

As regards to enhancement I think PIP implementation can have some ability to manage its communication with media3. I know it is not part of media3, but at the larger Android UI level it should be considered for future development.

I will have a look at handling multiple sessions from the Media3 library, can you guide me where to look for it, is there any example for it?