Custom MediaSources, resolving MediaItems from remote API, and metadata updates

[sampengilly]

I've searched around extensively on this repository, on the ExoPlayer repository, on stack overflow, and I've been browsing through the source to determine the best course of action for my use case but I haven't really found much, and I've run into issues or gaps in the explanation on the information I have found.

The use case I have seems like it calls for custom media sources, for the most part it isn't complex, it's a case of loading the media based upon an internal URI system. But there are some additional complexities when it comes to the type of media to load.

Key details:

• I'm using ExoPlayer as my player implementation under Media3
• I am aiming to treat Media3 (and the underlying ExoPlayer) as my single source of truth for playback, i.e. all information shown in the player UI and media notification should come from Media3 and the media metadata. (With the notification being automatic out of the box in media3).
• The content I am loading comes from a remote datastore API, I am aiming to encapsulate this within the Media3/ExoPlayer part of my app. In other words, As far as my app UI is concerned, it just needs to add a MediaItem that uses request metadata to point to one of my internal URIs (something like mydatastorescheme://podcast/{id}) and the rest is handled opaquely inside the Media code.
• I have many different types of content to load, all audio, but things like podcasts, audio books, live radio, etc.

In Media3 we have the MediaSession callback with the required onAddMediaItems function. This function works fine for loading the correct audio URLs and metadata for a particular bit of content referenced by one of my internal URIs. e.g. I can take the request metadata, lookup in my system what mydatastorescheme://podcast/1234 is, get its title, description, presenter name, audio stream URL, etc and I can fill out a MediaItem with that information.

However this approach seems somewhat naive and could balloon out a bit when it comes to supporting a wide variety of content types as simple as it is initially. I encountered a roadblock here when it comes to live radio streams, as the MediaItem is immutable and I have a need to hit another part of my API at scheduled times in order to update certain parts about the metadata (i.e. what radio program is currently airing, who is presenting, etc). This lead me down the path of looking more at the integration with ExoPlayer and creating custom media sources.

I've been attempting to set this up for the last few days with limited success and with some uncertainty over whether I'm using things in the right way.

The first problem I'm tackling is loading simple, on demand content with unchanging metadata (like the podcast example above). I've looked at both MediaSource factories and DataSource factories for achieving this (particularly the ResolvingDataSource). There's a few different roadblocks I've run into with trying each of these approaches:

• Using the DefaultMediaSourceFactory with a ResolvingDataSource doesn't work to resolve my content before creating a media source as the media sources are created before the data source is used, the DefaultMediaSourceFactory also aborts if the local configuration of the MediaItem is null, which it is for me at this stage since the RequestMetadata still needs to be resolved.
• It would be wrong to simply put my internal URL from the RequestMetadata into the local configuration, as the type of the media is still unknown at this point, so the DefaultMediaSourceFactory wouldn't be able to correctly select the right MediaSource and would just fall through to the default ProgressiveMediaSource.
• Using a CompositeMediaSource I am able to set up a loader to resolve my internal URI to an actual MediaItem with local configuration and metadata, I can then pass this to DefaultMediaSourceFactory to get a MediaSource which I can prepare as a child of my CompositeMediaSource. However, I'm not certain if this is an ideal approach (using a media source factory inside another media source), I've seen a question asked about this without really getting an answer on it specifically (here: MediaSourceFactory for different content types combined with the dynamic stream links #17)
• Using the CompositeMediaSource approach I can get the audio content playing but the metadata doesn't seem to update. From tracing through the code execution it seems like the block that builds new metadata from incoming playback info (in ExoPlayer::updatePlaybackInfo) isn't getting run due to either mediaItemTransitioned being false or the staticMetadata being the same across the previous and new playback info objects. My setup is very similar to that seen in the above linked question. In onChildSourceInfoRefreshed I'm simply calling refreshSourceInfo with the new timeline as seen in some built-in composite media sources like WrappingMediaSource. I'm unsure why this doesn't carry through and update the metadata in the player.
• Essentially I'm hoping to reuse as much of the existing media source setup as possible, just wrap it, but I have a need to perform some network operations before deciding upon which media source to go with, the MediaSource.Factory seems like an ideal place for this except that it must immediately return a value.

If I can get that working then the next step would be working out what's required (likely in another custom MediaSource) to load a stream for playback, then call an API at specific times to fetch the metadata for the next airing program (API has the current and next available with timestamps for the start and end times of each). This feels like it's a job for periods on a timeline? But the mechanism through which to make those schedules API calls and surface the metadata is still unclear to me.

The documentation doesn't really have much info in it for worked examples on customized Media Sources, and resolving media lazily from an internal API seems to be a common-ish question just with different nuances. I read through another example of this here: google/ExoPlayer#5883 though it still leaves some questions unanswered (such as my questions above on the use of a factory inside a media source, or the issue with metadata now flowing through from the added child source to the player.

If I could get some guidance on these questions or if this is completely the wrong direction some guidance of what the right direction is, then that would be much appreciated.

[icbaker]

I think you're hitting a couple of different existing issues/feature requests:

• How to update current media metadata (user rating) without interrupting the playback? #33
• Automatically identify any stream type google/ExoPlayer#3165

---

To specifically answer some of the parts of your post:

I encountered a roadblock here when it comes to live radio streams, as the MediaItem is immutable and I have a need to hit another part of my API at scheduled times in order to update certain parts about the metadata (i.e. what radio program is currently airing, who is presenting, etc).

I think this would be resolved by #33? (without needing a custom media source)

• Using the DefaultMediaSourceFactory with a ResolvingDataSource doesn't work to resolve my content before creating a media source as the media sources are created before the data source is used, the DefaultMediaSourceFactory also aborts if the local configuration of the MediaItem is null, which it is for me at this stage since the RequestMetadata still needs to be resolved.

If you want to use ResolvingDataSource I think you should move the URI into MediaItem.localConfiguration.uri during MediaSession.Callback.onAddMediaItems. This means that MediaItem.localConfiguration.uri will be non-null when it hits the MediaSource.Factory and is passed to ResolvingDataSource, at which point you can transform it to the real URI.

• It would be wrong to simply put my internal URL from the RequestMetadata into the local configuration, as the type of the media is still unknown at this point, so the DefaultMediaSourceFactory wouldn't be able to correctly select the right MediaSource and would just fall through to the default ProgressiveMediaSource.

This reflects a limitation of DefaultMediaSourceFactory, rather than something inherently 'wrong' imo. Improving this is tracked by #3165. Given this limitation, and the fact you're able to run all your resolution code inside MediaSession.Callback.onAddMediaItems, that seems like possibly the best approach until #3165 is implemented? If I read your post properly, the only part missing when you followed this approach is updating the metadata during playback (note that ICY metadata from internet radio will get updated during playback).

The documentation doesn't really have much info in it for worked examples on customized Media Sources

This is sort of intended - customising MediaSource implementations is very fiddly, and we don't expect many developers to need to go down this route. It should only really be needed if you're trying to support a completely different streaming format (e.g. something like HLS or DASH, but different), or something with very fiddly metadata like server-side ad insertion. Just adding an async resolution layer (as you're trying to do here) shouldn't need a custom MediaSource.

[sampengilly]

Is there an expected resolution timeframe for #33? I can populate my MediaItem in onAddMediaItems with my best available metadata at the time of adding (i.e. podcast episode name and show name for a podcast, radio station name for a radio station), this would also allow the DefaultMediaSourceFactory to correctly pick up HLS streams. But within the next 1-3 months I will have to ensure that the metadata is updated with information about the current program/presenter, sometimes the current track, chapters in audiobooks, etc and as far as I'm aware we don't have this information coming through a mechanism like ICY, only via an API.

[icbaker]

Is there an expected resolution timeframe for #33?

I'm afraid we don't have a timeline for this.

---

Picking up another comment you made on google/ExoPlayer#11031:

Regarding the topic of metadata during playback. The docs mention a mechanism for receiving this metadata by "adding a MetadataOutput to the player" but provides no further information on how to do that. https://exoplayer.dev/retrieving-metadata.html#during-playback, if there is a mechanism through which other customizations can write to such a MetadataOutput would that be an appropriate solution to support the live radio case?

This documentation is a little stale, apologies: MetadataOutput has been merged into Player.Listener. I will update the docs.

[sampengilly]

So, in the absence of #33 and #3165, is there any other mechanism for sideloading the pushing of metadata updates to the player that doesn't get too fiddly? A way to manually trigger Player.Listener updates? or a custom MetadataRenderer? or a thin wrapping MediaSource that injects information into the Timeline.Periods?

Or at this stage does this need to be handled separately to Media3 (i.e. updating the app player UI and OS Media Notification without reference to the MediaMetadata coming from the Player)

[icbaker]

I've just fixed a small mistake in my comment above, where I originally said:

If you want to use ResolvingDataSource I think you should either always put your custom URI in the MediaItem.localConfiguration.uri (before you even pass the MediaItem to MediaController), or move it there during MediaSession.Callback.onAddMediaItems. This means that MediaItem.localConfiguration.uri will be non-null when it hits the MediaSource.Factory and is passed to ResolvingDataSource, at which point you can transform it to the real URI.

This missed the fact that MediaItem.localConfiguration.uri is not transferred from MediaController to the session, so you have to move the URI from MediaItem.requestMetadata to MediaItem.localConfiguration.uri during MediaSession.Callback.onAddMediaItems - sorry for that.

---

A way to manually trigger Player.Listener updates?

I have not tested this, but you may be able to use ForwardingPlayer to achieve this. You would subclass ForwardingPlayer and hook the addListener method to add your own wrapper around the listener, such that you can manually invoke onMediaMetadataChanged whenever you like. You would then pass this ForwardingPlayer subclass to new MediaSession.Builder(...). This should result in your manual metadata changes propagating to the media session, and subsequently ending up in notifications etc. If you get this working it would be great if you could reply with your findings on #33.

For consistency, you will need to ensure that you also change the result of Player.getMediaMetadata when you invoke onMediaMetadataChanged. See general notes on maintaining consistency when subclassing ForwardingPlayer here: https://exoplayer.dev/customization.html#intercepting-method-calls-with-forwardingplayer

One thing to note: When you override ForwardingPlayer.addListener, you should delegate to super.addListener and not directly to delegate.addListener. This will ensure that the correct Player instance (your ForwardingPlayer subclass) is passed to Player.Listener.onEvents.

[marcbaechinger]

you may be able to use ForwardingPlayer to achieve this

This works. People are doing this: #256, #246

[sampengilly]

Thank you, I'll try that approach in the next week and report back

[icbaker]

Closing because I think the question has been answered and the pending feature requests are tracked elsewhere.

[sampengilly]

I know I said next week in my comment previously but priorities ended up shifting a bit and I haven't had much of a chance to look at this.

Taking a look now I assume the approach is to override the getMediaMetadata() method in the ForwardingPlayer to return the desired metadata, and also triggering appropriate calls to onMetadata() in any ForwardingListeners added to the player to assist with UI/Notification updates. Does this sound correct and in line with the expected usage of ForwardingPlayer?

I assume that since the linked issue #246 is still open there is still a need for some additional invalidation in order to force the media notification to pull updated metadata? or forcing the media notification to check for new metadata is currently not possible?

[icbaker]

Taking a look now I assume the approach is to override the getMediaMetadata() method in the ForwardingPlayer to return the desired metadata

Yep, this sounds right.

and also triggering appropriate calls to onMetadata() in any ForwardingListeners added to the player to assist with UI/Notification updates.

You will want to trigger onMediaMetadataChanged(), not onMetadata() - it is documented as follows:

Called when the value of getMediaMetadata changes.

Also ForwardingPlayer.ForwardingListener is private, so I wouldn't expect you to interact with that type directly. You should just be able to interact with the Player.Listener type, and as described above in #258 (comment) you can get a reference to Player.Listener instances being added to your ForwadingPlayer subclass by overriding ForwardingPlayer.addListener.

[6rism0]

Hey @icbaker,

Thanks for the suggestion with the ForwardingPlayer. I've implemented it, in order to update the media item (audio stream) metadata from a different source other than the initial or in-stream metadata.

So far it works quite well, but somehow the ICY metadata keeps getting delivered to listers that attach to the MediaController build from the Session which uses the ForwardingPlayer. (The notification e.g. only shows the metadata I propagate and not the ICY metadata)

I assumed that the ForwardingPlayer subclass which is passed to new MediaSession.Builder(...), would be the MediaController I get with MediaController.Builder(context, mediaSessionToken), but that doesn't seem to be the case.

Am I getting something wrong here?

[icbaker]

So far it works quite well, but somehow the ICY metadata keeps getting delivered to listers that attach to the MediaController build from the Session which uses the ForwardingPlayer. (The notification e.g. only shows the metadata I propagate and not the ICY metadata)

Just to check I'm understanding your problem statement: You're trying to stop ICY metadata from reaching Player.Listener.onMediaMetadataChanged callbacks that are registered on your MediaController, but currently this is being invoked for both ICY and your 'injected' metadata changes (which is a problem for you)?

[6rism0]

@icbaker Yes, totally right.

The icy metadata is quite limited and we try to present only the richer information from an external API, but we might need the icy metadata as a trigger, though. So we can't totally block any icy onMediaMetadataChanged calls from within the actual player.

The idea based on your suggestion was, catch onMediaMetadataChanged calls in the ForwardingPlayer (maybe use them as a trigger to fetch 'richer' metadata from an api) and then call onMediaMetadataChanged with custom metadata on every attached listener.

[icbaker]

Thanks - this sounds slightly different to the original question in this thread. Rather than re-opening this issue, would you mind filing a new issue (you can just copy the content from your two comments here), and I'll follow up there. It would also be helpful to include a code snippet showing exactly how you've 'intercepted' the onMediaMetadataChanged call inside your ForwardingPlayer subclass.