[iOS] Player unresponsive after playback-error caused by certain MP3's with APIC metadata.

[curiousdustin]

Apologies for the essay, but this is a complicated issue and I want to provide as much info as possible.

The Symptoms

In my app, I was experiencing strange behavior after certain tracks would fail to load. After one of these tracks failed to load, no future tracks would load either, even when they had loaded and played just fine previously.

I was able to track down the problem tracks and realized that they all had APIC metadata (album art embedded in the mp3).

After stripping out this artwork, the track load fine and do not cause the issue.

Many tracks DO load just fine even when they have this image metadata. So, perhaps in some cases it is corrupt or something. Either way, they load fine after removing the image metadata.

Reference

For reference, here are the files I am using to test with, and without, this image metadata:
https://livestage.curiousmedia.com/public/pin/audio/problem-track-with-image-metadata.mp3
https://livestage.curiousmedia.com/public/pin/audio/problem-track-without-image-metadata.mp3

Here is a comparison with a branch where I have modified the basic example to show the behavior, log a bit more info, and also attempt a solution. The solution is not complete, more on that later.

Testing

The app behaves as expected and plays the demo tracks that are in the example without issue. However, when I skip to the with-image track, it loads for longer than usual, and then gives a playback-error.

After this occurs, skipping to either next or previous tracks continues to result in the same playback-error. This continues until the app is closed and reopened.

(Press Play)

Resetting player.
Adding tracks: [<RNTrackPlayer.Track: 0x10b9003c0>, <RNTrackPlayer.Track: 0x10b901190>]
Loading: <RNTrackPlayer.Track: 0x10b9003c0>
load https://drive.google.com/uc?export=download&id=1AjPwylDJgR8DOnmJWeRgZzjsohi-7ekj
Adding tracks: [<RNTrackPlayer.Track: 0x107e06700>]
Adding tracks: [<RNTrackPlayer.Track: 0x10bc00c40>]
Adding tracks: [<RNTrackPlayer.Track: 0x108c00df0>]
Starting/Resuming playback

(Press Next)

Skipping to next track
Loading: <RNTrackPlayer.Track: 0x10b901190>
load https://drive.google.com/uc?export=download&id=1VM9_umeyzJn0v1pRzR1BSm9y3IhZ3c0E

(Press Next)

Skipping to next track
Loading: <RNTrackPlayer.Track: 0x107e06700>
load https://livestage.curiousmedia.com/public/pin/audio/problem-track-with-image-metadata.mp3
2019-03-22 15:01:26.137 [info][tid:com.facebook.react.JavaScript] 'playback-error', { error: 'Cannot Complete Action' }
2019-03-22 15:01:26.137385-0600 example[2968:1609527] 'playback-error', { error: 'Cannot Complete Action' }

(Press Next)

Skipping to next track
Loading: <RNTrackPlayer.Track: 0x10bc00c40>
load https://livestage.curiousmedia.com/public/pin/audio/problem-track-without-image-metadata.mp3
2019-03-22 15:01:52.700 [info][tid:com.facebook.react.JavaScript] 'playback-error', { error: 'Cannot Complete Action' }
2019-03-22 15:01:52.700222-0600 example[2968:1609527] 'playback-error', { error: 'Cannot Complete Action' }

(Press Next)

Skipping to next track
Loading: <RNTrackPlayer.Track: 0x108c00df0>
load file:///var/containers/Bundle/Application/42C9249A-92F4-4FA1-B747-6191D6C90012/example.app/assets/react/resources/pure.m4a
AVWrapper failed with Error:  Optional(Error Domain=AVFoundationErrorDomain Code=-11819 "Cannot Complete Action" UserInfo={NSLocalizedDescription=Cannot Complete Action, NSLocalizedRecoverySuggestion=Try again later.})
2019-03-22 15:02:12.117 [info][tid:com.facebook.react.JavaScript] 'playback-error', { error: 'Cannot Complete Action' }
2019-03-22 15:02:12.117378-0600 example[2968:1609527] 'playback-error', { error: 'Cannot Complete Action' }

Attempted Solution

The AVWrapper error is not all that helpful, but it lead me to this thread, which suggested using loadValuesAsynchronouslyForKeys, before creating the AVPlayerItem.

You can see my attempt at doing that in the loadAsync method in the branch comparison.

This actually DOES work to some degree. The track finally does load after longer than expected, and the playback-error is avoided. However, this is asynchronous, so it breaks certain things because the promise that caused the load to happen is resolved before the track is actually loaded, and before the currentItem is set/updated.

So for instance, with this solution, you would not be able to do something like:

await TrackPlayer.play();
await TrackPlayer.seekTo(123);

play() resolves before the player is actually ready to seek.

So, this is by no means a final solution, but it DID help the issue.

I think there are really 2 issues that need to be solved here:

1. Can this library handle this sort of metadata in a better way that does not result in a failure to play?

2. What needs to be done in order to gracefully recover from this error, if it cannot be avoided?

[curiousdustin]

@dcvz or @Guichaguri, Any thoughts on this?

[curiousdustin]

@dcvz, I have found a potential solution to 2. mentioned above.

It seems that SwiftAudio is not creating a new AVPlayer instance after a status of failed, as the Apple docs describe.

For more detail see the issue I have created on the SwiftAudio repo.

I implemented a quick version of the fix described by Apple here: curiousdustin/SwiftAudio@8b91573

[curiousdustin]

@dcvz, regarding 1. above:

For some reason loading tracks with this problematic image metadata in an async manner such as in this PR, jorgenhenrichsen/SwiftAudio#51, avoids the issue. The track still takes much longer to load than a normal track, but at least it does not completely fail.

However, I have concerns about how this change would affect the ability to use TrackPlayer commands in order even with promises, because the promise from play() is returned before an outcome, and subsequent commands will not apply to the track that was just told to play.

#548 (comment)

[manish-dalal]

@curiousdustin Hi, Do you have any luck on this. Can you please share the same with us. I am also facing the same issue. Have you fixed/completed the same in any of the branches. Thank you in advance.

[curiousdustin]

First of all the best is to fix your media files if possible. For our project we just stripped out the image meta data.

Next fixing the unresponsiveness involves modifying the 3rd party library SwiftAudio.

The solution I am currently using is my own attempt on my own fork/branch: curiousdustin/SwiftAudio@6a06eba

However, the author of SwiftAudio has since merged a PR that is his own (more complete) solution: jorgenhenrichsen/SwiftAudio#60

The current version of react-native-track-player does not include this update. However, I believe the next release will, as I noticed @dcvz has recently switched from including a copy of Swift Audio to referencing a forked version that will be maintained for RNTP. #615

[curiousdustin]

The fix for this is now included in SwiftAudio.