[expo-av][iOS] Lock screen controls not working

[redpandatronicsuk]

🐛 Bug Report

I am using react-native-music-control for lock screen controls when playing audio in background mode. The pause button on the lock screen works, but when play is pressed again, it does not resume playing. When the function that listens to the lock screen play button press runs, it says that there is no active sound session.

A a workaround, having this function:

expo/packages/expo-av/ios/EXAV/EXAVPlayerData.m

Line 359 in 491dde5

- (BOOL)_isPlayerPlaying

always return true results in the lock screen controls working.

The app I am working on is already in production. Is it safe to use this workaround as a temporary fix to get lock screen controls working?

Environment

Expo CLI 2.17.1 environment info:
System:
OS: macOS 10.14.4
Shell: 5.3 - /bin/zsh
Binaries:
Node: 11.10.1 - ~/.nvm/versions/node/v11.10.1/bin/node
Yarn: 1.15.2 - /usr/local/bin/yarn
npm: 6.7.0 - ~/.nvm/versions/node/v11.10.1/bin/npm
Watchman: 4.9.4 - /usr/local/bin/watchman
IDEs:
Android Studio: 3.4 AI-183.5429.30.34.5452501
Xcode: 10.2.1/10E1001 - /usr/bin/xcodebuild
npmPackages:
react: 16.8.3 => 16.8.3
react-native: 0.59.5 => 0.59.5
react-navigation: 3.9.1 => 3.9.1
npmGlobalPackages:
expo-cli: 2.17.1

iOS

Steps to Reproduce

Create new app with bare template -> add react-native-music-control as dependency -> configure MusicControl.on('play', ...) and MusicControl.on('pause', ...) and other required configurations -> play a sound -> whilst sound is playing lock the device -> on the lock screen press pause -> on the lock screen press play, this should print no active sound session... to error log

Expected Behavior

In the section above, the last step should resume the paused sound, rather than printing an error to the log

Actual Behavior

The last step two section above, i.e. printing to the error log that there is no active av session

Reproducible Demo

https://github.com/redpandatronicsuk/expoavtest

[cruzach]

Hi @redpandatronicsuk ,

Shouldn't this be posted to the react-native-music-control repo?

If you are using Expo APIs, then please include all your steps, and (preferably) a reproducible demo 😄

Thanks!
-Charlie

[redpandatronicsuk]

Hi @cruzach,

What is wrong with what I wrote in the Steps to Reproduce section?

Shouldn't this be posted to the react-native-music-control repo?

I thought the issue is with expo-av and the function I am referencing above, but maybe you are right. I'll open an issue on that repository too.

I'll leave this issue open until the issue is resolved, if that is OK.

Cheers,
Dominik

[expjess]

Hi @redpandatronicsuk - it helps a ton to have a reproducible demo, as @cruzach mentioned. Like this one here you included here: #3935. Can you re-open when you've updated with that?

Thanks a ton, esp for your quick responses!

[redpandatronicsuk]

Hi @expjess, sorry for taking long to get back this time, haven't had much time to work on this.

I've opened an issue with react-native-music-control here: tanguyantoine/react-native-music-control#257
and made a reproducible demo here: https://github.com/redpandatronicsuk/expoavtest

Also, many thanks for publishing expo-av-5.0.0-rc.0 to npm!

[redpandatronicsuk]

Hi @expjess, I misread your last comment. I thought you were going to re-open the issue once I add the reproducible demo. I don't know how I can re-open the issue, maybe I don't have the privileges? If you have permissions, could you please re-open it? Thanks

[redpandatronicsuk]

I just updated the dependencies the the repo with the sample code for this issue (https://github.com/redpandatronicsuk/expoavtest) now using expo-av: 5.0.0 and react-native-music-control: 0.10.4 and the issue persists.

[redpandatronicsuk]

@expjess any chance you could re-open this please?

[redpandatronicsuk]

Hi @cruzach and @expjess ,

I've added react-native-sound to the reproducible demo here: https://github.com/redpandatronicsuk/expoavtest

With react-native-sound lockscreen controls work. I thought that might help you figure out what the differences are and hopefully how to fix it. In the README in the demo repo, I have also further described the workaround, mentioned in the initial issue report, to make lockscreens work.

I would appreciate it if you could have a look at that workaround and tell me whether it is safe to use or not.

[redpandatronicsuk]

Hi @cruzach and @expjess ,

I've found another workaround to make lockscreen controls work. This time I think I found an actual bug in expo-av.

In expo-av/ios/EXAV/EXAVPlayerData.m the function getAudioSessionModeRequired uses _isPlayerPlaying and _shouldPlayerPlay to test if the session is active. When the player is paused _isPlayerPlaying and _shouldPlayerPlay are both false, I assume. This seems to check if the player is playing. Playing and being active are not the same things however. I think when the sound is playable (maybe even when it is buffering) the session is active. Please let me know what you think. I'd be more than happy to make a PR to fix this.

[brettdh]

@redpandatronicsuk I have the same issue; thanks for digging into this.

If I'm understanding your linked repro project, the workaround appears to be a patch to the EXAV pod - how do you maintain that patch and avoid a future pod install blowing it away?

Also, you mentioned two different workarounds in your last two comments, but I could only find one workaround described - the aforementioned EXAV patch. Could you describe the first workaround you mentioned? Or maybe just compare the two?

[redpandatronicsuk]

Hi @brettdh,

I only maintain the patch by hand, i.e. after pod update/install I edit the files again by hand. I suppose that could be automated with a hook/script or something like that. Maybe you could also make a local pod repository with the modifications and pull from there instead 🤔?

The workarounds:

1. Modify the function - (BOOL)_isPlayerPlaying in file EXAVPlayerData.m to always return true.
2. Check here: https://github.com/redpandatronicsuk/expoavtest/blob/1c5441b24e2eadf4086ba63740b3751436252c82/node_modules/expo-av/ios/EXAV/EXAVPlayerData.m#L823

So patch 1 keeps the Audio Session active, but the patch is logically wrong, as this function should not return true when the player is paused.

Patch 2 I think is the right approach, see the comments inlines 819-823 in the above referenced file

Hope that helps :)

[brettdh]

Perfect, thank you! I found the wonderful patch-package tool and saved your second patch for now. ✨

[redpandatronicsuk]

Thanks man, patch-package looks great!

[cmlarsen]

Any word on this being officially fixed? (I am also experiencing this issue using expo-av@7.0.1 and react-native-music-control@0.10.7)

[pavithra-expedux]

I am also getting same issue with ios

[cruzach]

Sorry everyone, looks like this was lost in the weeds

I'd need to look more into this to see if the issue lies with expo-av, or react-native-music-control (or both). If someone can provide more info, or a PR with an explanation and tests, that describes what the issue with expo-av is and maybe isolate it from react-native-music-control, that would be a huge help

[jamsch]

Came across the same issue. What seems to be working for me is setting interruptionModeIOS to AUDIO.INTERRUPTION_MODE_IOS_DO_NOT_MIX on Audio.setAudioModeAsync({...}). By default, expo-av configures the audio mode to EXAudioInterruptionModeMixWithOthers which MPNowPlayingInfoCenter won't listen for. iOS only considers non-mixing apps for inclusion in MPNowPlayingInfoCenter.

In other words, before playback, try calling the following:

 Audio.setAudioModeAsync({
      allowsRecordingIOS: false,
      staysActiveInBackground: true,
      interruptionModeIOS: Audio.INTERRUPTION_MODE_IOS_DO_NOT_MIX,
      playsInSilentModeIOS: true,
      shouldDuckAndroid: true,
      interruptionModeAndroid: Audio.INTERRUPTION_MODE_ANDROID_DO_NOT_MIX,
      playThroughEarpieceAndroid: false
  });

[redpandatronicsuk]

@cruzach sorry, I haven't seen your message earlier, but thanks for getting work on this issue started again. Can you please look at what I have written earlier in this tread. I am providing a workaround in the issue description all the way at the top. I also made a reproducible demo here: https://github.com/redpandatronicsuk/expoavtest
It uses old versions now, but I think the issue is still the same, please have a look at the comment here: https://github.com/redpandatronicsuk/expoavtest/blob/1c5441b24e2eadf4086ba63740b3751436252c82/node_modules/expo-av/ios/EXAV/EXAVPlayerData.m#L362

Thanks

[redpandatronicsuk]

@cruzach and the comment here too: https://github.com/redpandatronicsuk/expoavtest/blob/1c5441b24e2eadf4086ba63740b3751436252c82/node_modules/expo-av/ios/EXAV/EXAVPlayerData.m#L819

[cruzach]

Seems like your second comment is accurate, would you like to open a PR for this?

[valeriiamykhalova]

Have the same issue with expo-av 8.1.0. Is there any information when it will be fixed?

[IjzerenHein]

I've been debugging this to see whether we can get this fixed. What I've found so far is that with the latest expo-av it doesn't work because of this check in promoteAudioSessionIfNecessary in EXAM.m:

if (_isBackgrounded && ![_kernelAudioSessionManagerDelegate isActiveForModule:self]) {
  return UMErrorWithMessage(@"This experience is currently in the background, so the audio session could not be activated.");
}

This check does not allow audio to be played when the app is in the background. When disabling this check it starts working though.
Nevertheless this check serves a purpose and we cannot just disable it. More research and a solid approach for triggering audio in the background is needed. To be continued..