Register playback as spokenAudio with iOS

[esarver]

Problem

I'm frustrated when I am listening to a podcast while navigating and Siri speaks over my podcast rather than pausing audio despite having the "Direction Pause Spoken Audio" setting enabled.

Proposed solution

I'd like to see the playback session registered as spokenAudio when playback is started so iOS knows that it should pause audio in PodVerse when interrupting notification (such as directions) are being played.

Additional context

iOS 16 does not recognize PodVerse is playing "spoken audio".

It looks like this might be gated on #1179. When doing setupPlayer(), options.iosCategoryMode should be set to spokenAudio so that iOS will know to optionally pause the track (per user preference) when in applications such as Maps that could interrupt playback.

[esarver]

Some relevant links:

• React Native Track Player Docs on player setup
• Apple docs on AVAudioSession.Mode

[mitchdowney]

@esarver thanks for the bug report and solution! I believe react-native-track-player v2 supports iosCategoryMode, so I'll give this a try in the v4.6.11 release.

If you'd like to test the iOS beta we'd appreciate it. I'm trying to get a new Beta sent to the stores tonight. TestFlight invite link

[esarver]

I installed TestFlight and got the beta. I'll give it a shot the next time I get a chance! After 4.6.11 drops, that is.

[esarver]

I tested it. The pausing works but it seems like there is an issue with resuming. It is possible that this is an issue with iOS, but I don't know. Didn't have time to look into it.

[esarver]

It could also be iOS Maps trying to be intelligent. I wasn't going very far and my directions were pretty close together. (I don't drive as much as I used to)

[esarver]

Actually, based on what I'm reading from Apple's docs, it seems like the onus for restarting might be on Podverse?

Apple's docs have an API for knowing when and interruption ends. I don't know how you would accomplish that with react-native-track-player. I wasn't able to find anything in the API that looked like what is needed (version 2 or 3).

UPDATE: This setting makes that last part sound incorrect... Sorry I can't be of more help!

[mitchdowney]

@esarver thanks for the update. We've had problems getting this to work in the past, which I have since forgot what we have done :/

The issue I imagine lies with our remote-duck handling, and possibly with how the react-native-track-player v2.1.3 handles remote-duck. Here's the corresponding logic in Podverse: https://github.com/podverse/podverse-rn/blob/develop/src/services/playerAudioEvents.ts#L267

There 12 closed issues in RNTP for "resume after duck" so...I'll have to do research to determine if we can fix this with our own logic, or if this is something RNTP v3 would fix: https://github.com/doublesymmetry/react-native-track-player/issues?q=is%3Aissue+resume+after+duck+is%3Aclosed

I'll see if I can at least reproduce the issue by using Siri with a local dev build of the app.

I'll take another stab at this this weekend.

[mitchdowney]

@esarver I made a change that seems to work locally when I test with Siri. Tbh though the remote-duck event seems to behave quirky and I'm concerned I might introduce a different issue with it. I'm going to try to release 4.6.11 build 1794 to TestFlight tonight. If you have a chance this week, could you try re-testing that?

  let wasPausedByDuck = false
  PVAudioPlayer.addEventListener('remote-duck', (x: any) => {
    (async () => {
      console.log('remote-duck', x)
      const { paused, permanent } = x
      const currentState = await audioGetState()
      const isPlaying = audioCheckIfIsPlaying(currentState)

      if (Platform.OS === 'ios') {
        if (permanent && isPlaying) {
          audioHandlePauseWithUpdate()
        } else if (isPlaying && paused) {
          wasPausedByDuck = true
          audioHandlePauseWithUpdate()
        } else if ((!permanent && wasPausedByDuck) || !paused) {
          wasPausedByDuck = false
          audioHandlePlayWithUpdate()
        }
      } else {
        if (permanent && isPlaying) {
          audioHandlePauseWithUpdate()
        } else if (isPlaying && paused) {
          wasPausedByDuck = true
          audioHandlePauseWithUpdate()
        } else if (!permanent && wasPausedByDuck) {
          wasPausedByDuck = false
          audioHandlePlayWithUpdate()
        }
      }
    })()
  })

[esarver]

Yeah, I would be happy to test it. I'll look for the update.

[mitchdowney]

@esarver according to my tests and Chad Farrow's, the latest change seems to work as intended (except for a failing case when many interruptions happened quickly).

I just went ahead and released to the App Store. I'm going to close all the "ready to deploy" tickets including this one. If there are bugs related to the spokenAudio setting I'd recommend creating a separate specific bug ticket for it. Feel free to reply here or reopen this one if you think it's needed though.

[esarver]

It worked well with Siri and Maps in my tests. However I happened to get a call right after I activated Siri while playing a podcast and it started playing again as soon as I answered the call.

I'm not sure if it would work like that if the call would have come through without Siri activated. It is definitely a corner case and should probably be a separate bug report.

[mitchdowney]

@esarver ah very interesting...hmm yes that could be a corner case.

I just tried to reproduce the issue by:

1. Played an episode.
2. Activated Siri. Playback paused as expected.
3. Called phone from a different phone.
4. Answered the call. (playback didn't play)
5. Hung up call. Playback resumed as expected.

So...I'm not sure what's going on with that. A race condition seems possible. If you ever run into it again and would like to create a bug report I'd appreciate it.

[esarver]

I absolutely will! Thanks so much for you responsiveness. This is a really great project and I can't wait to see where it goes!