New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS: Audio spikes/cuts when the screen is locked. #997

Closed
notlion opened this Issue Jun 24, 2015 · 13 comments

Comments

Projects
None yet
2 participants
@notlion
Contributor

notlion commented Jun 24, 2015

If the screen is locked manually or the device goes to sleep while sound is playing a tone is played at full volume. After this happens all audio is muted on the device until it is rebooted.

I've tested this with the NodeBasic sample on an iPad Air 1st gen (iOS 8.3, current cinder/master), but have also seen the same behavior on an iPad Mini 1st gen.

Perhaps AVFoundation emits some callback we are expected to respond to by reducing the volume?

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Jun 24, 2015

Collaborator

Problem is that currently I'm not handling the iOS notifications for when to pause / resume audio. It's long been on my TODO list, but I'd love it if someone using iOS more oftern than myself (almost never) were to look into it.

Here's what I think it is: I couldn't ever get these notifications to fire, once they are working, we can add some signal callbacks to DeviceManager, perhaps getSignalInturruptionBegan() and getSignalInterruptionEnd(). Then the audio::OutputDeviceNode can hook into these to pause / resume processing.

Collaborator

richardeakin commented Jun 24, 2015

Problem is that currently I'm not handling the iOS notifications for when to pause / resume audio. It's long been on my TODO list, but I'd love it if someone using iOS more oftern than myself (almost never) were to look into it.

Here's what I think it is: I couldn't ever get these notifications to fire, once they are working, we can add some signal callbacks to DeviceManager, perhaps getSignalInturruptionBegan() and getSignalInterruptionEnd(). Then the audio::OutputDeviceNode can hook into these to pause / resume processing.

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Jun 24, 2015

Collaborator

Oh, and we don't need to worry about the pre-iOS 6 route, we're bumping the min runtime target to 6 so that code should just be removed.

Collaborator

richardeakin commented Jun 24, 2015

Oh, and we don't need to worry about the pre-iOS 6 route, we're bumping the min runtime target to 6 so that code should just be removed.

@richardeakin richardeakin added the ios label Sep 14, 2015

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Sep 14, 2015

Collaborator

I finally had a chance to look at this and I can't say I have a solution. AVAudioSessionInterruptionNotification doesn't seem to fire when you hit the power button (although it does when you get a phone call), so I don't think the AVAudioSession notifications are any good to us.

Trying a different angle, using the getSignalDidEnterBackground() / getSignalDidBecomeActive(), which correspond to UIApplicationDelegate's applicationDidEnterBackground: and applicationWillEnterForeground appear to be too late - I get a sudden pop when I hit the power button, and when I come back there is no audio. Afterwards, the only way to get audio for any app is to restart the device, just as you found.

I'm curious what other projects are doing. Frankly, my knowledge of iOS is far outdated as I don't do much on that platform these days (and I dread having to try, since it usually takes more than an hour just to get an app to deploy to the device from Xcode). I'd like to find a solution to this nasty problem but I don't know of any.

Collaborator

richardeakin commented Sep 14, 2015

I finally had a chance to look at this and I can't say I have a solution. AVAudioSessionInterruptionNotification doesn't seem to fire when you hit the power button (although it does when you get a phone call), so I don't think the AVAudioSession notifications are any good to us.

Trying a different angle, using the getSignalDidEnterBackground() / getSignalDidBecomeActive(), which correspond to UIApplicationDelegate's applicationDidEnterBackground: and applicationWillEnterForeground appear to be too late - I get a sudden pop when I hit the power button, and when I come back there is no audio. Afterwards, the only way to get audio for any app is to restart the device, just as you found.

I'm curious what other projects are doing. Frankly, my knowledge of iOS is far outdated as I don't do much on that platform these days (and I dread having to try, since it usually takes more than an hour just to get an app to deploy to the device from Xcode). I'd like to find a solution to this nasty problem but I don't know of any.

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Sep 14, 2015

Collaborator

Ah, a little hope - I'm able to register for AppCocoaTouch::getSignalWillResignActive() and disable the Context there - I still hear a pop and some audio ringing as if something is going through the buffer when I hit the power button, but at least when I turn the device back on I can hear audio again.

Collaborator

richardeakin commented Sep 14, 2015

Ah, a little hope - I'm able to register for AppCocoaTouch::getSignalWillResignActive() and disable the Context there - I still hear a pop and some audio ringing as if something is going through the buffer when I hit the power button, but at least when I turn the device back on I can hear audio again.

@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Sep 14, 2015

Contributor

Hmm. Thanks again for looking into this, Rich. I read up on AVAudioSessionInterruptionNotification and came to the same conclusion you did. It's for phone calls and such and won't fire when the device sleeps.

My band-aid fix has been to call audio::master()->deviceManager()->getDefaultInput() to set the session category to AVAudioSessionCategoryPlayAndRecord which allows audio to play when the device is sleeping, and when the app is in the background. This has the side-effect of causing playback to not properly switch to a bluetooth speaker though, so it's not really a long term fix.

I wonder if there is a grace period from when getSignalWillResignActive is called where it's possible to fade the audio out. I'll give that a try.

Contributor

notlion commented Sep 14, 2015

Hmm. Thanks again for looking into this, Rich. I read up on AVAudioSessionInterruptionNotification and came to the same conclusion you did. It's for phone calls and such and won't fire when the device sleeps.

My band-aid fix has been to call audio::master()->deviceManager()->getDefaultInput() to set the session category to AVAudioSessionCategoryPlayAndRecord which allows audio to play when the device is sleeping, and when the app is in the background. This has the side-effect of causing playback to not properly switch to a bluetooth speaker though, so it's not really a long term fix.

I wonder if there is a grace period from when getSignalWillResignActive is called where it's possible to fade the audio out. I'll give that a try.

@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Sep 15, 2015

Contributor

I just tested disabling audio in the getSignalWillResignActive callback. It still emits a loud beep, and kills audio for me, forcing a restart every time. I also tried fading the audio out for 0.25s and 1.0s, both disabling the output after the sound had faded completely, and not. All result in the same behavior as long as the audio session is categorized as AVAudioSessionCategoryAmbient, which is the default (and arguably should be).

Contributor

notlion commented Sep 15, 2015

I just tested disabling audio in the getSignalWillResignActive callback. It still emits a loud beep, and kills audio for me, forcing a restart every time. I also tried fading the audio out for 0.25s and 1.0s, both disabling the output after the sound had faded completely, and not. All result in the same behavior as long as the audio session is categorized as AVAudioSessionCategoryAmbient, which is the default (and arguably should be).

@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Sep 15, 2015

Contributor

FWIW the libpd Objective-C sample does not exhibit this behavior. Even when the category is set to AVAudioSessionCategoryAmbient the audio fades smoothly in and out. Comparing what Cinder is doing to the libpd implementation might be informative.

Contributor

notlion commented Sep 15, 2015

FWIW the libpd Objective-C sample does not exhibit this behavior. Even when the category is set to AVAudioSessionCategoryAmbient the audio fades smoothly in and out. Comparing what Cinder is doing to the libpd implementation might be informative.

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Dec 29, 2015

Collaborator

A different clue was reported here, where creating an InputDeviceNode (without connected to graph) suppresses the spike.

Collaborator

richardeakin commented Dec 29, 2015

A different clue was reported here, where creating an InputDeviceNode (without connected to graph) suppresses the spike.

@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Dec 29, 2015

Contributor

Does creating an input call audio::DeviceManager::getDefaultInput()? If so, it's the same as my workaround above, since AVAudioSessionCategoryPlayAndRecord causes audio to play in the background. Thus, no cut-outs when the app is backgrounded.

Contributor

notlion commented Dec 29, 2015

Does creating an input call audio::DeviceManager::getDefaultInput()? If so, it's the same as my workaround above, since AVAudioSessionCategoryPlayAndRecord causes audio to play in the background. Thus, no cut-outs when the app is backgrounded.

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Dec 29, 2015

Collaborator

Ah, you're right - missed your note about that.. sorry for being daft, but you're saying that AVAudioSessionCategoryAmbient is the culprit? If so, I wonder about AVAudioSessionCategoryPlayback or AVAudioSessionCategorySoloAmbient as well, or how the other session categories respond. You can easily test out in your own code, at the end of your app setup() do something like the following (compiled as obj-c++):

NSError *error = nil;
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:&error];

If that works, maybe its just time that we add an option to DeviceManager that toggles optimal settings for playing in the background, with it only being functional on iOS for starters.

I really haven't had any time to look into what is different in libpd's implementation, or if there have been changes to it since I originally wrote it..

Collaborator

richardeakin commented Dec 29, 2015

Ah, you're right - missed your note about that.. sorry for being daft, but you're saying that AVAudioSessionCategoryAmbient is the culprit? If so, I wonder about AVAudioSessionCategoryPlayback or AVAudioSessionCategorySoloAmbient as well, or how the other session categories respond. You can easily test out in your own code, at the end of your app setup() do something like the following (compiled as obj-c++):

NSError *error = nil;
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayback error:&error];

If that works, maybe its just time that we add an option to DeviceManager that toggles optimal settings for playing in the background, with it only being functional on iOS for starters.

I really haven't had any time to look into what is different in libpd's implementation, or if there have been changes to it since I originally wrote it..

@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Dec 30, 2015

Contributor

Here's the code in libpd that sets up the AudioSession category:

https://github.com/libpd/libpd/blob/cd26fab53e3b7f602bb859bc6cdbd0d13f571087/objc/PdAudioController.m#L164-L191

There must be something else that differs from what Cinder is doing, though. Even when that block is commented out and replaced with this, the audio still fades out nicely when the app is backgrounded.

[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:&error];
Contributor

notlion commented Dec 30, 2015

Here's the code in libpd that sets up the AudioSession category:

https://github.com/libpd/libpd/blob/cd26fab53e3b7f602bb859bc6cdbd0d13f571087/objc/PdAudioController.m#L164-L191

There must be something else that differs from what Cinder is doing, though. Even when that block is commented out and replaced with this, the audio still fades out nicely when the app is backgrounded.

[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryAmbient error:&error];
@notlion

This comment has been minimized.

Show comment
Hide comment
@notlion

notlion Aug 27, 2016

Contributor

I made some progress on handing interruptions here today.

Testing with BasicNode; If an alarm interrupts the app, audio now resumes. There's no fade-in, but i suppose that can be added by listening to the getSignalInterruptionBegan() and getSignalInterruptionEnded() signals that the DeviceManager now has. Which, is also as how the OutputNodes are notified of the interruption, as per your suggestion.

This unfortunately doesn't do anything to prevent the audio from killing the speaker on sleep, though..

Contributor

notlion commented Aug 27, 2016

I made some progress on handing interruptions here today.

Testing with BasicNode; If an alarm interrupts the app, audio now resumes. There's no fade-in, but i suppose that can be added by listening to the getSignalInterruptionBegan() and getSignalInterruptionEnded() signals that the DeviceManager now has. Which, is also as how the OutputNodes are notified of the interruption, as per your suggestion.

This unfortunately doesn't do anything to prevent the audio from killing the speaker on sleep, though..

@richardeakin

This comment has been minimized.

Show comment
Hide comment
@richardeakin

richardeakin Aug 27, 2016

Collaborator

Excellent progress, thank you @notlion! Yes I think handling the fade out / in is best done by the user since currently, we don't install a GainNode before sending out sound to the speakers. I believe only OS X has this feature built in (in the HAL audio unit) and so we decided to keep it slim at the graph level.

It is crazy that the sleep literally kills audio - I've seen it too where you have to restart your phone in order to hear anything. This doesn't happen with libpd, or JUCE?

Collaborator

richardeakin commented Aug 27, 2016

Excellent progress, thank you @notlion! Yes I think handling the fade out / in is best done by the user since currently, we don't install a GainNode before sending out sound to the speakers. I believe only OS X has this feature built in (in the HAL audio unit) and so we decided to keep it slim at the graph level.

It is crazy that the sleep literally kills audio - I've seen it too where you have to restart your phone in order to hear anything. This doesn't happen with libpd, or JUCE?

notlion added a commit to notlion/Cinder that referenced this issue Sep 26, 2016

Render multiple times if number of requested frames is larger than th…
…e internal buffer size

This should fix cinder#997, the root cause of which seems to be that numFrames jumps from 1024 to 4096 during the transition into sleep. Previously only 1024 frames were rendered, resulting in audible glitches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment