iOS Callee Side audio issue and Audiounit Can't work with iOS CallKit #784

kannanmaran · 2019-08-26T10:49:11Z

Hi,

Issue 1

No audio in Callee side. Caller side can able to hear the audio. this issue is having in iOS only. the same library version for android it just works fine in both side audio.

audiounit: AudioRouteChange - reason 8
�[31mmain: long async blocking: 853>100 ms (h=0x10078c5dc arg=0x2819e45c0)
�[;maudiounit: AudioRouteChange - reason 8
audiounit: AudioRouteChange - reason 8

audio: Set audio decoder: PCMA 8000Hz 1ch
audio: player started with sample format S16LE
audio: Set audio encoder: PCMA 8000Hz 1ch
audiounit: AudioRouteChange - reason 8
audiounit: AudioRouteChange - reason 8
audiounit: record: enable resampler 0.0 -> 8000 Hz
2019-08-26 16:02:30.808738+0530 App[1590:593027] [aid] AudioIssueDetectorNode.cpp:161:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /BuildRoot/Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-251.70.4.1/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:597
audio: source started with sample format S16LE
audio tx pipeline: audiounit ---> vumeter ---> PCMA
audio rx pipeline: audiounit <--- vumeter <--- PCMA

stream: incoming rtp for 'audio' established, receiving from xxx.xxx.xxx.xxx:54504

XXXX@XXX.XXX.XXX.XXX: Call established: sip:XXXX@XXX.XXX.XXX.XXX
2019-08-26 16:02:30.838917+0530 App[1590:593027] ua event (CALL_ESTABLISHED) sip:XXXX@XXX.XXX.XXX.XXX
audiounit: AudioRouteChange - reason 8
audiounit: AudioRouteChange - reason 8
2019-08-26 16:02:31.070211+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.071446+0530 App[1590:593027] handleStatus = event = call established uap = 4318049984 callp = 4316077984

audiounit: AudioRouteChange - reason 8
�[31mmain: long async blocking: 1627>100 ms (h=0x10078c5dc arg=0x2819e45c0)
�[;m2019-08-26 16:02:31.138418+0530 App[1590:593027] ua event (VU_TX_REPORT) -44.72
stream: incoming rtp for 'audio' established, receiving from 192.168.113.173:54504
2019-08-26 16:02:31.139566+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.162756+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.185926+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.209104+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.232365+0530 App[1590:593158] 55: EXCEPTION (-1): ""

[0:00:01] audio=0/0 (bit/s)
2019-08-26 16:02:31.255618+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.278882+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.302090+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:31.325543+0530 App[1590:593158] 55: EXCEPTION (-1): ""

[0:00:04] audio=354868/0 (bit/s)
2019-08-26 16:02:34.275237+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:34.298633+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:34.321680+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:34.345052+0530 App[1590:593158] 55: EXCEPTION (-1): ""

[0:00:07] audio=354868/0 (bit/s)
2019-08-26 16:02:37.084999+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.108330+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.131440+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.154925+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.166286+0530 App[1590:593027] ua event (VU_TX_REPORT) -41.90
2019-08-26 16:02:37.167796+0530 App[1590:593027] ua event (VU_RX_REPORT) -59.71
2019-08-26 16:02:37.177911+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.200468+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.224164+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.247441+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.270688+0530 App[1590:593158] 55: EXCEPTION (-1): ""

[0:00:07] audio=351882/63552 (bit/s)
2019-08-26 16:02:37.293810+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.317203+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.340251+0530 App[1590:593158] 55: EXCEPTION (-1): ""
2019-08-26 16:02:37.363525+0530 App[1590:593158] 55: EXCEPTION (-1): ""

Issue 2

getting this error with ios callkit for incoming call. outgoing call it works fine.
and also i checked without callkit its working fine for both incoming and outgoing calls. when using callkit only having this issue.

answering call from sip:xxxx@xxx.xxx.xxx.xxx with 200
audio: Set audio decoder: PCMA 8000Hz 1ch
�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_player failed (audiounit.nil): Function not implemented
�[;maudio: Set audio encoder: PCMA 8000Hz 1ch
�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_player failed (audiounit.nil): Function not implemented
�[;m�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_source failed (audiounit.nil): Function not implementedaudio: Set audio decoder: PCMA 8000Hz 1ch
�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_player failed (audiounit.nil): Function not implemented
�[;maudio: Set audio encoder: PCMA 8000Hz 1ch
�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_player failed (audiounit.nil): Function not implemented
�[;m�[31maudiounit: AudioSessionSetActive: 561017449
�[;m�[31maudio: start_source failed (audiounit.nil): Function not implemented

alfredh · 2019-08-27T12:23:49Z

which version of iOS and Xcode are you using ?

can you find out what this reason value means:

audiounit: AudioRouteChange - reason 8

kannanmaran · 2019-08-28T06:24:18Z

Hello @alfredh

In the above i mentioned issue 2 is fixed. it's due to i called the ua_answer method inside iOS Callkit -> performAnswerCallAction method. Now am calling the ua_answer inside Callkit -> didActivateAudioSession callBack method. its working fine with Callkit. But still am having the Audio issue in Callee side pls look into it.

i tried with this old lib https://github.com/miche-atucha/taresip. it's working fine. am able to hear the audio in both side.

Current version having the issue. i compiled from this link for ios

These are the version am using
BARESIP_SRC := baresip-0.6.3.tar.gz
LIBREM_SRC := rem-0.6.0.tar.gz
LIBRE_SRC := re-0.6.0.tar.gz

iOS Version : 12.3.1
Xcode : 10.2.1

audiounit: AudioRouteChange - reason 8

AVAudioSessionRouteChangeReasonRouteConfigurationChange

here is the link for the print
https://github.com/alfredh/baresip/blob/9df4f3c2e7042723db65adf858f8bb9167ba5f11/modules/audiounit/sess.c#L52

alfredh · 2019-08-28T10:30:02Z

how did you build baresip for ios? did you use this project:

https://github.com/alfredh/baresip-ios

if I change the target version to 7.0 we get some warnings:

  CC [m]  /Users/alfredh/git/baresip-ios/build/aarch64/baresip/modules/audiounit/audiounit.o
modules/audiounit/audiounit.c:162:8: warning: 'AudioSessionInitialize' is deprecated: first deprecated in iOS
      7.0 [-Wdeprecated-declarations]
        ret = AudioSessionInitialize(NULL, NULL, interruptionListener, 0);
              ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:649:2: note: 
      'AudioSessionInitialize' has been explicitly marked deprecated here
        AudioSessionInitialize(             CFRunLoopRef                        inRunLoop, 
        ^
1 warning generated.

perhaps it is time to upgrade the audiounit module to use more recent APIs ?

https://developer.apple.com/documentation/audiotoolbox/1618360-audiosessioninitialize

https://stackoverflow.com/questions/19710046/a-fix-for-audiosessioninitialize-deprecated

kannanmaran · 2019-08-28T13:46:26Z

i didn't change the target version. i build the same target version as you defined 6.0.

yes we have to update the AudioUnit module to use the latest API. i will try to build with 7.0 target and update you..

Update!!!
i compiled with 9.0 target and am getting the same warning

modules/audiounit/sess.c:72:2: warning: 'AudioSessionRemovePropertyListenerWithUserData' is deprecated: first deprecated in iOS 7.0 - no longer supported [-Wdeprecated-declarations]
AudioSessionRemovePropertyListenerWithUserData(id, propListener, as);
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:820:2: note:
'AudioSessionRemovePropertyListenerWithUserData' has been explicitly marked deprecated here
AudioSessionRemovePropertyListenerWithUserData( AudioSessionPropertyID inID,
^
modules/audiounit/sess.c:73:2: warning: 'AudioSessionSetActive' is deprecated: first deprecated in iOS 7.0 - no longer supported [-Wdeprecated-declarations]
AudioSessionSetActive(false);
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:667:2: note: 'AudioSessionSetActive' has been
explicitly marked deprecated here
AudioSessionSetActive( Boolean active)
^
modules/audiounit/sess.c:101:8: warning: 'AudioSessionSetProperty' is deprecated: first deprecated in iOS 7.0 - no longer supported [-Wdeprecated-declarations]
ret = AudioSessionSetProperty(kAudioSessionProperty_AudioCategory,
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:748:2: note: 'AudioSessionSetProperty' has been
explicitly marked deprecated here
AudioSessionSetProperty( AudioSessionPropertyID inID,
^
modules/audiounit/sess.c:117:8: warning: 'AudioSessionSetActive' is deprecated: first deprecated in iOS 7.0 - no longer supported [-Wdeprecated-declarations]
ret = AudioSessionSetActive(true);
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:667:2: note: 'AudioSessionSetActive' has been
explicitly marked deprecated here
AudioSessionSetActive( Boolean active)
^
modules/audiounit/sess.c:124:8: warning: 'AudioSessionAddPropertyListener' is deprecated: first deprecated in iOS 7.0 - no longer supported [-Wdeprecated-declarations]
ret = AudioSessionAddPropertyListener(id, propListener, as);
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator12.2.sdk/System/Library/Frameworks/AudioToolbox.framework/Headers/AudioSession.h:786:2: note: 'AudioSessionAddPropertyListener'
has been explicitly marked deprecated here
AudioSessionAddPropertyListener( AudioSessionPropertyID inID,

alfredh · 2019-08-30T11:05:47Z

can you please try to replace calls to deprecated functions with calls
to updated functions and make sure the code is building with zero warnings ?

kannanmaran · 2019-09-04T06:19:04Z

Hello @alfredh
Sure will update it and let you know.

alfredh · 2019-09-07T14:55:04Z

did you manage to update the code ?

I also noticed some other strange things in your logs:

audiounit: record: enable resampler 0.0 -> 8000 Hz

the value 0.0 should be a valid samplerate e.g. 48000

audiounit: AudioSessionSetActive: 561017449

the error code means AVAudioSessionErrorCodeInsufficientPriority

alfredh · 2019-09-08T18:03:14Z

@kannanmaran

if the Hardware Sample rate is 0.0 Hz then the recording will probably
not work at all.

could you please test this patch:

diff --git a/modules/audiounit/recorder.c b/modules/audiounit/recorder.c
index 4b891f6..2bd80e0 100644
--- a/modules/audiounit/recorder.c
+++ b/modules/audiounit/recorder.c
@@ -268,7 +268,7 @@ int audiounit_recorder_alloc(struct ausrc_st **stp, const struct ausrc *as,
 
        ret = AudioUnitGetProperty(st->au_in,
                                   kAudioUnitProperty_SampleRate,
-                                  kAudioUnitScope_Input,
+                                  kAudioUnitScope_Output,
                                   inputBus,
                                   &hw_srate,
                                   &hw_size);

an AudioUnit has 2 "scopes", input and output. for the recording audiounit
the "output" scope is the one interfacing to the application.

I have tested the patch on OSX and it reports the correct Hardware samplerate.

kannanmaran · 2019-09-09T11:12:07Z

Hello @alfredh

i updated your patch and recompiled the lib. But still am facing the same issue

record: enable resampler 0.0 -> 8000 Hz

ret = AudioUnitGetProperty(st->au_in, kAudioUnitProperty_SampleRate, kAudioUnitScope_Output, inputBus, &hw_srate, &hw_size);

am i missing something?

audio: Set audio decoder: PCMA 8000Hz 1ch
audio: player started with sample format S16LE
audio: Set audio encoder: PCMA 8000Hz 1ch
audiounit: record hardware sample rate is now at 0.000000 Hz
audiounit: AudioRouteChange - reason 8
audiounit: AudioRouteChange - reason 8
audiounit: record: enable resampler 0.0 -> 8000 Hz
2019-09-09 16:39:31.195229+0530 App[3964:3173266] [aid] AudioIssueDetectorNode.cpp:161:Initialize: Caught analyzer graph exception 1718775073 !mrf slice duration must be set before configure. in /BuildRoot/Library/Caches/com.apple.xbs/Sources/AudioDSP/AudioDSP-251.70.4.1/CoreAudioUtility/Source/CADSP/DSPGraph/DSPGraph_Graph.cpp:597
audio: source started with sample format S16LE
audio tx pipeline: audiounit ---> vumeter ---> PCMA
audio rx pipeline: audiounit <--- vumeter <--- PCMA

CanaJalil · 2019-09-10T00:29:20Z

on iOS the following
ret = AudioUnitGetProperty(st->au_in, kAudioUnitProperty_SampleRate, kAudioUnitScope_Input, inputBus, &hw_srate, &hw_size);
ret = AudioUnitGetProperty(st->au_in, kAudioUnitProperty_SampleRate, kAudioUnitScope_Output, inputBus, &hw_srate, &hw_size);
both return 0 for hw_srate
I fixed the issue with sampe rate (work around)
after line 282 of audio unit/recorder.c
I did the followings
if (-0.01f <= hw_srate || hw_srate >= 0.1.f)
{
fmt.mSampleRate = prm->state;
fat->sample_rate = 1.0f;
}

then encode start working and stop having garbled audio, I know this not the solution, but work for now!

kannanmaran · 2019-09-10T04:57:13Z

Hello @CanaJalil

Thank you so much for your work around. Its working fine now. in your comments some typo in code. Below code is working fine.

after line 282 of audio unit/recorder.c

if (-0.01f <= hw_srate || hw_srate >= 0.1f){
    fmt.mSampleRate = prm->srate;
		// fmt->sample_rate = 1.0f;
}

alfredh · 2019-09-10T13:00:42Z

thanks for the patch

can you please test this patch:

diff --git a/modules/audiounit/recorder.c b/modules/audiounit/recorder.c
index 4b891f6..b212a79 100644
--- a/modules/audiounit/recorder.c
+++ b/modules/audiounit/recorder.c
@@ -266,6 +266,10 @@ int audiounit_recorder_alloc(struct ausrc_st **stp, const struct ausrc *as,
                goto out;
 #endif
 
+#if TARGET_OS_IPHONE
+       hw_srate = prm->srate;
+       (void)hw_size;
+#else
        ret = AudioUnitGetProperty(st->au_in,
                                   kAudioUnitProperty_SampleRate,
                                   kAudioUnitScope_Input,
@@ -274,6 +278,7 @@ int audiounit_recorder_alloc(struct ausrc_st **stp, const struct ausrc *as,
                                   &hw_size);
        if (ret)
                goto out;
+#endif
 
        debug("audiounit: record hardware sample rate is now at %f Hz\n",
              hw_srate);

please test the following:

Different sample rates (8000Hz, 16000Hz, 48000Hz etc.)
Outgoing and incoming call
Different iPhone / iOS devices

kannanmaran · 2019-09-11T13:33:07Z

Hello @alfredh

I tested the patch. its working fine in iOS devices. Thanks for your update. you can close the issue.

alfredh · 2019-09-11T14:07:57Z

Great, I pushed the patch to master.

Thanks for reporting the bug.

alfredh added the iOS label Aug 27, 2019

alfredh closed this as completed in b9f76da Sep 11, 2019

LonestarX91 mentioned this issue Jul 15, 2020

AudioUnit hell baresip/baresip-ios#28

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

iOS Callee Side audio issue and Audiounit Can't work with iOS CallKit #784

iOS Callee Side audio issue and Audiounit Can't work with iOS CallKit #784

kannanmaran commented Aug 26, 2019

Issue 1

Issue 2

alfredh commented Aug 27, 2019

kannanmaran commented Aug 28, 2019 •

edited

alfredh commented Aug 28, 2019 •

edited

kannanmaran commented Aug 28, 2019 •

edited

alfredh commented Aug 30, 2019

kannanmaran commented Sep 4, 2019

alfredh commented Sep 7, 2019

alfredh commented Sep 8, 2019

kannanmaran commented Sep 9, 2019

CanaJalil commented Sep 10, 2019 •

edited

kannanmaran commented Sep 10, 2019 •

edited

alfredh commented Sep 10, 2019

kannanmaran commented Sep 11, 2019

alfredh commented Sep 11, 2019

iOS Callee Side audio issue and Audiounit Can't work with iOS CallKit #784

iOS Callee Side audio issue and Audiounit Can't work with iOS CallKit #784

Comments

kannanmaran commented Aug 26, 2019

Issue 1

Issue 2

alfredh commented Aug 27, 2019

kannanmaran commented Aug 28, 2019 • edited

alfredh commented Aug 28, 2019 • edited

kannanmaran commented Aug 28, 2019 • edited

alfredh commented Aug 30, 2019

kannanmaran commented Sep 4, 2019

alfredh commented Sep 7, 2019

alfredh commented Sep 8, 2019

kannanmaran commented Sep 9, 2019

CanaJalil commented Sep 10, 2019 • edited

kannanmaran commented Sep 10, 2019 • edited

alfredh commented Sep 10, 2019

kannanmaran commented Sep 11, 2019

alfredh commented Sep 11, 2019

kannanmaran commented Aug 28, 2019 •

edited

alfredh commented Aug 28, 2019 •

edited

kannanmaran commented Aug 28, 2019 •

edited

CanaJalil commented Sep 10, 2019 •

edited

kannanmaran commented Sep 10, 2019 •

edited