Sound device management on MAC using CoreAudio

[gbfarah]

With reference to #2445 and 64characters/Telephone@42286e6

I would like to report the following encountered issues that are a little strange

1- When setting a specific audio device that I would like used on calls using pjsua_set_snd_dev (pjsua_aud.c) I am finding that the application immediately drops the volume of other applications as if a call is in play as mentioned in threads above. How do I stop this behavour

2- After calling pjsua_set_snd_dev with the capture and playback device ids as assigned by PJSIP. I am finding that phone calls still utilize the default device of the MAC operating system. ie the set snd device appears not to work correctly. How does one simply set which devices to use for calls

3- I was previously using an older version of PJSIP with PortAudio which had the capability of detecting default device change during a call and switching device accordingly. This is particularly important as Apple Airpods function simply by changing themselves to become the default device whilst in users ear. Is there a was of enabling this behaviour in core_audio_dev

4 -In following from points above I was previously using set_snd_dev to specify device in use (whether in a call or not. Because of point 2 above it doesn't work whilst idle and seemingly has not effect during a call

Any feedback would be greatly appreciated

[eofster]

2- After calling pjsua_set_snd_dev with the capture and playback device ids as assigned by PJSIP. I am finding that phone calls still utilize the default device of the MAC operating system. ie the set snd device appears not to work correctly. How does one simply set which devices to use for calls

Strange, this function has always been working for me on the Mac.

[gbfarah]

Thanks for the followup. I ran the following in a test harness. The strange thing is that although from a logs perspective PJSIP is claiming to connect the right devices to conference (in this case a Logitect BluetoothHeadset) the audio is actually physically being played on the default device.

There are 4 devices reported by PJSIP as below

 Device 0, Built-in Microphone (capture=2, playback=0)
 Device 1, Built-in Output (capture=0, playback=2)
 Device 2, H800 Logitech Headset (capture=1, playback=0)
 Device 3, H800 Logitech Headset (capture=0, playback=2).

These were extracted with

int i, count;
pjmedia_aud_dev_refresh() ;
count = pjmedia_snd_get_dev_count();
for (i=0; i<count; ++i) 
{
  const pjmedia_snd_dev_info *info;
  info = pjmedia_snd_get_dev_info(i);
  PJ_LOG(1,(THIS_FILE, "Device %d, %s (capture=%d, playback=%d)",i, info->name, info->input_count, info->output_count));
}

I do however notice that core audio is printing the lines below in the logs but I can't see how the device IDs (40,59,69,73) are exposed by pjmedia

18:29:37.413        coreaudio_dev.c   dev_id 40: Built-in Microphone  (in=2, out=0) 44100Hz
18:29:37.414        coreaudio_dev.c   dev_id 59: Built-in Output  (in=0, out=2) 44100Hz
18:29:37.414        coreaudio_dev.c   dev_id 69: H800 Logitech Headset  (in=1, out=0) 8000Hz
18:29:37.415        coreaudio_dev.c   dev_id 73: H800 Logitech Headset  (in=0, out=2) 48000Hz

As such I set the sound devices as below with pjsua_set_snd_dev( 2, 3 ) where 2 and 3 are the IDs based on index above

18:29:37.415            pjsua_aud.c  Set sound device: capture=2, playback=3
18:29:37.415            pjsua_aud.c  .Opening sound device (speaker + mic) PCM@16000/1/20ms

During a phone call pjmedia is claiming conferencing of the correct device however audio is being heard on the Built-In output which is the current default device

18:29:40.396          pjsua_media.c  ......Audio updated, stream #0: PCMU (sendrecv)
18:29:40.396            pjsua_aud.c  .....Conf connect: 1 --> 0
18:29:40.396           conference.c  ......Port 1 (sip:xxxxx@xxxxx.com) transmitting to port 0 (H800 Logitech Headset)
18:29:40.396            pjsua_aud.c  .....Conf connect: 0 --> 1
18:29:40.396           conference.c  ......Port 0 (H800 Logitech Headset) transmitting to port 1 (sip:xxxxx@xxxxxx.com)

[sauwming]

Make sure that hardware EC is disabled, otherwise audio device settings may get ignored, i.e.:

09:58:01.576        coreaudio_dev.c  ...Using VoiceProcessingIO audio unit
09:58:01.576        coreaudio_dev.c  ...Warning: audio device id settings are ignored when using VPIO

[gbfarah]

Thanks for that feedback. Can you shed some light on how to do this via pjsua. Also I would presume hardware acoustic echo canceller is quite important for VoIP calls (ie wouldn't disabling it have adverse effect on call quality ? )

Thanks

[sauwming]

Set PJMEDIA_ECHO_USE_SW_ECHO in pjsua_media_config.ec_options.

Yes, it provides a superior performance, but it only works with specific hardware, i.e. their built-in sound device.

I will close this issue. For further questions, please use any other means such as StackOverflow.

[gbfarah]

Respectfully , I think this issue has been closed prematurely. After adjusting PJMEDIA_ECHO_USE_SW_ECHO to be enabled this has resolved selection of specific device and this can be done at any time in or out of a call . Thanks

Unfortunately, the following issues still remain- some of which are new. Please note in all of examples below default device selection is done by manually selecting device under MAC preferences

1. Default device actually not identified correctly (bug?). As a result the call is being played through wrong device. With hardware EC previously enabled PJSIP correctly picks the correct default device on which a call is to be played. With software echo canceller enabled PJSIP believes it is using the default device as below. In example below default device selected on OS is in fact Built-in Output and Built-in Microphone not the Logitech (69)

13:19:33.079        coreaudio_dev.c  core audio detected 3 devices
13:19:33.079        coreaudio_dev.c   dev_id 69: Logitech Wireless Headset  (in=1, out=2) 48000Hz
13:19:33.080        coreaudio_dev.c   dev_id 59: Built-in Output  (in=0, out=2) 44100Hz
13:19:33.081        coreaudio_dev.c   dev_id 40: Built-in Microphone  (in=2, out=0) 44100Hz



13:19:33.081            pjsua_aud.c  Set sound device: capture=-1, playback=-2
13:19:33.134            pjsua_aud.c  .Opening sound device (speaker + mic) PCM@16000/1/20ms
13:19:33.134        coreaudio_dev.c  ..Using default audio unit
13:19:33.136        coreaudio_dev.c  ..Setting current device 69     -------------- (WRONG DEVICE!!!!)
13:19:33.137        coreaudio_dev.c  ..Creating audio resample from 48000 to 16000
13:19:33.143       ec0x7ff0734878c0  ..Creating Speex AEC
13:19:33.143       ec0x7ff0734878c0  ..Speex AEC created, clock_rate=16000, channel=1, samples per frame=320, tail length=200 ms, latency=0 ms
13:19:33.143        coreaudio_dev.c  ..core audio stream started

2. Default device change when not in a call . Because of the above issue this is also erroneous. With hardware EC enabled this also works correctly. Changing default device when not in call results in the same issue as in (1)

3. Default device change mid call. This change is not detected at all. This one is remarkably important for airpod support. When an operator physically takes off/puts on an airpod , the OS automatically switches the default device. When this happens midcall the expectation is the call should continue on the new default device as selected by the OS

[sauwming]

Default device is only identified during startup and it won't change unless you call pjmedia_aud_dev_refresh() (please refer to: https://trac.pjsip.org/repos/wiki/FAQ#snd-hot-plug).

[gbfarah]

The precise issue is that even at startup the incorrect device is picked up as the default. This is only happening with SW echo canceller enabled. The logs above are at startup

[sauwming]

Can you check what default I/O device is returned by the code here:
https://github.com/pjsip/pjproject/blob/master/pjmedia/src/pjmedia-audiodev/coreaudio_dev.m#L492

[gbfarah]

Thanks for your help thus far. Something is a little strange. I added logging to code snippet below

/* Find default audio input device */
	addr.mSelector = kAudioHardwarePropertyDefaultInputDevice;
	addr.mScope = kAudioObjectPropertyScopeGlobal;
	addr.mElement = kAudioObjectPropertyElementMaster;
	size = sizeof(dev_id);
	
	ostatus = AudioObjectGetPropertyData(kAudioObjectSystemObject,
					     &addr, 0, NULL,
					     &size, (void *)&dev_id);
	if (ostatus == noErr && dev_id != dev_ids[idx]) {
	    AudioDeviceID temp_id = dev_ids[idx];
	    
	    for (i = idx + 1; i < dev_count; i++) {
		if (dev_ids[i] == dev_id) {
		    dev_ids[idx++] = dev_id;
		    dev_ids[i] = temp_id;
		    break;
		}
	    }
		PJ_LOG(4, (THIS_FILE, " ****** Input temp_id  is %d",
		       temp_id
			));
	    		
		
	}

	/* Find default audio output device */
	addr.mSelector = kAudioHardwarePropertyDefaultOutputDevice;	
	ostatus = AudioObjectGetPropertyData(kAudioObjectSystemObject,
					     &addr, 0, NULL,
					     &size, (void *)&dev_id);
	if (ostatus == noErr && dev_id != dev_ids[idx]) {
	    AudioDeviceID temp_id = dev_ids[idx];
	    
	    for (i = idx + 1; i < dev_count; i++) {
		if (dev_ids[i] == dev_id) {
		    dev_ids[idx] = dev_id;
		    dev_ids[i] = temp_id;
		    break;
		}
	    }
		PJ_LOG(4, (THIS_FILE, " ****** Output temp_id  is %d",
		       temp_id
			));
	    		
		
		
	}}

Curiously the only logs printed on startup are as below the temp_id printed for an output device is actually the microphone. The log for input device is not hit

22:53:11.571        coreaudio_dev.c  ..core audio detected 3 devices
22:53:11.571        coreaudio_dev.c  .. ****** Output temp_id  is 40
22:53:11.574        coreaudio_dev.c  .. dev_id 59: Built-in Output  (in=0, out=2) 44100Hz
22:53:11.575        coreaudio_dev.c  .. dev_id 40: Built-in Microphone  (in=2, out=0) 44100Hz
22:53:11.576        coreaudio_dev.c  .. dev_id 69: Logitech Wireless Headset  (in=1, out=2) 48000Hz
22:53:11.576        coreaudio_dev.c  ..core audio initialized

Also when making a call I get

22:53:33.081            pjsua_aud.c  Set sound device: capture=-1, playback=-2
22:53:33.134            pjsua_aud.c  .Opening sound device (speaker + mic) PCM@16000/1/20ms
22:53:33.134        coreaudio_dev.c  ..Using default audio unit
22:53:33.136        coreaudio_dev.c  ..Setting current device 69     -------------- (WRONG DEVICE!!!!)
22:53:33.137        coreaudio_dev.c  ..Creating audio resample from 48000 to 16000
22:53:33.143       ec0x7ff0734878c0  ..Creating Speex AEC
22:53:33.143       ec0x7ff0734878c0  ..Speex AEC created, clock_rate=16000, channel=1, samples per frame=320, tail length=200 ms, latency=0 ms
22:53:33.143        coreaudio_dev.c  ..core audio stream started

[gbfarah]

Would it be possible to reopen this ticket as I believe the above log is showing incorrect behaviour