Adding mSBC mode into HFP

[kuikka]

Great piece of software! I am currently working on integrating this into our product in lieu of pulseaudio due to former's excessive CPU usage with BT audio.

I am enhancing the HFP support by adding more AT command handling (some headsets drop the connection if the standard handshake is not followed) and by adding mSBC support.

Before I dive in to the mSBC I wanted to ask if you have thought about how to best implement it? The problematic detail is that the codec negotiation happens in the RFCOMM thread but the information needs to go to both the SCO IO thread and into the control thread since the sampling rates differ between CVSD and mSBC.

I could either delay the creation of the accompanying SCO transport until the codec has been decided upon, or add a more complex logic, maybe a fifo instead of the eventfd to pass more complex communications between the threads.

There is also a race condition where the plugin may attempt to grab the SCO transport before the AT negotiation has finished, this leads me to prefer delaying the creation of the SCO IO thread.

Thoughts?

[arkq]

Hi,

First of all, I'm glad, that you find this software useful.

Currently, there is no HFP support what so ever. The RFCOMM and SCO IO threads are tailored for HSP profile. I thought for a while about actual HFP support, however I don't know how to accomplish that and keep bluealsa simple (bluealsa should be only for audio management). I thought about forwarding AT command to some external "server", which will handle all HFP-related actions (there is much more in the HFP specification, than simple audio transfer). There is a project called nohands, however is seems that integrating it with bluealsa might not be so simple.

So basically, this issue is an open question. What I would not want to do, is to make yet another bloatware, which will be impossible to maintain :) I'm scarred a little bit right now, to be honest, because the code is getting out of control in few places - scheduled for future refactoring.

My first thought about HFP, is that, the support for this profile should be separated from the HSP one - even though the HFP is a successor of the HSP and the HSP can be handled (?) with the HFP implementation. But I'm open to any suggestions - maintainability is a very strong argument.

As for communication problem between the RFCOMM and SCO threads.... Yes, that's a nightmare. The eventfd is used only because there was no need for anything fancy, not yet. So, if there is a need for some logic within the communication channel, FIFO seems to be OK.

To answer to your question more thoroughly, I will have to read once more the HFP specification. I will try to share some constructive thoughts in a next few days.

[kuikka]

I should explain.

We are looking to use HFP over HSP only because it supports mSBC and therefore offers better audio quality compared to HSP. If we could do A2DP full duplex that would be awesome but no devices support it.

I have no need to implement any of the cellular-specific functionality, outside of what little handshaking is required to complete the control channel establishment and codec negotiation. I like that bluez-alsa is audio only and I would like to keep it that way. Just make it a little better by using mSBC.

Based on what you write, would you like to see both the rfcomm and io threads separated into hsp&hfp models? I.e. separate transports types? While cleaner, it would lead to some code multiplication though some could probably be shared.

[arkq]

I'm just after a brief reading of the "Hands-Free Profile 1.7.1" :)

So, it seems, that indeed there is no need to separate RFCOMM transports. It should be possible to handle both HFP and HSP with a single logic.

It is more complicated with the SCO IO thread, but I'm leaving it up to you. I do not know how the eSCO link is implemented in the bluez. If it is transparent to the user (encoding/decoding is performed in the bluez), the sco thread can be shared. If the encoding/decoding process is simple enough, that there is no need for lots of "ifs" in the current IO thread, this process can be incorporated in the io_thread_sco. However, if the eSCO link is somehow different from the current SCO link implementation, I would rather make it as a separate IO thread function.

I have no need to implement any of the cellular-specific functionality

That's fine. However, keep in mind, that in the future I would like to make such a support, but I would like to implement it as an "external" functionality.

delaying the creation of the SCO IO thread.

I think, that the thread can be created, but the SCO acquisition should be delayed. I might be wrong, though. Like I said before, I don't know how the eSCO link should be handled :|

[kuikka]

Indeed it is possible to handle both with same logic, I am doing that currently.

The eSCO is a detail the kernel takes care of, if it detects that the link supports it, eSCO will be used. There is one socket option (BT_VOICE) that needs to be passed to change the SCO socket to transparent mode but that's it.

The problem is not the SCO thread creation per se but the transport creation. After the transport is created (currently it happens immediately when the profile is connected to) an application can try to use it. With mSBC this can lead to trouble since the sample rate differs from CVSD, the application can grab the transport at a point where we don't know yet whether it is CVSD or mSBC.

[arkq]

OK, so right now I'm fully aware of the problem you've got. Thanks for this explanation.

If you think, that it will be "cleaner" to postpone SCO transport creation, that's OK.

In my current design, the SCO and RFCOMM transports are tightly coupled - in order to make sure that there will be no dangling pointer, everything is initialized in the transport_new_rfcomm function. What I would do (I think), it will be a "flag"/enum in the sco structure (in the ba_transport structure) which will be indicating selected codec type (SBC, mSBC, CVSD and "undefined"). It will be required anyway, in order to determine the sampling rate (for the transport_get_sampling function). So, if the codec is undefined, the transport will not be exposed in the ctl list_transports request. What do you think?

[bencoh]

Hey guys,
First and foremost thanks a lot for this project! As many others I'm not really keen on using pulseaudio "just" to be able to handle audio services, and as such I am currently trying to integrate bluez-alsa into our product.
A2DP works here (both playback and capture), and I eventually tried using bluez-alsa as a HFP unit (connect to iOS/android phones), which did not work for me thus far.

It looks like bluetoothd eventually receives a connection reset event on the rfcomm socket, leading to bluealsa closing the RFCOMM/HFP thread(s).
"""
Jan 7 04:55:32 MyMe daemon.err bluetoothd[518]: Unable to get io data for Hands-Free unit: getpeername: Transport endpoint is not connected (107)
Jan 7 04:55:32 MyMe daemon.debug bluetoothd[518]: src/service.c:change_state() 0xfd1fa0: device 90:60:F1:3C:11:E3 profile Hands-Free unit state changed: connected -> disconnected (0)
Jan 7 04:55:32 MyMe daemon.debug bluetoothd[518]: src/service.c:btd_service_unref() 0xfd1fa0: ref=2
"""
"""
bluealsa: bluez.c:438: A2DP Sink (SBC) configured for device 90:60:F1:3C:11:E3
bluealsa: bluez.c:440: Configuration: channels: 2, sampling: 44100
bluealsa: transport.c:683: State transition: 0 -> 0
bluealsa: bluez.c:1039: Signal: PropertiesChanged: org.bluez.MediaTransport1
bluealsa: io.c:1150: RFCOMM disconnected: Connection reset by peer
bluealsa: transport.c:683: State transition: 2 -> 4
bluealsa: transport.c:851: Closing RFCOMM: 9
bluealsa: transport.c:402: Freeing transport: HFP Hands-Free
bluealsa: transport.c:402: Freeing transport: HFP Hands-Free
bluealsa: transport.c:66: node=/tmp/bluealsa/bluealsa:HCI=hci0,DEV=90:60:F1:3C:11:E3,PROFILE=sco
bluealsa: transport.c:66: node=/tmp/bluealsa/bluealsa:HCI=hci0,DEV=90:60:F1:3C:11:E3,PROFILE=rfcomm
"""

I've read the current thread, juha's pull request and his different work branches, and was wondering if any of you guys succeeded with this? Note I fully understand HFP in bluez-alsa is not exactly ready for production, but I wouldn't mind helping with debugging it if possible :-)

[arkq]

Hi,

Currently I'm not involved with the HFP support (It's sad to say, by I haven't tested kukika's PR yet - regarding HFP). I'm still working on PCM ALSA plugin....

[arkq]

Hi,

After a long battle with other issues, I've finally started to work with this feature. I've managed to "merge" your mSBC code with the master branch (it's rather far from the original code, however the logic is still there, I hope). It's not pushed to the master branch yet, because I've got one problem...

I've got Jabra MOVE v2.3.0 headset, which I'm using for testing. Fortunately, it supports mSBC codec, so it seems to be a nice test-case for this feature. I'm able to receive and decode signal from the microphone (yeah!), however sending data to the headset does not produce any audio... If writing MTU is set to 24 bytes, then some glitches are audible, however I wouldn't say that I can hear sound.

I've tried to use your code (checkout onto the PR commit), however it's the same issue. Microphone works, speaker is silent. So, I've got a question: have you been able to send mSBC signal to BT headset (or some other device) using the code from the PR? Because, maybe my headset is broken and the code itself is OK. @kuikka (or anyone who stumbles upon this issue :)) could you test msbc branch and provide some info whether it works for you or not?

Side note:
I've checked if encoded data is transferred correctly, and it seems so. The check was done as follows:

• record BT packages with: tcpdump -pn -i bluetooth0 -w /tmp/bt-msbc
• select SCO packages using wireshark filter: hci_h4.type == 0x3 && hci_h4.direction == 0
• extract payload from the HCI SCO package using some linux console voodoo
• feed raw data through the msbc_decode() function and save PCM (PCM was playable)

[asari-fd]

Hi,
Many thanks to @arkq and @kuikka for the great software!

I'd love to record wideband audio with my BT headsets, so I built @arkq's msbc branch, and tested it with my LG HBS-780 (healthy) and Jabra Halo Smart (speakers broken).

Following arecord command outputs wideband audio PCM stream (totally I don't understand what SCO is, but it's just playable):

arecord -t raw -f S16_LE -c1 -r16000 -D bluealsa:HCI=hci0,DEV=XX:XX:XX:XX:XX:XX,PROFILE=sco

... but similar aplay command plays silence for me.

During recording, bluealsa process sometimes shows following messages. I don't know whether it's related to this topic or not though.

bluealsa: mSBC decoding error: No such process
bluealsa: mSBC decoding error: No such file or directory

[arkq]

@asari-fd many thank for your testing and reply!

So it seems, that the playback part is not working after all... The problem might be on my side, however it's hard to tell. I will try to tinker a little bit more.

During recording, bluealsa process sometimes shows following messages [...]

Yeap, I've seen something similar. The problem is definitely in bluealsa code. However, this part (recording) sometimes works, so the game is not lost completely :).

[asari-fd]

Hi,

I got another Jabra Halo Smart (healthy) from Amazon, so I tried @arkq's msbc branch again.
I checked with two Linux machines and two headsets. On Ubuntu 16.04 machine ("Machine 1"), I could see both arecord and aplay working OK. But on Ubuntu 17.10 machine ("Machine 2"), I see arecord recording a narrow band data in a wide band format. The two headsets worked generally in the same way.

Headsets:
HBS-780 (healthy as last time) and Jabra Halo Smart (healthy now)

Tried commands:

arecord -t raw -f S16_LE -c1 -r16000 -D bluealsa:HCI=hci0,DEV=XX:XX:XX:XX:XX:XX,PROFILE=sco /tmp/XX.raw
aplay -t raw -f S16_LE -c1 -r16000 -D bluealsa:HCI=hci0,DEV=XX:XX:XX:XX:XX:XX,PROFILE=sco /tmp/XX.raw

Machine 1:

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.3 LTS
Release:	16.04
Codename:	xenial

$ uname -a 
Linux XXXXXXX 4.4.0-97-generic #120-Ubuntu SMP Tue Sep 19 17:28:18 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg -l libasound2-dev libsbc-dev libbluetooth-dev
 :
ii  libasound2-dev:amd64      1.1.0-0ubuntu1    amd64             shared library for ALSA applications -- development fil
ii  libbluetooth-dev          5.37-0ubuntu5.1   amd64             Development files for using the BlueZ Linux Bluetooth l
ii  libsbc-dev:amd64          1.3-1             amd64             Sub Band CODEC library - development

$ lspci
 :
03:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8821AE 802.11ac PCIe Wireless Network Adapter

Machine 2:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu Artful Aardvark (development branch)
Release:        17.10
Codename:       artful

$ uname -a
Linux XXXXXX 4.13.0-15-generic #16-Ubuntu SMP Wed Oct 4 21:59:25 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ dpkg -l libasound2-dev libsbc-dev libbluetooth-dev
 :
ii  libasound2-dev:amd64      1.1.3-5           amd64             shared library for ALSA applications -- development file
ii  libbluetooth-dev          5.46-0ubuntu3     amd64             Development files for using the BlueZ Linux Bluetooth li
ii  libsbc-dev:amd64          1.3-2             amd64             Sub Band CODEC library - development

$ lsusb
 :
Bus 001 Device 004: ID 0a12:0001 Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode)
 :

Unfortunately, for now I can't reproduce the situation of my last post, for I have upgraded "Machine 2" from Ubuntu 17.10 beta to 17.10 (release) and rebuilt bluez-alsa. I'm sorry if I bothered you by my post based on the beta release.

( By the way, I have difficulty in making "Machine 1" connect to multiple headsets simultaneously. Perhaps newer bluez will do, so I'll try to upgrade it later. )

[arkq]

Hi @asari-fd,

So, you are saying that playing using the mSBC codec works for you? Because using aplay -t raw -f S16_LE -c1 -r16000 -D bluealsa:HCI=hci0,DEV=XX:XX:XX:XX:XX:XX,PROFILE=sco does not guarantees sampling 16kHz. Default bluealsa PCM uses plug plugin, which will convert 16kHz to 8kHz if necessary. The only reliable way is to see in logs of the bluealsa that the mSBC codec has been selected: +BCS:2 (or add some custom logging :)).