HID over GATT feasibility

[Slion]

Considering using this library to implement HID over GATT. Would it be suitable? I reckon it should be doable since it is just a GATT server with multiple services but I thought I should ask first just in case.

[philips77]

It depends. Do you want to implement the server (emulate mouse/keyboard)? Than it may be possible, 50/50. I never tried, but perhaps there are no restrictions like there are for the client.
Client is already supported natively on KitKat+, and you won't have access to hid characteristics of a remote device from your app. Only the system can use them, so having own client isn't possible.

[Slion]

Yes I was talking about the server. Thus your android device can be used as mouse and keyboard for a PC from instance. I already have a working prototype using just the Android API. I'm hoping using this library will ease the pain of managing bonding and reconnect for instance.

[philips77]

That's so cool! Good luck with the project.
Don't know if the lib will help anything with bonding, as it's managed by the system, but the ble API should be much more readable. There lib supports server-only mode since the latest version using #450.

[Slion]

There lib supports server-only mode since the latest version using #450.

Good to know, though I'm not sure I understand why it would not work as server only before. The BleServerManager doc still mentions that it does not work as server-only. I'm assuming it's just that the doc needs updating.

[philips77]

The BleServerManager doc still mentions that it does not work as server-only. I'm assuming it's just that the doc needs updating.

I must have missed that. I'l fix the doc.

Before 2.6.0 to be able to use the BleManager one had to call connect(BluetoothDevice) method to set the device. It was connecting back to central device and doing service discovery even if the client was not intended to be used. From 2.6.0 it skips this part.

[Slion]

If I'm just a server but want to initiate a connection to a client device which function should I call?
In my earlier prototype I could only connect after bonding. If the connection got lost I had to delete all pairing and redo it.

[philips77]

For a connection as a client just call the connect(..) method, like before.
When implementing the server using BLE library you'll get onDeviceConnectedToServer callback from where you may either call connect(...) for bi-directional connection, or the new attachClientConnection(...) for server-only.
The short code snippet is in the mentioned PR.

[Slion]

Is there a mechanism in place that makes sure sendResponse is called no matter what for each onCharacteristicReadRequest and onDescriptorReadRequest or do I need to explicitly set every characteristic and descriptor using setCharacteristicValue and setDescriptorValue?

The doc says you must call sendResponse and indeed if you don't clients get stuck.

[philips77]

That should be called bo matter what even with no callbacks for the write request.

[Slion]

Looks like I got most of it working but very much like my earlier prototype it fails to properly reconnect to a bound host. When I restart the server after successful bounding somehow the connection is automatically reestablished for a short time, the host reads a few characteristics but then it disconnects and Windows displays "Driver error" as device status. From there I need to delete the paired device and do the bounding anew.
Not sure how to debug that, go back to the specs see if I missed anything. Maybe it should not advertise once bound…

Here are some of the last bits of logs I get before it drops out:

2023-01-22 22:41:14.017 17063-17086 ServerManager           net.slions.hidovergatt               D  [Server callback] Read request for characteristic 00002a4a-0000-1000-8000-00805f9b34fb (requestId=28, offset: 0)
2023-01-22 22:41:14.017 17063-17086 ServerManager           net.slions.hidovergatt               I  [Server] READ request for characteristic 00002a4a-0000-1000-8000-00805f9b34fb received
2023-01-22 22:41:14.018 17063-17086 ServerManager           net.slions.hidovergatt               D  server.sendResponse(GATT_SUCCESS, offset=0, value=0x11010003)
2023-01-22 22:41:14.021 17063-17086 ServerManager           net.slions.hidovergatt               V  [Server] Response sent
2023-01-22 22:41:14.482 17063-17083 System                  net.slions.hidovergatt               W  A resource failed to call close.
2023-01-22 22:41:14.482 17063-17083 System                  net.slions.hidovergatt               W  A resource failed to call close.
2023-01-22 22:41:15.986 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onConnectionUpdated() - Device=0A:61:12:18:7C:88 interval=48 latency=0 timeout=960 status=0
2023-01-22 22:41:17.609 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServerConnectionState() - status=0 serverIf=5 device=0A:61:12:18:7C:88
2023-01-22 22:41:17.610 17063-17086 ServerManager           net.slions.hidovergatt               I  [Server] 0A:61:12:18:7C:88 is disconnected
2023-01-22 22:41:17.610 17063-17086 ServerManager           net.slions.hidovergatt               D  Device disconnected 0A:61:12:18:7C:88

[Slion]

I wonder if the problem could be in the start-up sequence:

2023-01-22 22:41:13.360 17063-17063 BluetoothGattServer     net.slions.hidovergatt               D  registerCallback()
2023-01-22 22:41:13.361 17063-17063 BluetoothGattServer     net.slions.hidovergatt               D  registerCallback() - UUID=84f258b1-0a23-4ee5-aaaa-a988fa894ded
2023-01-22 22:41:13.362 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServerRegistered() - status=0 serverIf=5
2023-01-22 22:41:13.362 17063-17063 ServerManager           net.slions.hidovergatt               I  [Server] Server started successfully
2023-01-22 22:41:13.362 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServerConnectionState() - status=0 serverIf=5 device=0A:61:12:18:7C:88
2023-01-22 22:41:13.363 17063-17063 BluetoothGattServer     net.slions.hidovergatt               D  addService() - service: 00001812-0000-1000-8000-00805f9b34fb
2023-01-22 22:41:13.363 17063-17086 ServerManager           net.slions.hidovergatt               I  [Server] 0A:61:12:18:7C:88 is now connected
2023-01-22 22:41:13.363 17063-17086 ServerManager           net.slions.hidovergatt               D  Device connected 0A:61:12:18:7C:88
2023-01-22 22:41:13.368 17063-17063 BluetoothAdapter        net.slions.hidovergatt               D  isLeEnabled(): ON
2023-01-22 22:41:13.369 17063-17063 BluetoothAdapter        net.slions.hidovergatt               D  isLeEnabled(): ON
2023-01-22 22:41:13.384 17063-17086 ServerManager           net.slions.hidovergatt               D  Before set characteristics
2023-01-22 22:41:13.422 17063-17063 ble-advertiser          net.slions.hidovergatt               I  LE Advertise Started.
2023-01-22 22:41:13.423 17063-17086 ServerManager           net.slions.hidovergatt               D  After set characteristics
2023-01-22 22:41:13.424 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServiceAdded() - handle=40 uuid=00001812-0000-1000-8000-00805f9b34fb status=0
2023-01-22 22:41:13.424 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  addService() - service: 0000180a-0000-1000-8000-00805f9b34fb
2023-01-22 22:41:13.426 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServiceAdded() - handle=53 uuid=0000180a-0000-1000-8000-00805f9b34fb status=0
2023-01-22 22:41:13.426 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  addService() - service: 0000180f-0000-1000-8000-00805f9b34fb
2023-01-22 22:41:13.427 17063-17086 BluetoothGattServer     net.slions.hidovergatt               D  onServiceAdded() - handle=60 uuid=0000180f-0000-1000-8000-00805f9b34fb status=0
2023-01-22 22:41:13.427 17063-17086 ServerManager           net.slions.hidovergatt               I  [Server] All services added successfully
2023-01-22 22:41:13.427 17063-17086 ServerManager           net.slions.hidovergatt               I  Gatt server ready

It looks like the device connects before the server is setup properly. TBH I'm not even sure how the device connects in this case. I'm guessing the host is trying to connect as soon the server boots but that could be a bit too early…

[Slion]

Looks like my issue is similar to that:
https://devzone.nordicsemi.com/f/nordic-q-a/40874/windows-10-driver-error-with-ble-hid-devices

Certainly the report map is not read properly for some reason. In the logs I can see that during pairing the MTU is changed to 517 and this is not happening when the connection fails.

[philips77]

Try delaying the advertising after you get the server ready event.
Otherwise the services discovered by the remote device relays on race condition.

[philips77]

Also, dod you try a different phone?

[Slion]

Try delaying the advertising after you get the server ready event. Otherwise the services discovered by the remote device relays on race condition.

Yes I just did that, should help fix the start-up sequence. Though the issue remain, the HID report map is not read properly it seems.

[Slion]

Also, dod you try a different phone?

Good point I should definitely try another phone.

[Slion]

Now I still have the problem that I don't know how to reconnect a paired client. Things work after pairing but once the server is shutdown and the connection is lost, more often than not I can't establish a connection with a paired host. I'm not sure which API I should use to do that. I guess I need to look into BluetoothGattServer.connect but as far as I can tell that API is not accessible through that library for some reason. I'm planning to fork the library an use it as a sub-module so that I can experiment some more.

In the few cases where the connection could be reestablished on its own the report map read was corrupted so I'm guessing that was somehow just a case of the new connection being mistaken for the old connection somehow and the MTU got mixed up maybe.

My earlier prototype without using the library had the same issue and my attempt at using BluetoothGattServer.connect failed then. Let's see how it goes this time around.

[philips77]

Hi,
As far as I understand, you have the following setup:

1. Client runs on Microsoft Windows
2. Server runs on Android

The Android is to work as a HID device, so you have a HID Service.

You advertise from Android with some HID service and connect to it on Windows. This causes initial pairing and, as I understand, you are able to send HID messages correctly.

Then you restart the phone and try to connect again. Usually, in this case a proper HID device would use direct advertising to automatically trigger connection from the Client. On Android you can't advertise directly, so you're using the only advertising possible, which is the normal, like before pairing.

Android should use resolvable private address in this case, so Windows should be able to resume encryption without problems by matching LTK to the resolved MAC address.

You should get a standard onDeviceConnectedToServer callback, like before, and everything should work fine.

The BluetoothGattServer.connect(..) method is there to let know the Android system that you will use that connection. For server-only mode you're not calling BluetoothDevice.connect(..), so you don't have BluetoothGatt client object. The system needs to know whether your app is using, or not, the new connection. If not, the connection will be terminated in a second. To "use the connection" any app (including system apps) needs to connect to it as client (thus creating BluetoothGatt object) or declare a will to use it as sever only (by calling BluetoothGattServer.connect).

Indeed, BLE library doesn't call BluetoothGattServer.connect method. This may be a bug. The feature was added recently and perhaps needs to be improved. Please, experiment 👍

You may also, instead of using attachClientConnection to use server-only mode, use connect method to "connect back" to your central device (you don't need to use it at all, just return true from isRequiredServiceConnected in that case). This should also make the connection persistent.

[philips77]

When it comes to resuming encryption issue, we would need to see sniffer traces of logs. It's hard to say which side is responsible for the failure and why. Anyway, it's OS job to manage pairing, so the library can't help here and using native API may not help.

Did you try a different phone, like Pixel 6 or 7 to rule out phone problem?

[Slion]

As far as I understand, you have the following setup:

1. Client runs on Microsoft Windows
2. Server runs on Android

The Android is to work as a HID device, so you have a HID Service.

You advertise from Android with some HID service and connect to it on Windows. This causes initial pairing and, as I understand, you are able to send HID messages correctly.

Correct, pairing is triggered from Windows 10/11 settings by searching for new device and selecting the smartphone that's advertising. Once pairing is complete I can control the PC mouse cursor from the Android phone as my prototype's HID device is a mouse which is sending HID reports through that sendNotification API in response to touch events.

Though there is something a bit funny with the pairing process. I need to accept two pairing request on the Android phone. The first one without pin and a second one with pin.

[Slion]

Then you restart the phone and try to connect again.

Not the phone, just the app.

[Slion]

Usually, in this case a proper HID device would use direct advertising to automatically trigger connection from the Client. On Android you can't advertise directly, so you're using the only advertising possible, which is the normal, like before pairing.

Good to know and yes that's what I do. I moved the advertising to start once the server is ready from onServerReady. By the way since I don't advertise from the start of the app, the broken automatic reconnect is not happening anymore. That's why I was guessing this was only happening when I quickly restart the app from Android Studio and advertising starts ASAP.

[Slion]

Android should use resolvable private address in this case, so Windows should be able to resume encryption without problems by matching LTK to the resolved MAC address.
You should get a standard onDeviceConnectedToServer callback, like before, and everything should work fine.

As mentioned above this was only happening when the advertising was started first and I'm guessing when doing a quick app restart. But then the HID report map which is the largest data chunk the client needs to read was not transmitted properly and Windows would go Driver error as mentioned earlier.

[Slion]

You may also, instead of using attachClientConnection to use server-only mode, use connect method to "connect back" to your central device (you don't need to use it at all, just return true from isRequiredServiceConnected in that case). This should also make the connection persistent.

I just tried that and using connect instead of attachClientConnection behaves much the same. First pairing works, then nothing once the connection is gone.

[Slion]

The BluetoothGattServer.connect(..) method is there to let know the Android system that you will use that connection. For server-only mode you're not calling BluetoothDevice.connect(..), so you don't have BluetoothGatt client object. The system needs to know whether your app is using, or not, the new connection. If not, the connection will be terminated in a second. To "use the connection" any app (including system apps) needs to connect to it as client (thus creating BluetoothGatt object) or declare a will to use it as sever only (by calling BluetoothGattServer.connect).

Interesting, when I tried using BluetoothGattServer.connect(..) in an earlier prototype without that library it still would not reconnect. So I wonder what's the way for a server to ask a paired client to connect. I've seen third-party apps doing that.

Indeed, BLE library doesn't call BluetoothGattServer.connect method. This may be a bug. The feature was added recently and perhaps needs to be improved. Please, experiment 👍

Will do, maybe I can just force access to private data members instead of forking and using a sub-module just now.

[Slion]

BTW I had another case of what looks like a reconnect when restarting the app from Android Studio despite the late advertising. Sometimes it still does it, must be a timing thing, though it always end in driver error because of incomplete report map read.

[Slion]

Also maybe I'm missing something from the HID over GATT specs:

5.1.3 Device-Initiated Connection Procedure for Bonded Devices

This procedure is used after the HID Device has bonded with the Host using the
connection procedure in section 5.1.2. The HID Device may initiate the connection
procedure when commanded by the user or autonomously when a notification is
pending.
A HID Device shall enter the GAP Undirected Connectable Mode or Directed
Connectable Mode either when commanded by the user to initiate a connection to a
HID Host or when the HID Device has one or more notifications to send to a previously
connected HID Host.
The HID Device when bonded should use whichever advertising filter policy it has
previously configured when using the connection procedure in section 5.1.2.
The HID Device should use the recommended advertising interval values shown in
Table 5.2. The interval values in the first row are designed to attempt fast connection
during the first 1.28 seconds; however, if a connection is not established within that
time, the interval values in the second row are designed to reduce power consumption
for devices that continue to advertise.
Advertising Duration Parameter Value
1.28 seconds (low latency) – Option 1 Advertising mode Directed
30 seconds (higher latency) - Option 2 Advertising mode Undirected
Advertising Interval 20 ms to 30 ms
Table 5.2: Recommended advertising parameters for device-initiated connection of bonded devices
The advertising interval and time to perform advertising should be configured with
consideration for user expectations of connection establishment time.
The HID Device shall accept any valid values for connection interval and connection
latency set by the HID Host until service discovery, bonding and/or encryption is
BLUETOOTH PROFILE SPECIFICATION Page 27 of 38
HID over GATT Profile Specification
complete. Only after that should the HID Device request to change to its preferred
connection parameters which best suit its use case.
If a connection is not established within a time limit defined by the HID Device, the HID
Device may exit the GAP connectable mode or switch to the Advertising parameters
shown in Table 5.2 if NormallyConnectable is TRUE.
When a connection is established with a notification pending, the HID Device shall send
one or more notifications to the HID Host.
If the Client Characteristic Configuration descriptor has been configured to enable
notifications but the HID Device has no data to transfer, it should wait for an idle
connection timeout (refer to section 5.1.6) to allow the HID Host to terminate the
connection once its actions are complete.
If the Client Characteristic Configuration descriptor has been configured to enable
notifications and the HID Device has data to transfer, after it has completed its transfer,
it should perform the GAP Terminate Connection procedure after waiting for an idle
connection timeout.
Refer to Appendix A for details on NormallyConnectable behavior.

Though I did double check that this NormallyConnectable flag was set in my HID Information characteristic.

[Slion]

I did something like that from onServerReady but that does not help, as experienced with my earlier prototype:

        val device = bluetoothManager.adapter.getRemoteDevice("my:mac:address")
        getGattServer().connect(device,true)

So I'm a bit clueless there. Not sure what else I could try. I did double check the specs and it looks like my advertisement is set as connectable so that should do it really… but instead nothing 😁

[Slion]

Things to try:

• Do a scan to find the Bluetooth device we want and use it to call BluetoothGattServer.connect.
• Try initiate pairing from the server maybe? Not sure which API to use though.
• Go through the HID over GATT specs again and see i we forgot anthing.
• Implement the optional Scan Parameters Service.

[philips77]

If you want automatic reconnection, Android would have to advertise directly, I believe. It's actually up to Windows implementation.
Also, this may be the reason for the pairing problem during reconnection. Perhaps for automatic it would resume encryption, but clicking a device in Windows triggers new pairing and as you already have LTK, the process fails.

The BluetoothGattServer.connect() has nothing to do with reconnection. It's just to flag whether you want to use the active connection in your app after you get a connection event on the server, or not. And I don't think it works as intended...

What you may also try is to remove bonding after each disconnection. There's a removeBond() request in the manager. I hope it still works, as it's using private methods using reflection. Then each time Windows would pair. Or would try to resume pairing, which would fail, and create a new? Worth trying if such scenario is OK.

Anyway, you can't get direct adv from Android so perhaps that's the only option. And I can't say whether it will work.

[Slion]

There must be ways to get it working somehow. I've seen third party app do that. I've ordered a couple of nRF52840-Dongle to see if I can check how regular devices are doing it.

[Slion]

It seems to me like most BLE device including Android ones run a couple of standard services. There is notably this Generic Attribute Service with a Service Changed Characteristic which supports indicate. Is there a way to fire up that indication on Android to tell our clients our services have changed just after we created our service? Could this be used to trigger the bounded clients to reconnect?

[Slion]

There must be ways to get it working somehow. I've seen third party app do that.

Actually the app I thought got this working is actually using classic Bluetooth rather than BLE. In fact it seems they are using something like that:
https://github.com/raghavk92/Kontroller

[Slion]

When trying this implementation which is not working out of the box, notably it does not wait for the callback while adding multiple GATT services, the reconnect actually worked but not the HID device. Reconnect works I'm guessing as long as the HID device is not added properly. So I was in a state where reconnects would work and the battery status was displayed so somehow the battery status service worked but the HID device would not work and was not detected by Windows. Fixing it by making sure I wait a bit after adding a service makes the HID device work after bounding but reconnect is broken.

That makes me think that there could really be a configuration issue in my HID service maybe that NormallyConnectable flag not at the right place somehow. Looking forward to sniff traffic from other devices to try and spot what I could be doing wrong.

Those nRF52840-Dongles should be there next week.

[philips77]

Could this be used to trigger the bounded clients to reconnect?

No, that indication can only indicate during a connection that a new service discovery is required, but the device needs to be already connected and encryption should be resumed by then.

[Slion]

I wonder if the key to get the reconnect to work with Windows is to use startAdvertisingSet instead of startAdvertising. Now I just need to understand how to configure it properly.

[philips77]

As far as I know, the legacy startAdvertising is using the new API under the hood.