subscribeToCharacteristic fails when Client Characteristic Config descriptor is not present

[LironKomfort]

I'm working on the example project of the library (https://github.com/PhilipsHue/flutter_reactive_ble/tree/master/example),
I added a few buttons to test the API - readCharacteristic, writeCharacteristicWithResponse and subscribeToCharacteristic.

Subscribing to notifications doesn't work as expected (read/write works well).

Flow:
scanForDevices
connectToDevice/connectToAdvertisingDevice (tried both)
subscribeToCharacteristic (after status is connected)

Flow's code (serviceId ,readCharacteristicId & writeCharacteristicId are defined in the BLE server):

 final Uuid serviceId = Uuid.parse('06391ebb-4050-42b6-ab55-8282a15fa094');
 final Uuid readCharacteristicId = Uuid.parse('010d815c-031c-4de8-ac10-1ffebcf874fa'); 
 final Uuid writeCharacteristicId = Uuid.parse('f2926f0f-336a-4502-9948-e4e8fd2316e9');   

  Future<void> connect(String deviceId) async {
    if (_connection != null) {
      await _connection.cancel();
    }

    _connection = _ble
        .connectToDevice(
          id: deviceId,
          servicesWithCharacteristicsToDiscover: {
            serviceId: [readCharacteristicId, writeCharacteristicId]
          },
          connectionTimeout: const Duration(seconds: 10),
        )
        .listen(
          _deviceConnectionController.add,
        );
  }

void subscribeCharacteristic(String deviceId) {
    final characteristic = QualifiedCharacteristic(
        serviceId: serviceId,
        characteristicId: readCharacteristicId,
        deviceId: deviceId);

    _ble.subscribeToCharacteristic(characteristic).listen((data) {
      // code to handle incoming data
      print('Data received $data');
    }, onError: (dynamic error) {
      // code to handle errors
      print('Subscribe error $error');
    });
  }

  Future<void> readCharacteristic(String deviceId) async {
    final characteristic = QualifiedCharacteristic(
        serviceId: serviceId,
        characteristicId: readCharacteristicId,
        deviceId: deviceId);
    await _ble.readCharacteristic(characteristic).then(
          printCharacteristic,
        );
  }

  Future<void> writeCharacteristic(String deviceId) async {
    final characteristic = QualifiedCharacteristic(
        serviceId: serviceId,
        characteristicId: writeCharacteristicId,
        deviceId: deviceId);
    await _ble.writeCharacteristicWithResponse(characteristic,
        value: [0xAD, 0xDE]).then(
      (value) => print('writeCharacteristic CB'),
    );
  }

Debug output:
I/flutter ( 4075): REACTIVE_BLE: Start subscribing to notifications for QualifiedCharacteristic(characteristicId: 010d815c-031c-4de8-ac10-1ffebcf874fa, serviceId: 06391ebb-4050-42b6-ab55-8282a15fa094, deviceId: 5F:99:E1:54:4A:89)
D/BluetoothGatt( 4075): discoverServices() - device: 5F:99:E1:54:4A:89
D/BluetoothGatt( 4075): onSearchComplete() = Device=5F:99:E1:54:4A:89 Status=0
D/BluetoothGatt( 4075): setCharacteristicNotification() - uuid: 010d815c-031c-4de8-ac10-1ffebcf874fa enable: false
I/flutter ( 4075): REACTIVE_BLE: Received CharacteristicValue(characteristic: QualifiedCharacteristic(characteristicId: 010d815c-031c-4de8-ac10-1ffebcf874fa, serviceId: 06391ebb-4050-42b6-ab55-8282a15fa094, deviceId: 5F:99:E1:54:4A:89), result: CharacteristicValue)
I/flutter ( 4075): Subscribe error Exception: GenericFailure(code: CharacteristicValueUpdateError.unknown, message: "Cannot find client characteristic config descriptor (code 2) with characteristic UUID 010d815c-031c-4de8-ac10-1ffebcf874fa")

Please explain:
Why is the 'enabled' boolean (setCharacteristicNotification) false? how can I set it to true?
What is the reason for the "Subscribe error Exception"? Am I supposed to configure a descriptor? if so, how?

Additional info

My Android based BLE server, creation code:

    UUID SENSOR_SERVICE = UUID.fromString("06391ebb-4050-42b6-ab55-8282a15fa094"); // Sensor Service 
    UUID DATA_R = UUID.fromString("010d815c-031c-4de8-ac10-1ffebcf874fa");                        // Readable Characteristic 
    UUID CLIENT_CONFIG = UUID.fromString("7385e060-b9a8-4853-848d-c70178b0e01e");  // Config Descriptor
    UUID DATA_W = UUID.fromString("f2926f0f-336a-4502-9948-e4e8fd2316e9");                  // Writable Characteristic
    
    BluetoothGattService service = new BluetoothGattService(SensorProfile.SENSOR_SERVICE,
            BluetoothGattService.SERVICE_TYPE_PRIMARY);
    
    // Readable Data characteristic
    BluetoothGattCharacteristic dataR = new BluetoothGattCharacteristic(DATA_R,
            //Read-only characteristic, supports notifications
             BluetoothGattCharacteristic.PROPERTY_NOTIFY |
                        BluetoothGattCharacteristic.PROPERTY_READ |
                        BluetoothGattCharacteristic.PROPERTY_INDICATE |
                        BluetoothGattCharacteristic.PROPERTY_BROADCAST,
            BluetoothGattCharacteristic.PERMISSION_READ);
    
      //Read/write descriptor
      BluetoothGattDescriptor configDescriptor = new BluetoothGattDescriptor(CLIENT_CONFIG,
          BluetoothGattDescriptor.PERMISSION_READ);
    
      dataR.addDescriptor(configDescriptor);
      service.addCharacteristic(dataR);
    
      // Writeable Data characteristic
      BluetoothGattCharacteristic dataW = new BluetoothGattCharacteristic(DATA_W,
              BluetoothGattCharacteristic.PROPERTY_WRITE, BluetoothGattCharacteristic.PERMISSION_WRITE);
      service.addCharacteristic(dataW);

My Android based BLE client (notifications works as expected), code:

 BluetoothGattService service = mBluetoothGatt.getService(SENSOR_SERVICE);
      // Get the counter characteristic
      BluetoothGattCharacteristic characteristic = service.getCharacteristic(DATA_R);
    
      // Enable notifications for this characteristic locally
      mBluetoothGatt.setCharacteristicNotification(characteristic, true);
    
      // Write on the config descriptor to be notified when the value changes
      BluetoothGattDescriptor descriptor = characteristic.getDescriptor(CLIENT_CONFIG);
      descriptor.setValue(BluetoothGattDescriptor.ENABLE_NOTIFICATION_VALUE);
      mBluetoothGatt.writeDescriptor(descriptor);

Flutter ble client:
https://github.com/LironKomfort/reactive_ble_test

BLE android client/server projects:
https://github.com/LironKomfort/android_ble_server_client

Client tested on Android version 9 (Xiaomi mi6 + Pixel 4A)
Server tested on Android version 7.1

[isAlmogK]

Following

[remonh87]

@LironKomfort thank you for the clear description of the issue this really helped triaging the issue. We are using for android the reactive ble library from Polidea and their implementation. The boolean you mentioned setCharacteristicsNotification is set to true in the library we use so that is not the issue. The main issue indeed that your GATT descriptor does not have a client characteristic config descriptor: see for example issue. To be honest I have no idea how to do this on the server side.

The library we are using however does support a compatibility mode where it does something similar what you do in the native implementation, link. We can consider supporting the compatibility mode that is used in Polidea however I see this as a workaround and the issue is with the peripheral itself.

I am a bit cautious with implementing android specific workarounds in our external Flutter API because we support both platforms and a compatibility mode is not something that is familiar to iOS devs. @werediver what are your thoughts?

[isAlmogK]

@remonh87 @werediver Hi guys I'm working @LironKomfort on this project, just wanted to give my 2cents I think a workaround would be good because I see a number of peripheral devices not having compatibility mode supported, and updating the device might not be so easy as it is hardware, therefore, making the package limited in use case. Having the workaround provides better options and more use cases. If that makes sense

[remonh87]

I think adding following is less intrusive and quite simple to realise

 Stream<List<int>> subscribeToCharacteristic(
    QualifiedCharacteristic characteristic,
    {bool androidCompatibilityMode}
  ) 

Also it takes away my concerns about confusing iOS devs

[isAlmogK]

@remonh87 that looks really good.

[werediver]

@remonh87 If this "compatibility mode" is practically useful and is easy to implement, we can make some people happier by implementing it.

If that compatibility mode should be controlled on per-call basis (so that one subscription can be established using the compatibility mode and another the normal way), extra optional parameter to subscribeToCharacteristic(⋯) method looks fine to me.

If the compatibility mode is expected to be used for all subscriptions, when enabled, I'd suggest to introduce some kind of setFlags(⋯) or configure(⋯) method to control such options.

But an even better solution would be to automatically detect whether the compatibility mode is required. Is this possible?

[werediver]

Actually, after skimming through dariuszseweryn/RxAndroidBle#202 it seems the compatibility mode is controlled on per-connection basis. This may be tricky to implement in our library.

[remonh87]

@werediver see my answers:

If that compatibility mode should be controlled on per-call basis (so that one subscription can be established using the compatibility mode and another the normal way), extra optional parameter to subscribeToCharacteristic(⋯) method looks fine to me.

It is per call basis according to the docs: * @param setupMode Configures how the notification is set up. For available modes see {@link NotificationSetupMode}.

But an even better solution would be to automatically detect whether the compatibility mode is required. Is this possible?

I found that Android exposes the descriptor of a characteristic. I can check if there is no notify descriptor and then go for compatibility mode. This would be a super simple and less intrusive fix but on Android ble is always very unpredictable so not sure about the side effects.

Actually, after skimming through dariuszseweryn/RxAndroidBle#202 it seems the compatibility mode is controlled on per-connection basis. This may be tricky to implement in our library.

It may look like at first but it is established per notification itself so shouldn't be all too tricky.

[werediver]

I found that Android exposes the descriptor of a characteristic. I can check if there is no notify descriptor and then go for compatibility mode. This would be a super simple and less intrusive fix but on Android ble is always very unpredictable so not sure about the side effects.

@remonh87 I'd strongly consider the automatic check approach first. We can release the automatic check and if it doesn't satisfy users, we can reconsider.

[LironKomfort]

@remonh87 @werediver
After doing some more research & tests, I found that if I use the reserved Client Characteristic Configuration UUID (00002902-0000-1000-8000-00805f9b34fb) as the descriptor of my characteristic (on the server side), subscribing to notifications works as expected.
If I use a custom descriptor on my server (with the same properties of Notify and Read), subscribing to notifications fails.

Custom descriptors are not supported?

[werediver]

Custom descriptors are not supported?

This is a question about iOS and Android BLE stacks, not about this library [as I understand the situation]. A peripheral should comply to the standards to work without such workarounds.

[remonh87]

@LironKomfort I did create a test branch with the automatic detection. Can you check if this works for you: test-automatic-descriptor-detection-subscribe-char-android . Please also check with both custom and also reserved

[werediver]

@remonh87 Do you think the automatic detection has a noticeable performance impact (requires time [like 50+ ms] to do the check)?

[remonh87]

@werediver we already have discovered the services so the only thing I do is checking the descriptors for this char is empty. I wouldn't expect it to be very costly.

[LironKomfort]

@LironKomfort I did create a test branch with the automatic detection. Can you check if this works for you: test-automatic-descriptor-detection-subscribe-char-android . Please also check with both custom and also reserved

@remonh87
I ran the following tests:
A custom service with custom characteristic where:

1. Server doesn't configure a descriptor at all.
   result: NotificationSetupMode is set to COMPAT & set notifications works.
2. Server configured the reserved Client Characteristic Configuration descriptor UUID (00002902-0000-1000-8000-00805f9b34fb).
   result: NotificationSetupMode is set to DEFAULT & set notifications works.
3. Server configured a custom descriptor UUID.
   result: Descriptor isn't empty so NotificationSetupMode is set to DEFAULT, set notifications fails.

I edited the if statement you added in the Kotlin code to check for empty OR different from the reserved CCC descriptor.
result: NotificationSetupMode is set to DEFAULT & set notifications works.

[LironKomfort]

Custom descriptors are not supported?

This is a question about iOS and Android BLE stacks, not about this library [as I understand the situation]. A peripheral should comply to the standards to work without such workarounds.

@werediver
I know Android supports custom descriptors since I implemented a BLE client & server with custom service, characteristics and descriptors.
I don't know about iOS though...

[remonh87]

@LironKomfort @AlmogRnD is it possible to not set a custom descriptor on your BLE device? If yes then I propose to create a separate issue regarding the custom descriptors.

[LironKomfort]

@LironKomfort @AlmogRnD is it possible to not set a custom descriptor on your BLE device? If yes then I propose to create a separate issue regarding the custom descriptors.

@remonh87
Yes, it is possible, I'll create a separate issue.
Thanks