Paired property is not reset after Authentication Failure (when only remote clears pairing data)

[bojanpotocnik]

Consider the following use case (using kernel 5.4.0-131-generic and bluez 5.66):

1. Central and peripheral BLE devices pair using passkey pairing
2. Paired property of the device is set to True
3. Peripheral device erases pairing data
4. Central tries to connect but always fails with Remote User Terminated Connection, while Paired property stays True. It does not try to pair because Paired is already True, but if it does call Pair, the process is stuck.

If devices are not paired (1) and wrong passkey is provided or pairing is cancelled, then org.bluez.Device1.Pair will fail with org.bluez.Error.AuthenticationFailed or org.bluez.Error.AuthenticationCanceled.
But if the connection fails with Connection terminated due to authentication failure, pairing status should also be cleared or the disconnect reason at least propagated higher, instead of device just disconnecting because of Remote User Terminated Connection.

I am not attaching the logs of successful initial connection (1, 2), because that is normal/usual behaviour. Here is the btmon --mgmt output of 4:

< HCI Command: LE Extended Create Connection (0x08|0x0043) plen 26                                 #266 [hci0] 11:04:29.417243
        Filter policy: Accept list is not used (0x00)
        Own address type: Public (0x00)
        Peer address type: Public (0x00)
        Peer address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Initiating PHYs: 0x01
        Entry 0: LE 1M
          Scan interval: 60.000 msec (0x0060)
          Scan window: 60.000 msec (0x0060)
          Min connection interval: 30.00 msec (0x0018)
          Max connection interval: 50.00 msec (0x0028)
          Connection latency: 0 (0x0000)
          Supervision timeout: 420 msec (0x002a)
          Min connection length: 0.000 msec (0x0000)
          Max connection length: 0.000 msec (0x0000)
> HCI Event: Command Status (0x0f) plen 4                                                          #267 [hci0] 11:04:29.419341
      LE Extended Create Connection (0x08|0x0043) ncmd 2
        Status: Success (0x00)
> HCI Event: LE Meta Event (0x3e) plen 31                                                          #268 [hci0] 11:04:29.667157
      LE Enhanced Connection Complete (0x0a)
        Status: Success (0x00)
        Handle: 3585
        Role: Central (0x00)
        Peer address type: Public (0x00)
        Peer address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Local resolvable private address: 00:00:00:00:00:00 (Non-Resolvable)
        Peer resolvable private address: 00:00:00:00:00:00 (Non-Resolvable)
        Connection interval: 45.00 msec (0x0024)
        Connection latency: 0 (0x0000)
        Supervision timeout: 420 msec (0x002a)
        Central clock accuracy: 0x00
@ MGMT Event: Device Connected (0x000b) plen 44                                                {0x0002} [hci0] 11:04:29.667383
        LE Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Flags: 0x00000000
        Data length: 31
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        16-bit Service UUIDs (complete): 1 entry
          Plume Design Inc (0xfe71)
        Company: not assigned (2583)
          Data: 054843383439303032334300000015006c8110de
@ MGMT Event: Device Connected (0x000b) plen 44                                                {0x0001} [hci0] 11:04:29.667383
        LE Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Flags: 0x00000000
        Data length: 31
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        16-bit Service UUIDs (complete): 1 entry
          Plume Design Inc (0xfe71)
        Company: not assigned (2583)
          Data: 054843383439303032334300000015006c8110de
< HCI Command: LE Read Remote Used Features (0x08|0x0016) plen 2                                   #269 [hci0] 11:04:29.667465
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
@ RAW Open: btmon (privileged) version 2.22                                                           {0x0003} 11:04:29.667662
@ RAW Close: btmon                                                                                    {0x0003} 11:04:29.667666
> HCI Event: LE Meta Event (0x3e) plen 4                                                           #270 [hci0] 11:04:29.668122
      LE Channel Selection Algorithm (0x14)
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Algorithm: #2 (0x01)
> HCI Event: Command Status (0x0f) plen 4                                                          #271 [hci0] 11:04:29.669122
      LE Read Remote Used Features (0x08|0x0016) ncmd 1
        Status: Success (0x00)
> ACL Data RX: Handle 3585 flags 0x02 dlen 7                                                       #272 [hci0] 11:04:29.709077
      ATT: Exchange MTU Request (0x02) len 2
        Client RX MTU: 247
> HCI Event: LE Meta Event (0x3e) plen 12                                                          #273 [hci0] 11:04:29.710340
      LE Read Remote Used Features (0x04)
        Status: Success (0x00)
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Features: 0x2d 0xf9 0x00 0x00 0x00 0x00 0x00 0x00
          LE Encryption
          Extended Reject Indication
          Peripheral-initiated Features Exchange
          LE Data Packet Length Extension
          LE 2M PHY
          LE Coded PHY
          LE Extended Advertising
          LE Periodic Advertising
          Channel Selection Algorithm #2
          LE Power Class 1
< HCI Command: LE Start Encryption (0x08|0x0019) plen 28                                           #274 [hci0] 11:04:29.710387
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Random number: 0x0000000000000000
        Encrypted diversifier: 0x0000
        Long term key: 6667d524cd1ba29616c3452d9b76fd1e
> HCI Event: Command Status (0x0f) plen 4                                                          #275 [hci0] 11:04:29.711177
      LE Start Encryption (0x08|0x0019) ncmd 1
        Status: Success (0x00)
> ACL Data RX: Handle 3585 flags 0x02 dlen 6                                                       #276 [hci0] 11:04:29.798664
      SMP: Security Request (0x0b) len 1
        Authentication requirement: Bonding, MITM, SC, No Keypresses (0x0d)
> HCI Event: LE Meta Event (0x3e) plen 11                                                          #277 [hci0] 11:04:29.800198
      LE Data Length Change (0x07)
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Max TX octets: 251
        Max TX time: 17040
        Max RX octets: 251
        Max RX time: 17040
> HCI Event: Encryption Change (0x08) plen 4                                                       #278 [hci0] 11:04:30.024350
        Status: PIN or Key Missing (0x06)
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Encryption: Disabled (0x00)
< HCI Command: Disconnect (0x01|0x0006) plen 3                                                     #279 [hci0] 11:04:30.024411
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Reason: Authentication Failure (0x05)
> HCI Event: Command Status (0x0f) plen 4                                                          #280 [hci0] 11:04:30.025271
      Disconnect (0x01|0x0006) ncmd 1
        Status: Command Disallowed (0x0c)
> HCI Event: Disconnect Complete (0x05) plen 4                                                     #281 [hci0] 11:04:30.069206
        Status: Success (0x00)
        Handle: 3585 Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Reason: Remote User Terminated Connection (0x13)
@ MGMT Event: Device Disconnected (0x000c) plen 8                                              {0x0002} [hci0] 11:04:30.069267
        LE Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Reason: Connection terminated due to authentication failure (0x04)
@ MGMT Event: Device Disconnected (0x000c) plen 8                                              {0x0001} [hci0] 11:04:30.069267
        LE Address: 7C:9F:07:00:15:29 (OUI 7C-9F-07)
        Reason: Connection terminated due to authentication failure (0x04)

together with journalctl -o short-precise -fu bluetooth (bluetooth.service is started with --experimental -d)

nov 29 11:04:28.710917 bojan bluetoothd[26768]: src/device.c:device_connect_le() Connection attempt to: 7C:9F:07:00:15:29
nov 29 11:04:29.667594 bojan bluetoothd[26768]: src/shared/mgmt.c:can_read_data() [0x0000] event 0x000b
nov 29 11:04:29.667618 bojan bluetoothd[26768]: src/adapter.c:connected_callback() hci0 device 7C:9F:07:00:15:29 connected eir_len 31
nov 29 11:04:29.710498 bojan bluetoothd[26768]: src/device.c:device_attach_att() Elevating security level since LTK is available
nov 29 11:04:29.710559 bojan bluetoothd[26768]: attrib/gattrib.c:g_attrib_ref() 0x55b7de5dee40: g_attrib_ref=1
nov 29 11:04:29.710576 bojan bluetoothd[26768]: src/device.c:device_accept_gatt_profiles() initiator true
nov 29 11:04:29.710582 bojan bluetoothd[26768]: src/gatt-client.c:btd_gatt_client_connected() Device connected.
nov 29 11:04:29.710653 bojan bluetoothd[26768]: src/shared/att.c:can_read_data() (chan 0x55b7de5e3340) ATT PDU received: 0x02
nov 29 11:04:29.710660 bojan bluetoothd[26768]: src/shared/gatt-server.c:exchange_mtu_cb() MTU exchange complete, with MTU: 247
nov 29 11:04:30.069424 bojan bluetoothd[26768]: src/shared/mgmt.c:can_read_data() [0x0000] event 0x000c
nov 29 11:04:30.069447 bojan bluetoothd[26768]: src/adapter.c:dev_disconnected() Device 7C:9F:07:00:15:29 disconnected, reason 4
nov 29 11:04:30.069454 bojan bluetoothd[26768]: src/adapter.c:adapter_remove_connection()
nov 29 11:04:30.069482 bojan bluetoothd[26768]: plugins/policy.c:disconnect_cb() reason 4
nov 29 11:04:30.069490 bojan bluetoothd[26768]: src/adapter.c:bonding_attempt_complete() hci0 bdaddr 7C:9F:07:00:15:29 type 1 status 0xe
nov 29 11:04:30.069510 bojan bluetoothd[26768]: src/device.c:device_bonding_complete() bonding (nil) status 0x0e
nov 29 11:04:30.069519 bojan bluetoothd[26768]: src/device.c:device_bonding_failed() status 14
nov 29 11:04:30.069533 bojan bluetoothd[26768]: src/adapter.c:resume_discovery()
nov 29 11:04:30.069593 bojan bluetoothd[26768]: src/shared/att.c:disconnect_cb() Channel 0x55b7de5e3340 disconnected: Connection reset by peer
nov 29 11:04:30.069604 bojan bluetoothd[26768]: src/shared/gatt-client.c:exchange_mtu_cb() MTU Exchange failed. ATT ECODE: 0x00
nov 29 11:04:30.069612 bojan bluetoothd[26768]: src/device.c:gatt_client_ready_cb() status: failed, error: 0
nov 29 11:04:30.069619 bojan bluetoothd[26768]: src/device.c:device_svc_resolved() /org/bluez/hci0/dev_7C_9F_07_00_15_29 err -5
nov 29 11:04:30.069637 bojan bluetoothd[26768]: src/device.c:att_disconnected_cb()
nov 29 11:04:30.069644 bojan bluetoothd[26768]: src/device.c:att_disconnected_cb() Connection reset by peer (104)
nov 29 11:04:30.069651 bojan bluetoothd[26768]: src/gatt-client.c:btd_gatt_client_disconnected() Device disconnected. Cleaning up.
nov 29 11:04:30.069664 bojan bluetoothd[26768]: src/device.c:att_disconnected_cb() Automatic connection disabled
nov 29 11:04:30.069677 bojan bluetoothd[26768]: src/gatt-database.c:btd_gatt_database_att_disconnected()
nov 29 11:04:30.069692 bojan bluetoothd[26768]: attrib/gattrib.c:g_attrib_unref() 0x55b7de5dee40: g_attrib_unref=0

The problem is solved by applying the following patch to

bluez/src/device.c

Lines 6040 to 6052 in 9f50368

	if (status) {
	device_cancel_authentication(device, TRUE);
	/* Put the device back to the temporary state so that it will be
	* treated as a newly discovered device.
	*/
	if (!device_is_paired(device, bdaddr_type) &&
	!btd_device_is_trusted(device))
	btd_device_set_temporary(device, true);
	device_bonding_failed(device, status);
	return;
	}

diff --git a/src/device.c b/src/device.c
index 995d39f2c..a41370ade 100644
--- a/src/device.c
+++ b/src/device.c
@@ -6048,6 +6048,9 @@ void device_bonding_complete(struct btd_device *device, uint8_t bdaddr_type,
                        btd_device_set_temporary(device, true);
 
                device_bonding_failed(device, status);
+
+               device_remove_stored(device); /* or device_remove(device, TRUE); */
+
                return;
        }

but I'm not sure if this is the correct way/place to do it. I found a patch device: Fix pairing has failed due to the error of Already Paired (0x13) which solved a similar problem of removing bonding data, but for a different case.

[Vudentz]

@bojanpotocnik afaik there isn't a clear text on the spec regarding how to treat half-paired cases since there is a chance a rogue device could be impersonating another dropping the pairing shall be a user initiated action, we could however ask the pairing agent what to do which can then popup a dialog asking if the user wants to repair, etc, but I wouldn't drop keys automatically like that.

[Phlip]

how do completely headless devices like Bluetooth speakers handle the half-paired situation?

my speaker can unpair and then SSP pair again, while my Raspberry Pi cannot

[Vudentz]

how do completely headless devices like Bluetooth speakers handle the half-paired situation?

my speaker can unpair and then SSP pair again, while my Raspberry Pi cannot

I guess that is up to the pairing agent to decide, if it really don't have a user to ask about then it can just remove the device automatically, anyway with headless devices there is probably no way to do man in the middle protection so the pairing is never secure so I assume that us fine, anyway the agent in this case is already a bit special since it is probably needs to be aware it is running on a headless session.

[bojanpotocnik]

Quoting #424 (comment)

Mission accomplished. Correctly compiling and running the patch in #433 fixed the bug, and I can re-pair. Thanks!

So maybe some related change is to be considered.

Reading above, the issue is that someone could spoof the MAC address of paired device and then reject the pairing, effectively maliciously erasing the pairing data on the central.

My initial idea was to somehow provide the information that device disconnected because Connection terminated due to authentication failure, not just Remote User Terminated Connection. For all existing dialogs the behaviour would stay the same, but one could detect this and decide whether to unpair using org.bluez.Adapter1.RemoveDevice or do nothing.
As I don't know how to do this, my patch above worked for my use case: there are many reasons for Remote User Terminated Connection - when I should not erase pairing, but only one for Connection terminated due to authentication failure (in such case I simply need to re-pair, because pairing data on the remote "timed out"). There must be some way to distinguish these two 🤔

[Phlip]

to help understand why we don't care if anyone spoofs a MAC address:

We ship a robot with a controller running the BlueZ stack and a permanent, dedicated pair to an Android tablet. We are not creating a civilian app that pairs promiscuously. Our main requirement is that the pair never drops in the field, and if it does, we don't need to get inside the robot to erase its half of the pair. The tablet should just request a pair, and the robot accept it, no questions asked.