Don't work on micropython v1.20.0

[GhostMayor]

Hi

I have some problem if use your example for keyboard on v1.20.0 micropython

• on my PC (windows) driver has errored
• on my android device no any events for keyboard (but was notified)

but i downgread micropython to 1.19.1 version and solved problem with driver on my PC

[sunpub]

I have the same problem
it will be got a response "driver error" when connected windows 11
and the phone can't receive any value when connected.

[Heerkog]

Hi,

Thank you for the report. Sadly, I'm currently not able to test with v1.20.0 Micropython.

If you find the issue, I would really appreciate it if you could make a pull request.

Thanks.

[pidou46]

Hi,
I have started to investigate:

Here is an extract of the error message:

Server started
Started advertising
Central connected:  0
MTU exchanged:  23
encryption update 0 1 0 1 16
Connection update
Unhandled IRQ event:  4```

_IRQ_GATTS_READ_REQUEST = const(4) is declared in hid_services.py but it is not handled in ble_irq callback function that probably is why the error is raised.

I will try to investigate further if I can

[Heerkog]

Thank you so much for investigating!

[pidou46]

I have tried to add a elif clause for IRQ_GATTS_READ_REQUEST event, and make it "gatts-write" zero back to the client to accept the read, but I dont know which "value handle" I should write to ?

#			https://docs.micropython.org/en/latest/_sources/library/bluetooth.rst.txt
#			A client has issued a read. Note: this is only supported on STM32.
#			Return a non-zero integer to deny the read (see below), or zero (or None) to accept the read.
#
#             For the ``_IRQ_GATTS_READ_REQUEST`` event, the available return codes are::
#                 _GATTS_NO_ERROR = const(0x00)
#                 _GATTS_ERROR_READ_NOT_PERMITTED = const(0x02)
#                 _GATTS_ERROR_WRITE_NOT_PERMITTED = const(0x03)
#                 _GATTS_ERROR_INSUFFICIENT_AUTHENTICATION = const(0x05)
#                 _GATTS_ERROR_INSUFFICIENT_AUTHORIZATION = const(0x08)
#                 _GATTS_ERROR_INSUFFICIENT_ENCRYPTION = const(0x0f)
#             https://forum.micropython.org/viewtopic.php?t=11444
#			I assumed reads weren't implemented on the ESP32, but it's just the interrupt that's not called on a read.
#			So you *must* set a value using gatts_write and you can't do anything fancy when a central performs the read.
            conn_handle, attr_handle = data
            self._ble.gatts_write(conn_handle, 0x0) #wich handle should I respond to ?
            #attr_handle : TypeError: object with buffer protocol required
            #conn_handle : TypeError: object with buffer protocol required
            print("gatts read request: ", conn_handle, attr_handle)```            

[Heerkog]

I'm unsure, but I would try self._ble.gatts_notify(conn_handle, attr_handle, 0x00).

[pidou46]

I give it a try, but I get the following error:

Traceback (most recent call last):
  File "hid_services.py", line 328, in ble_irq
TypeError: object with buffer protocol required```

If I understand correctly it should be "value handle" and not "characteristic handle", but I stuck to find out which "value handle" it should be ?

[pidou46]

I have changed to:
self._ble.gatts_write(attr_handle, bytearray(0))
And I get no more error, but does it works ? I have to make tests.

[Heerkog]

I'm pretty sure that the value handle is the attr_handle returned in the conn_handle, attr_handle = data.

As for the error, it should probably be self._ble.gatts_notify(conn_handle, attr_handle, struct.pack("<B", 0))

[Heerkog]

I have changed to: self._ble.gatts_write(attr_handle, bytearray(0)) And I get no more error, but does it works ? I have to make tests.

@pidou46 does it work now?

[pidou46]

As for connection it works.
But after a few seconds, the disconnect event is raised and the status in windows bleutooth app is switched to: binding
I can't find out witch of those two event occur first and I'm a little bit stuck because I don't how to debug to get rid of this situation.
Here is the log:

MPY: soft reboot
((21, 23, 25, 27, 29, 31),)
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
#windows bleutooth app: add bleutooth device
connect:  1 (0, 0, <memoryview>)
other events: 21 (0, 256) #_IRQ_MTU_EXCHANGED
loop: True
isconnected, moving mouse...
other events: 28 (0, 1, 0, 1, 16) #_IRQ_ENCRYPTION_UPDATE
loop: True
isconnected, moving mouse...
other events: 27 (0, 16, 0, 960, 0) #_IRQ_CONNECTION_UPDATE
other events: 4 (0, 31) #_IRQ_GATTS_READ_REQUEST
other events: 4 (0, 27) #_IRQ_GATTS_READ_REQUEST
other events: 4 (0, 23) #_IRQ_GATTS_READ_REQUEST
other events: 4 (0, 21) #_IRQ_GATTS_READ_REQUEST
other events: 27 (0, 48, 0, 960, 0) #_IRQ_CONNECTION_UPDATE
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
other events: 27 (0, 12, 0, 200, 0) #_IRQ_CONNECTION_UPDATE
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
loop: True
isconnected, moving mouse...
#Windows bleutooth app : switch to binding
Disconnect 2 (0, 0, <memoryview>) #_IRQ_CENTRAL_DISCONNECT
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
loop: False
Traceback (most recent call last):
  File "<stdin>", line 161, in <module>
KeyboardInterrupt:

[pidou46]

During the period of connection, I'm unsure it really works because I can't really see mouse any movement despite reports are send.

[Heerkog]

The binding code currently included is very experimental. Could you first try without allowing binding?

[pidou46]

I have set bounding to False in hid_services.py

# Allow bonding
            if self.bond:  # calling this on ESP32 is unsupported
                self._ble.config(bond=False)

            if self.le_secure:  # calling these on ESP32 is unsupported
                # Require secure pairing
                self._ble.config(le_secure=False)
                # Require man in the middle protection
                self._ble.config(mitm=False)
                # Set our input/output capabilities
                self._ble.config(io=self.io_capability)

But the result pretty much the same:

MPY: soft reboot
Server created
get secret:  10 0 b'irk'
BLE on
Registering services
Writing service characteristics
Writing device information service characteristics
Writing battery service characteristics
Writing hid service characteristics
Advertiser created:  Mouse  with services:  [UUID(0x1812)]
Server started
Started advertising
Central connected:  0
MTU exchanged:  23
encryption update 0 1 0 1 16
Connection update
gatts read request:  0 50
gatts read request:  0 46
gatts read request:  0 42
gatts read request:  0 40
gatts read request:  0 36
gatts read request:  0 33
gatts read request:  0 31
gatts read request:  0 29
gatts read request:  0 27
gatts read request:  0 25
gatts read request:  0 23
gatts read request:  0 21
gatts read request:  0 36
Connection update
Connection update
Central disconnected:  0

The windows bluetooth app still report "coupling" for a few seconds then "driver error"

[pidou46]

Nota: I get the battery 0% notification before error status

[Heerkog]

Do you remove the device from windows before powercycling and attempting a test?

[pidou46]

Yes I do.

Something seems to going wrong during "Connection update" that lead to "Central disconnected: 0"

Do you have some trick to debug ble/hid/usb stack ?

I have checked with windows "event observer" (I'm note sure is the name in English because i'm Fench) :

I see an event from BTHUSB : La carte distante (54:43:b2:a9:0d:a6) s'est connectée correctement à la carte locale.

translation: the distant device connected correctly to the local card.

But no error event later on.

[Heerkog]

Something seems to going wrong during "Connection update" that lead to "Central disconnected: 0"

Let's find out what was updated in the connection update:

        elif event == _IRQ_CONNECTION_UPDATE:          # Connection parameters were updated
            self.conn_handle, conn_interval, conn_latency, supervision_timeout, status = data        # The new parameters
            print("Connection update: ", conn_interval, conn_latency, supervision_timeout, status)

Do you have some trick to debug ble/hid/usb stack ?

I do not, sorry.

[Heerkog]

If the updates are to supervision_timeout followed by status (idle?) then I think we need to implement a status update loop that updates our values within the supervision_timeout value. Currently, we only update when there are changes.

It might be that earlier MicroPython releases kept the connection alive and this version doesn't anymore. This is speculation of course.

[Heerkog]

Any updates on this @pidou46 ? :)

[pidou46]

No sorry, I haven't found a way to debug and understand form where the trouble come from.

[liangfyx]

The micropython1.2.1 version still cannot be used, and the driver still reports an error.

windows log:
Device BTHLEDevice{00001812-0000-1000-8000-00805f9b34fb}_Dev_VID&01fe61_PID&0001_REV&0123_dcda0cf92ade\8&3862bec&3&0026 is having trouble booting.

Driver name: hidbthle.inf
Class GUID: {745a17a0-74d3-11d0-b6fe-00a0c90f57da}
Service: mshidumdf
Low-level filter: WUDFRd
High level screening process:
Problem: 0x0
Problem status: 0xC00000E5

[0ut4t1m3]

@Heerkog I made the changes to the _IRQ_CONNECTION_UPDATE event you mentioned above, this is the response I see on my setup once the secrets are shared. It doesn't seem to matter if I'm sending mouse updates (mouse pointer never moves) it times out after a few seconds with "driver error" on the windows side.

Connection update: 12 0 960 0
Unhandled IRQ event: 4
...
Unhandled IRQ event: 4
Connection update: 12 0 200 0
Central disconnected: 1

Let me know if there's anything else I can do to help.

[Heerkog]

@0ut4t1m3 We might have found the issue. The supervision timeout seems to be counting down. Can you try calling notify_hid_report as soon as supervision_timeout falls below 500?

[Heerkog]

Let's also try and catch IRQ event 4.

    elif event == _IRQ_GATTS_READ_REQUEST:
        # A client has issued a read. Note: this is only supported on STM32.
        # Return a non-zero integer to deny the read (see below), or zero (or None)
        # to accept the read.
        conn_handle, attr_handle = data
        self._ble.gatts_write(attr_handle, b"\x01")

I'm denying the read here. If that doesn't work try b"\x00".

Thanks for helping. It's appreciated!

Please DO let me know if this works or not!

[0ut4t1m3]

Hi @Heerkog, I made the two changes as suggested, unfortunately it is still disconnecting. Windows is now reporting a battery level of 1% before the connection drops where before it said 100% and the logs do look slightly different.

This is how I handled notify_hid_report, let me know if this should be done differently:

elif event == _IRQ_CONNECTION_UPDATE:          # Connection parameters were updated
    #self.conn_handle, _, _, _, _ = data        # The new parameters
    #print("Connection update")
    self.conn_handle, conn_interval, conn_latency, supervision_timeout, status = data        # The new parameters
    print("Connection update: ", conn_interval, conn_latency, supervision_timeout, status)
    if supervision_timeout < 500:
        self.notify_hid_report()

This is the full output, first with self._ble.gatts_write(attr_handle , b"\x00"):

Server created
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 10 0 b'irk'
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
BLE on
Registering services
Writing service characteristics
Writing device information service characteristics
Writing battery service characteristics
Writing hid service characteristics
Advertiser created: Mouse with services: [UUID(0x1812)]
Server started
Started advertising
Central connected: 1
MTU exchanged: 23
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 1 0 b'\x00\x00\x00\x00\x00\x00\x00'
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
encryption update 1 1 0 1 16
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xc0\xf8\xe9\x0fQ\x10$\xb8\xca /\xbaC\xd3\x0e\x96\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xc0\xf8\xe9\x0fQ\x10$\xb8\xca /\xbaC\xd3\x0e\x96\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
Connection update: 12 0 960 0
Connection update: 48 0 960 0
Central disconnected: 1

And with b"\x01"

Server created
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 10 0 b'irk'
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
BLE on
Registering services
Writing service characteristics
Writing device information service characteristics
Writing battery service characteristics
Writing hid service characteristics
Advertiser created: Mouse with services: [UUID(0x1812)]
Server started
Started advertising
Central connected: 1
MTU exchanged: 23
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 1 0 b'\x00\x00\x00\x00\x00\x00\x00'
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
encryption update 1 1 0 1 16
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00W]\xe5A\r\x0fCA\x1d\xca\x7f-\x17\xe8J\xe5\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00W]\xe5A\r\x0fCA\x1d\xca\x7f-\x17\xe8J\xe5\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
Connection update: 12 0 960 0
Connection update: 48 0 960 0
Central disconnected: 1

[Heerkog]

The fact that the battery now is set to 1% indicates to me that the attr_handle we're writing is the self.h_bat. Therefore, we should not update the value, but use the stored value.

    elif event == _IRQ_GATTS_READ_REQUEST:
        # A client has issued a read. Note: this is only supported on STM32.
        # Return a non-zero integer to deny the read (see below), or zero (or None)
        # to accept the read.
        conn_handle, attr_handle = data
        self._ble.gatts_notify(conn_handle, attr_handle, None)

Similarly, I think we should try responding to connection updates with

elif event == _IRQ_CONNECTION_UPDATE:          # Connection parameters were updated
    #self.conn_handle, _, _, _, _ = data        # The new parameters
    #print("Connection update")
    self.conn_handle, conn_interval, conn_latency, supervision_timeout, status = data        # The new parameters
    print("Connection update: ", conn_interval, conn_latency, supervision_timeout, status)
    self._ble.gatts_notify(self.conn_handle, self.h_rep, None);

In case this doesn't work, we should create a loop that continuously notifies None every conn_interval.

[0ut4t1m3]

Still no luck I'm afraid. Adding self._ble.gatts_notify(conn_handle, attr_handle, None) causes a glitch and my board resets, the device I'm testing with only has native USB so I can't see the core dump sadly.
It's crashing earlier than the timeout I was seeing before this change. Before this Windows would act like the mouse was working for a few seconds (minus any movement etc) before the device would disconnect. Now the crash happens while windows is still setting up the device and doesn't show the battery level.

I added a print to _IRQ_GATTS_READ_REQUEST in case that gives you anything useful.

Server created
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 10 0 b'irk'
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
BLE on
Registering services
Writing service characteristics
Writing device information service characteristics
Writing battery service characteristics
Writing hid service characteristics
Advertiser created: Mouse with services: [UUID(0x1812)]
Server started
Started advertising
Central connected: 1
MTU exchanged: 23
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 1 0 b'\x00\x00\x00\x00\x00\x00\x00'
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
encryption update 1 1 0 1 16
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x84\xf7B\xcc_\xa0\x01m\x92\x99\x04LcT!\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00x\x84\xf7B\xcc_\xa0\x01m\x92\x99\x04LcT!\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
Connection update: 12 0 960 0
gatts read: 65535 37
gatts read: 65535 37
Unhandled exception in IRQ callback handler
Unhandled exception in IRQ callback handler
---connection is lost here as the USB connection reset---

Sorry I can't be more help than just running code. I'm a bit lost with BLE but this is an excellent library and I'm more than happy to help you get it working with 1.20+.

[Heerkog]

Oops. It should be:

elif event == _IRQ_GATTS_READ_REQUEST:
    conn_handle, attr_handle = data
    self._ble.gatts_notify(conn_handle, attr_handle)
elif event == _IRQ_CONNECTION_UPDATE:          # Connection parameters were updated
    conn_handle, conn_interval, conn_latency, supervision_timeout, status = data        # The new parameters
    print("Connection update: ", conn_interval, conn_latency, supervision_timeout, status)
    self._ble.gatts_notify(conn_handle, self.h_rep)

Things might behave a bit weird after this, but that's fixable.

[0ut4t1m3]

No still crashing:

Server created
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 10 0 b'irk'
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
BLE on
Registering services
Writing service characteristics
Writing device information service characteristics
Writing battery service characteristics
Writing hid service characteristics
Advertiser created: Mouse with services: [UUID(0x1812)]
Server started
Started advertising
Central connected: 1
MTU exchanged: 23
get secret: 2 0 None
get secret: 2 1 None
get secret: 2 2 None
get secret: 1 0 b'\x00\x00\x00\x00\x00\x00\x00'
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') None
get secret: 2 0 b'\x00\xa8\xf7|\xa6\xc0\xd8'
encryption update 1 1 0 1 16
set secret: (1, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00Jj\xce\xe5\x84^\xa6@\xdc\xd1d\xb0\xf9\xf9\xd9\x9c\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
set secret: (2, b'\x00\xa8\xf7|\xa6\xc0\xd8') b'\x00\xa8\xf7|\xa6\xc0\xd8\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00Jj\xce\xe5\x84^\xa6@\xdc\xd1d\xb0\xf9\xf9\xd9\x9c\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00'
Connection update: 12 0 960 0
Unhandled exception in IRQ callback handler
Unhandled exception in IRQ callback handler

I found this issue micropython/micropython#12832 looks like something did change last year with the change to IDFv5. I tried 1.30-preview in case the fix in that merge helped but I got the exact same error.

Note that if I comment out self._ble.gatts_notify(conn_handle, attr_handle) from _IRQ_GATTS_READ_REQUEST the old behavior is restored where it would seemingly connect and then time out after a few seconds but without the exception.

[Heerkog]

Can you try gatts_indicate instead of notify?

[0ut4t1m3]

Hey @Heerkog, no change I'm afraid,

[Heerkog]

I created a discussion on the micropython repository. No reactions yet. Any activity from others there would be helpful.

https://github.com/orgs/micropython/discussions/14258