Software caused connection to abort error

[dlee640]

• bleak version: 0.14.2
• Python version: 3.7.5
• Operating System: Linux Ubuntu 18.04
• BlueZ version (bluetoothctl -v) in case of Linux: 5.48

Description

I am running the BLE client within the python thread that utilizes behavior tree for robotics application. What's interesting is that it will run about 10% of the time (upon keep trying), but most of times, it gives me bleak.exc.BleakDBusError: [org.bluez.Error.Failed] Software caused connection abort error. Can someone explain how to address this issue and why this issue gets resolved upon just trying again?

P.S. It seems that getting rid of the thread is not a viable solution as already shown from: #666.

Exception in thread Thread-37:
Traceback (most recent call last):
  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "bluetooth_door_tool.py", line 402, in open_bluetooth_door
    run = asyncio.run(self.check_door_status())
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 579, in run_until_complete
    return future.result()
  File "bluetooth_door_tool.py", line 362, in check_door_status
    await self.client.connect()
  File "/home/BOSDYN/dlee/venv/ble_project/lib/python3.7/site-packages/bleak/backends/bluezdbus/client.py", line 278, in connect
    assert_reply(reply)
  File "/home/BOSDYN/dlee/venv/ble_project/lib/python3.7/site-packages/bleak/backends/bluezdbus/utils.py", line 23, in assert_reply
    raise BleakDBusError(reply.error_name, reply.body)
bleak.exc.BleakDBusError: [org.bluez.Error.Failed] Software caused connection abort

[dlech]

Can you share code for a minimal reproducible example?

[dlee-bd]

I cannot directly post the code due to the company policy, but I can provide you with the skeleton code of how things are structured at the moment. I apologize for not being able to provide the full code.

Basically, my code uses BLE to connect to a bluetooth door, then sends commands to either open/close/query the door. The code errors at the self.client.connect() part of the async def open(self) & async def close(self) below.

My executable.py has following:

class BLEDoorTool:
...
    async def open(self):
        await self.client.connect()
        await self.client.start_notify(self.char_uuid, self.on_message)
        await self.authenticate()
        await asyncio.sleep(0.5)
        await self.sendOpen()
        await asyncio.sleep(12.0) # estimated time to open the door
        await self.sendQuery()
        await self.client.stop_notify(self.char_uuid)
        await self.client.disconnect()
        
    async def close(self):
        await self.client.connect()
        await self.client.start_notify(self.char_uuid, self.on_message)
        await self.authenticate()
        await self.sendClose()
        await asyncio.sleep(12.0) # estimated time to close the door
        await self.sendQuery()
        await self.client.stop_notify(self.char_uuid)
        await self.client.disconnect()
        
    async def check_door_status(self):
        await self.client.connect()
        await self.client.start_notify(self.char_uuid, self.on_message)
        await self.authenticate()
        await self.sendQuery()
        await self.client.stop_notify(self.char_uuid)
        await self.client.disconnect()

    def teardown(self):
        global messageConcatBuffer
        global messageCountToExpect
        global api_session_key
        global lastActionId
        global waitingForAuthenticationQueryActionResponse
        
        messageConcatBuffer = ''
        messageCountToExpect = 1
        api_session_key = None
        lastActionId = 0
        waitingForAuthenticationQueryActionResponse = False

  def open_bluetooth_door(self, session_id, exit_ret_val, inputs):
          global door_state
  
          # Step 1: Check waypoint and obtain the associated target bluetooth MAC address
          self.get_robot_id()
          self.check_waypoint_id_and_obtain_MAC_addr()
          
          # Step 2: Create BLE Client using the MAC address obtained in step 1
          self.client = BleakClient(self.target_address)
          
          # Step 3: Open door
          run = asyncio.run(self.check_door_status())
          self.teardown()
          
          if door_state == 'open':
              _LOGGER.info("Mission successful. Door is already open.")
              exit_ret_val[session_id] = remote_pb2.TickResponse.STATUS_SUCCESS
          else:
              while door_state == 'closed':
                  _LOGGER.info("Door is currently closed. Sending request to open the door.")
                  run = asyncio.run(self.open())
                  self.teardown()
                  exit_ret_val[session_id] = remote_pb2.TickResponse.STATUS_SUCCESS
                  
          return exit_ret_val[session_id]

    def close_bluetooth_door(self, session_id, exit_ret_val, inputs):
        global door_state

        # Step 1: Check waypoint and obtain the associated target bluetooth MAC address
        self.get_robot_id()
        self.check_waypoint_id_and_obtain_MAC_addr()
        
        # Step 2: Create BLE Client using the MAC address obtained in step 1
        self.client = BleakClient(self.target_address)
        
        # Step 3: Close door
        run = asyncio.run(self.check_door_status())
        self.teardown()
        
        if door_state == 'closed':
            _LOGGER.info("Mission successful. Door is already closed.")
            exit_ret_val[session_id] = remote_pb2.TickResponse.STATUS_SUCCESS
        else:
            while door_state == 'open':
                _LOGGER.info("Door is currently open. Sending request to close the door.")
                run = asyncio.run(self.close())
                self.teardown()
                exit_ret_val[session_id] = remote_pb2.TickResponse.STATUS_SUCCESS
                
        return exit_ret_val[session_id]
        
...

In the same executable.py file, I have the main function that runs the open_bluetooth_door and close_bluetooth_door above. The skeleton code for the main function is shown below.

  def main(argv):
  ...
  bluetooth_door_tool = BLEDoorTool(robot)
  
  # Create a service runner for remote mission callbacks to control the shared state
        not_running = True
        while not_running:
            try:
                open_service  = run_bluetooth_door_tool_service(bluetooth_door_tool.open_bluetooth_door, 
                                robot, OPEN_BLUETOOTH_DOOR_SERVICE_PORT, logger=_LOGGER)
                close_service = run_bluetooth_door_tool_service(bluetooth_door_tool.close_bluetooth_door, 
                                robot, CLOSE_BLUETOOTH_DOOR_SERVICE_PORT, logger=_LOGGER)
                not_running = False
            except Exception as e:
                _LOGGER.fatal("An error occurred: {e}")
                not_running = True
  ...

In the separate helpers.py file, there is the run_bluetooth_door_service method that the open_bluetooth_door() and close_bluetooth_door() methods gets passed into to initiate & run a robot service. The code snippet is shown below.

  def run_bluetooth_door_tool_service(thread_fn, bosdyn_sdk_robot, port, logger=None):
      """Helper to create the remote mission service using a specific worker function."""
      # Proto service specific function used to attach a servicer to a server.
      add_servicer_to_server_fn = remote_service_pb2_grpc.add_RemoteMissionServiceServicer_to_server
  
      # Instance of the servicer to be run.
      service_servicer = BluetoothDoorToolRemoteMissionCallbackServicer(thread_fn, bosdyn_sdk_robot,
                                                                    logger=logger)
      return GrpcServiceRunner(service_servicer, add_servicer_to_server_fn, port, logger=logger)

Currrently, my workaround has been attempting the .connect() multiple times until it works using try & except. This is a hacky solution that works better than running it single time, but I would like to know if there is a better way to address this. Thank you so much.

[dlech]

I don't know if it will fix the problem, but there are know issues with reusing the same BleakClient for multiple connections. To work around this, you could change your code like this:

    async def open(self):
        async with BleakClient(self.device) as client:
            await client.start_notify(self.char_uuid, self.on_message)
            await self.authenticate(client)
            await asyncio.sleep(0.5)
            await self.sendOpen(client)
            await asyncio.sleep(12.0) # estimated time to open the door
            await self.sendQuery(client)

[dlee-bd]

Thank you for your response. I will try this today and let you know.

[dlee-bd]

It turns out that although the async with BleakClient(self.device) as client method has mitigated the frequency of the error occurrence to some extent, the issue still persists, and happens ~30% of the time I try to run the code.

Do you have any other recommendations on how to get around this issue?