[Feature] Support for Remote Controllers for PrimeHub/InventorHub

[TheVinhLuong102]

Hi @dlech and @laurensvalk,

With the new SPIKE/Inventor sets designed to work with in-app controllers set up in iPhone/iPad/laptop and standard controllers (e.g. XBox), what is your latest thinking re: PyBricks support for such controllers?

I notice that such support is currently not present in LEGO's SPIKE/Mindstorms Python API either. Technically, how complex is it for Python programs to receive inputs from the modern remote controllers?

Thank you.

Vinh

[laurensvalk]

Thanks for this request.

what is your latest thinking re: PyBricks support for such controllers?

It may be possible, but it's not currently high on our priority list. We might prioritize #186 for the time being.

[antonvh]

Basic bluetooth and BLE libraries would go a long way. Just make it convenient? Maybe support for specific controllers can be left up to users? Releasing mpy files to be included in your project?

[SpudGunMan]

Basic bluetooth and BLE libraries would go a long way. Just make it convenient? Maybe support for specific controllers can be left up to users? Releasing mpy files to be included in your project?

that would be nice! I agree at minimal the powered up remote would be good and the xbox controller sounds like it can work. additionally BLE connectivity for messaging with raspberry pi for example would be the bees knees

[kubastastny]

Hi @dlech and @laurensvalk,
I really like the idea of xbox controller support. Would it be easy to update the control+ controller code to support xbox controller? I'd like to do the update but it would take me a bit of tinkering to get familiar with the code. I have no experience with BLE.

[dlech]

No, I don't think it would be easy, especially for someone not familiar with BLE. If you think you are up to the challenge I would start with https://github.com/pybricks/pybricks-micropython/blob/master/pybricks/iodevices/pb_type_iodevices_lwp3device.c. It shows how to scan and connect to a device and enable notifications. Some research or reverse engineering will need to be done to figure out how the XBox controller works (what does the advertising data look like? is there any config needed after connecting or do we just enable notifications on an HID characteristic?, etc.) to know what sort of changes need to be made to that code.

[kubastastny]

Sounds like fun :) I'll have to buy xbox controller and give try to do it.
What development environment do you use?

[dlech]

We use VS Code on Ubuntu 22.04.

[kubastastny]

Hello @dlech,

I'm debugging the connection of xbox controller to technic hub. I'm fiddling with bluetooth_stm32_cc2640.c conditions based on sniffed advertisment packets. Is there any way how to connect the Visual Studio Code to live debugging on target? Because otherwise it is quite pain to debug (update code build/flash/run and guess).

[dlech]

Unfortunately, no. The hubs are protected to disallow debugging with a hardware probe. We feel your pain too.

[kubastastny]

Hi @dlech,
I have a couple questions:

1. Debug info limitation
   When I use printf function it prints the debug data into terminal but I found the data to be transmitted via uart service 0x0018. Is the standard output redirected via bluetooth connection?
   I found DBG() macro which utilize uart connection. The thing is that if I use printf or DBG() and try to print out more than 5 bytes of information from each captured event (trying to get the content of advertising packet from xbox controller in the loop) the technic hub starts to blink slowly, no output is shown in the terminal and when I hold the button on the technic hub it starts to blink very rapidly. I have to pull out batteries to recover it from this state.
   My question is whether there is any limitation in number of bytes which can be used for the debugging purposes. The dbg message seems to be 65 characters long at maximum. I guess when I accumulate more that that in the loop it will crash the thread. Could it be the explanation?

2. What is in the read_buf? Is it the raw packet data?
   I tried to print out the content of read_buf. This could not been done at once as described above but after multiple iterations of building and flashing I got advertising data by parts.
   This data do not correlate to data sniffed by nrf52840 dongle. I tried to search parts of the read_buf data with no luck. So I set the starting index manually to the beginning of the address in the read_buff to move forward.

Similar case applies also for the scanning of xbox parameters. I can see the communication between technic hub and controller but events I'm capturing in get_vendor_event function do not correlate to any data captured by by the dongle. (capturing data by wireshark following xbox controller address)

• There could be problem in the capturing (captured packets may be filtered too much so the captured data do not contain anything similar to read_buf during the event)?
• the read_buf is somehow pre-processed and do not show the raw data?
• the event capturing problem?

3. How to pair Controller and Technic hub?
   When I sniffed the connection of xbox controller to my Samsung phone (btsnoop_hci.log) the notifications with controller events (button pushed, etc...) start after the pairing is achieved.
   How can I pair the connected device? Could you provide me a little guidance how I can achieve it in the code? This part is not necessary with LEGO controller so it is missing in the code.

[dlech]

1. Is the standard output redirected via bluetooth connection?

yes

1. Could it be the explanation?

Please see pybricks/pybricks-micropython#80 (comment) for some explanation about why using print for debug is difficult.

2. What is in the read_buf?

It contains a "Legacy NPI" header and the HCI packet data for data received from the Bluetooth chip.

Most of the Bluetooth stack is running on the Bluetooth chip. The Bluetooth chip uses vendor specific commands to communicate with the MCU. So the MCU doesn't receive what is seen going over the air, but usually there is a very closely related vendor specific command with the same information.

3. How to pair Controller and Technic hub?

I would need to see more info on the Xbox Controller to know if it follows the standards that I am used to. But there is already code in Pybricks for the LEGO remote control that should show how to scan and connect to a BLE device.

[westurner]

3. How to pair Controller and Technic hub?

I would need to see more info on the Xbox Controller to know if it follows the standards that I am used to. But there is already code in Pybricks for the LEGO remote control that should show how to scan and connect to a BLE device.

Where is that at?

• https://github.com/pybricks/pybricks-micropython/blob/c8f672fcc30743efabf4014327476b759f2147e1/lib/pbio/drv/bluetooth/bluetooth_btstack.c#L522-L540 pbdrv_bluetooth_is_connected
• https://github.com/pybricks/pybricks-micropython/blob/master/lib/pbio/drv/bluetooth/bluetooth_btstack.c#L614
• https://github.com/pybricks/pybricks-micropython/blob/master/bricks/ev3dev/modules/pybricks/bluetooth.py pbdrv_bluetooth_init
• https://github.com/pybricks/pybricks-micropython/blob/master/bricks/ev3dev/modules/pybricks/bluetooth.py
• https://github.com/pybricks/pybricks-micropython/blob/master/lib/pbio/test/drv/test_bluetooth.c#L245 "this is based on simulator.py from btstack"
• https://docs.pybricks.com/en/v3.0/messaging.html#pybricks.messaging.BluetoothMailboxServer
• https://docs.pybricks.com/en/stable/search.html?q=bluetooth
• https://docs.pybricks.com/projects/pybricksdev/en/latest/api/ble/index.html?highlight=bluetooth#pybricksdev.ble.find_device
• https://docs.pybricks.com/projects/pybricksdev/en/latest/api/ble/pybricks.html?highlight=bluetooth#pybricksdev.ble.pybricks.Event
• https://github.com/pybricks/pybricksdev/tree/master/pybricksdev/ble
• https://github.com/pybricks/pybricksdev/blob/master/pybricksdev/ble/__init__.py
  • def find_device(), class BLEConnection
• https://github.com/pybricks/pybricksdev/blob/master/pybricksdev/ble/pybricks.py
• https://gist.github.com/CubeLegend/f77c6f0d8b2aaee78a30d2c6508cdaf2
  "Connecting to an Xbox one controller" (over BLE w/ ubluetooth)
• https://antonsmindstorms.com/2019/06/16/how-to-use-a-ps3-gamepad-with-micropython-on-the-ev3-brick/ (USB)
• https://docs.circuitpython.org/projects/hid/en/latest/examples.html#simple-gamepad
• Please add BLEHID gamepad support and example adafruit/Adafruit_nRF52_Arduino#228 (comment)
  • https://github.com/sparkfun/circuitpython/blob/673d2fb4b7c5e077fc5579af1d249438d17d93ad/tools/hid_report_descriptors.py#L177-L207
  • https://github.com/sparkfun/circuitpython/blob/673d2fb4b7c5e077fc5579af1d249438d17d93ad/tools/hid_report_descriptors.py#L246-L276
  • https://learn.adafruit.com/custom-hid-devices-in-circuitpython?view=all#report-descriptors-3103536