shared/tinyusb: (just rp2 for now) change USB device settings and add HID

[Molorius]

I understand this is a large commit, please let me know if there is anything I should add/modify. I have not added any documentation yet.

This moves most of the portable tinyusb code for the rp2 to shared/tinyusb. Other microcontrollers can be moved later, I wanted to get this reviewed first.

It also adds the ability to change device descriptors, change default device strings, and add HID device(s) for python use. It generally follows the naming convention of tinyusb but I tried to keep it fairly generic. This largely follows the singleton idea from rp2 rtc and rp2 watchdog.

The following example works on the Raspberry Pi Pico. boot.py changes a few settings and adds an HID mouse, main.py moves the mouse a little to the right every second. You can verify the string and id changes with lsusb on mac (cannot view cdc string) and linux, you can verify the HID portion on any OS by watching your cursor slowly move.

Further USB features can be added later using a similar pattern.

boot.py

import usb_device

descriptor = bytearray([
    0x05, 0x01,                    # USAGE_PAGE (Generic Desktop)
    0x09, 0x02,                    # USAGE (Mouse)
    0xa1, 0x01,                    # COLLECTION (Application)

    0x09, 0x01,                    #   USAGE (Pointer)
    0xa1, 0x00,                    #   COLLECTION (Physical)

    0x05, 0x09,                    #     USAGE_PAGE (Button)
    0x19, 0x01,                    #     USAGE_MINIMUM (Button 1)
    0x29, 0x03,                    #     USAGE_MAXIMUM (Button 3)
    0x15, 0x00,                    #     LOGICAL_MINIMUM (0)
    0x25, 0x01,                    #     LOGICAL_MAXIMUM (1)
    0x95, 0x03,                    #     REPORT_COUNT (3)
    0x75, 0x01,                    #     REPORT_SIZE (1)
    0x81, 0x02,                    #     INPUT (Data,Var,Abs)
    0x95, 0x01,                    #     REPORT_COUNT (1)
    0x75, 0x05,                    #     REPORT_SIZE (5)
    0x81, 0x03,                    #     INPUT (Cnst,Var,Abs)

    0x05, 0x01,                    #     USAGE_PAGE (Generic Desktop)
    0x09, 0x30,                    #     USAGE (X)
    0x09, 0x31,                    #     USAGE (Y)
    0x15, 0x81,                    #     LOGICAL_MINIMUM (-127)
    0x25, 0x7f,                    #     LOGICAL_MAXIMUM (127)
    0x75, 0x08,                    #     REPORT_SIZE (8)
    0x95, 0x02,                    #     REPORT_COUNT (2)
    0x81, 0x06,                    #     INPUT (Data,Var,Rel)

    0xc0,                          #   END_COLLECTION
    0xc0
])

hid = usb_device.HID()
hid.add_descriptor("python mouse", 2, len(descriptor), 64, descriptor)

desc = usb_device.Descriptor()
desc.device(idVendor=0x1234, idProduct=0xABCD)
desc.strings(Manufacturer="Test Manufacturer", Product="Test Product", CDC="CDC Test")

main.py

import usb_device
import time

hid = usb_device.HID()

def send_mouse(buttons, dx, dy):
    instance = 0
    report_id = 0
    buf = bytearray(3)
    buf[0] = buttons
    buf[1] = dx
    buf[2] = dy
    hid.report(instance, report_id, buf)

while True:
    print("mouse moving")
    send_mouse(0, 1, 0)
    time.sleep(1)

Filtered lsusb -v output on mac:

Test Product:

    Product ID: 0xabcd
    Vendor ID: 0x1234
    Version: 1.00
    Serial Number: xxxx
    Speed: Up to 12 Mb/s
    Manufacturer: Test Manufacturer
    Current Available (mA): 500
    Current Required (mA): 250
    Extra Operating Current (mA): 0

[Molorius]

This partially solves #6811. A lot of the discussion over there is focused on HID devices, which this fully solves.

[dpgeorge]

Thank you very much for the contribution. Refactoring code to make it reusable across ports is always a good thing to do.

Actually, @projectgus is working in the same area, adding general USB device support to TinyUSB-based ports (and stm32). The work here will clash somewhat with his work (which is not currently up anywhere public, so you couldn't have known about it).

Usually with such work we prefer to split it into bug fixes and refactoring, and new features. Having both in one PR makes it harder to review and takes longer to get things merged.

Given the above, would you be able to split this work into (1) refactoring and (2) adding HID support? The former will be a welcome improvement and should be easily mergable. The latter (adding HID support) needs quite a bit of discussion about the user-facing API, and also may not get merged (and feel free not to do the latter part so as not to potentially waste your efforts, although I suspect other users may find it very interesting and useful as a temporary solution for USB HID support).

[Molorius]

I structured this in my head in 3 phases:
(a) refactor the code to make it generic,
(b) refactor the descriptor code so additional entries can be dynamically added (only in C, no python bindings),
(c) HID support (which requires (b))

Do you want me to include (a) and (b) into the (1) refactor? Also do you want the usb_device.Descriptor python bindings?

[dpgeorge]

My main aim here is to try and not waste anyone's time, with work that won't be merged. (But don't get me wrong, the work you've done here looks good and useful!)

(a) by itself sounds like the best place to start, so if you could do just that, that would be great. And also easier to review and get merged.

Then we can move on to (b).

Also do you want the usb_device.Descriptor python bindings?

Not at this first stage.

[Molorius]

(a) done (along with some cleanup) in #9357.

[Molorius]

My end goal is to read a USB HID device using a second port acting as host on the rp2 (see https://github.com/sekigon-gonnoc/Pico-PIO-USB) and have the rp2 appear as an HID device on the main port. There are multiple steps to get there, should I continue trying to get my changes merged here, or should I explore other avenues (as they would conflict with @projectgus's work)?

[projectgus]

@Molorius Thanks for contributing this, sorry for the awkward timing!

Regarding what I'm working on, my immediate goals are:

• Deciding on a device API, as @dpgeorge mentioned. I'm keen to accommodate your requirements, ideas and maybe implementations from this PR. As well as any feedback you'd be able to give about whether an API is suitable for your purposes.
• Investigate implementing devices entirely in Python, rather than using the TUSB class drivers. Provided this is viable, TUSB can become the lower level endpoint & transfer plumbing layer and there will be a lot of flexibility for implementing custom USB behaviour in Python. (and once that is possible, build an HID driver and other class drivers in Python.)

Unfortunately I'm still at a much earlier stage with these tasks than you are with this PR, working on PoC for the second point.

My end goal is to read a USB HID device using a second port acting as host on the rp2 [...] and have the rp2 appear as an HID device on the main port

Sounds neat! So if I understand correctly, your end goal is to make the rp2 into a custom HID passthrough device?

Are you planning to also submit this "pio-usb-host" implementation to micropython, or will that be part of your project only?

(a) done (along with some cleanup) in #9357.

Thanks for putting this up, and I'll take a proper look at it ASAP.

Is having upstream MicroPython support a blocker for your project? If you're happy to keep working from this branch for now, until there's similar functionality available with a Python HID driver, does that work? I would be keen to get your feedback on such a driver, once it exists. 😁

[Molorius]

So if I understand correctly, your end goal is to make the rp2 into a custom HID passthrough device?

Correct. Coupled with gpios, etc. I have not thought about implementing custom USB behavior at the level you are planning.

Are you planning to also submit this "pio-usb-host" implementation to micropython, or will that be part of your project only?

It is already part of tinyusb, see the example. I would need to fix the micropython pio driver and part of the pio-usb driver so they can coexist first.

I have an implementation further than this in a Lua environment (not public yet), but it's only for the rp2040 and I prefer Python. Most of this work is easier for me to get running in Lua, so micropython work would be primarily for the benefit of others.

I'm not interested in fully developing USB specs within python so I will continue on my Lua project for now. I'm happy to provide feedback and code! I usually like to license my personal projects with MPL, would it need to be MIT for you to use it here?

[dhalbert]

Side note: we have done some USB host work in CircuitPython, but it is not done yet: adafruit#6125
adafruit#5986
RP2 USB host support in tinyusb: hathach/tinyusb#1412

[Molorius]

@dhalbert I saw that but I am waiting on adafruit#6247 (I wanted this fully done within a few months but I no longer think that is reasonable) and I was having a hard time updating tinyusb within circuitpython.

[dhalbert]

@Molorius Yes, sorry, usb host is not imminent in CircuitPython due to higher priorities, though we welcome outside contributions. I was more suggesting that some the code or API might come in handy for a MicroPython implementation.

[projectgus]

@Molorius That all sounds good!

I'm not interested in fully developing USB specs within python so I will continue on my Lua project for now. I'm happy to provide feedback and code!

Thank you for the offer, that would be great. Will comment again you once I have something to look at in that regard!

I usually like to license my personal projects with MPL, would it need to be MIT for you to use it here?

Yes please, ideally MIT. There is some code in the repo under other licenses (see LICENSE file) but as I understand it they are exceptions not the rule, and the actual firmware binaries generally only contain MIT-compatible code.

[paulhamsh]

For a USB Host it is possible to use the bare api rather than the TinyUSB classes (https://github.com/hathach/tinyusb/tree/master/examples/host/bare_api). Exposing that via micropython would be very nice - then we could write out own host processing.
Especially useful in cases where the host is simple, like for MIDI.

[dpgeorge]

For a USB Host it is possible to use the bare api rather than the TinyUSB classes (https://github.com/hathach/tinyusb/tree/master/examples/host/bare_api). Exposing that via micropython would be very nice - then we could write out own host processing.

This is the approach we'd like to take. See #9497 for a start, which allows writing USB devices in Python (host will follow).

[projectgus]

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.