Assigning new PID with Espressif VID to TinyUSB stack with Arduino ESP32 Core

[James-4DSystems]

Board

ESP32-S3R8 SoC

Device Description

Custom products we are building, using the ESP32-S3R8 SoC

Hardware Configuration

Many things are connected, including display, IO Expander, RTC, RTP/CTP touch.
D+/D- are going to a USB-C connector.
Hardware is working nicely and we can program it from the Arduino IDE (both 1.x and 2.x) without any problems, using the USB-C as well as the UART in conjunction with RESET and GPIO0 etc.

Version

v2.0.9

IDE Name

Arduino IDE

Operating System

Windows 10 / 11

Flash frequency

80Mhz

PSRAM enabled

yes

Upload speed

921600

Description

Not a problem as such, looking for directions as to how to set the Espressif allocated PID (ref: https://github.com/espressif/usb-pids) alongside the Espressif VID, to the TinyUSB Stack, so our products are identified as our own, rather than using the standard 0x1001 PID, and our board showing up as random other products which also use the ESP32.
I have already made contact with Espressif Support, and they suggested I raise the issue with you on this Github directly.
Their email response:

[ESP]You can configure and build one of the following USB example. and you may learn How to build ESP-IDF example first. and then you can find the related options in menuconfig of ESP-IDF USB example:

BTW, if you still want to use Arduino, since arduino-esp32 is maintained by another espressif team, it would be helpful if you could raise the issue on arduino-esp32 github directly. Thanks for your understanding and cooperation.

Is it possible to set the PID of the TinyUSB via the Arduino ESP32 core, or do we have to go about figuring out how to use the IDF and do it that way?
We are looking for a solution to be able to do this in production of our products. We have not used the IDF directly before, we are only using the Arduino ESP32 core, currently v2.0.9

It has been suggested to trying things such as USB.PID(0x8999) to set it, however this does not work.
(ref: espressif/usb-pids#101)

Please can you detail if its possible to set the PID using the Arduino ESP32 core, as we just cannot find the information required to do this. If we need to do it via the IDF directly, please indicate this, so we can focus our efforts there instead.

Sketch

N/A

Debug Message

N/A

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[mrengineer7777]

If you are using menuconfig, then it's likely you are generating a new sdkconfig.h file. At the Arduino level the sdkconfig options are pre-set, as they are pre-compiled into the binary files. It's likely you'll need to convert your project to IDF (possibly with Arduino as IDF component) to set the PID.

@SuGlider Can you confirm PID is an sdkconfig option?

[me-no-dev]

The USB class itself has getters and setters for VID/PID: https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/USB.h#L50-L54

[James-4DSystems]

If you are using menuconfig, then it's likely you are generating a new sdkconfig.h file. At the Arduino level the sdkconfig options are pre-set, as they are pre-compiled into the binary files. It's likely you'll need to convert your project to IDF (possibly with Arduino as IDF component) to set the PID.

Any idea if there are any guides on how to do this type of thing? We are only using the Arduino core, not using menuconfig in the IDF at all. Really needing to find some guidance here.

The USB class itself has getters and setters for VID/PID: https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/USB.h#L50-L54

Does this relate to what we already tried, here:

It has been suggested to trying things such as USB.PID(0x8999) to set it, however this does not work.
(ref: espressif/usb-pids#101)

[lbernstone]

As mentioned in that issue, post some code and output from setting USB.PID. Note that you must set the VID/PID before you call USB.begin().

[James-4DSystems]

Sorry after a bit more playing, it worked simply using this:

However, as expected, in Device Manager now the Native USB shows up as a device without a driver.

I just used the next available one in the Espressif PID list as a test (will change once we know the process works and will reserve one for us)

What is the solution to this now, to get it to load the same driver as before (usbser.sys) but with a different set of VID/PID values, for Windows 10/11 ?

Is there any information available to show how this is done?

I found this repo (https://github.com/kutukvpavel/Esp32-Win7-VCP-drivers), which looks to do the right type of thing, but this is more for Windows 7. Is it the driver signing in windows 10/11 which stops this method working, and we would have to get this new INF file signed in order to make this work?

Is this the type of thing required?
https://learn.microsoft.com/en-us/windows-hardware/drivers/develop/signing-a-driver

Does anyone know how this process works?

Thanks

[James-4DSystems]

Does anyone have any information as to how to practically go about doing this?
Changing the PID seems possible, however it doesnt stick with subsequent program loads.
When it has stuck, drivers dont work as they are not keyed to the VID/PID combo in Windows.
How do you get around this?
What is the process we need to follow to make a viable product with a Espressif VID and customised PID, that is workable with drivers, and that sticks over subsequent program loads?
Do we have to apply the USB.h library and setting the PID in a header library, so it does it every single program load for our product? or by making a Variant folder and adding in a pins_arduino.h file with PID/VID set in there?
And if we do that, how do we make a windows (and other OS) driver that works?
I just cannot find any information from Espressif about this stuff...

[lbernstone]

How much do you understand about USB? This is not the correct place to ask for tutorials, and google know a lot about this. https://en.wikipedia.org/wiki/USB_human_interface_device_class

[James-4DSystems]

I am specifically asking for support for ESP32-S3 for changing VID/PID based on the Espressif VID/PID register, and the ESP32-Arduino core. Every time I contact Espressif, they point me back here.
Googling does not produce any results to describe how to change the VID/PID of an Espressif ESP32-S3 and get the drivers working for the new combination.
If you have information, please share it, as we have exhausted our searches and keep being pushed back here by Support.

[igrr]

how to change the VID/PID of an Espressif ESP32-S3

Please excuse if I misunderstood, but in #8299 (comment) you wrote that you have already figured this part out: calling PID and VID methods of USB class was necessary to set the VID and the PID.

get the drivers working for the new combination

This, I'm afraid, is a bit out of scope of Arduino-esp32 project. I'll give you a few pointers to get started:

1. If your device has one of the standard classes (such as CDC) and it has the class set correctly in the device descriptor, then Windows will automatically load the correct driver. See e.g. https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-driver-installation-based-on-compatible-ids
2. If you have a vendor-specific device which is only used by one application in the system, you can load the generic WinUSB driver instead of writing your own driver or an inf file. This requires adding Windows-specific USB descriptors to your device. See https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/automatic-installation-of-winusb for more information.
3. Alternatively, you can create an .inf file and specify the driver which has to be loaded. You will also need to digitally sign the inf file. More information on Windows drivers and signing them can be found at https://learn.microsoft.com/en-us/windows-hardware/drivers/.

[James-4DSystems]

3. Alternatively, you can create an .inf file and specify the driver which has to be loaded. You will also need to digitally sign the inf file. More information on Windows drivers and signing them can be found at https://learn.microsoft.com/en-us/windows-hardware/drivers/.

Hi @igrr

Yes we have the VID and PID changed with the methods, which seems to only be for the current project loaded, but I think by making a board variant and setting it in the arduino_pins.h this gets around having to do that. So that is fine.

However when we change the VID / PID, Windows device manager does not know how to use it.
Looking at Device Manager and at the Details, Compatible ID's, the Class and SubClass are 00 I think.

How do you change it to 02 like your link in 1 suggests?
https://learn.microsoft.com/en-us/windows-hardware/drivers/usbcon/usb-driver-installation-based-on-compatible-ids

Using the methods for:
USB.usbClass(02);
USB.usbSubClass(02);

Did nothing to change what we have in the image

We did method 3, took an INF and added in our VID/PID detail, and digitally signed the driver, and installed it, but it still is not accepted, most likely as the Class and Subclass is still 00, which we dont know how you change...

It would seem in the ardino_pins.h for the variant, there is no obvious way to change it either...?

Hardware ID's changed to what we set in the sketch code though.

If you have any other suggestions, I would love to hear.

Much appreciated

[me-no-dev]

Can you please provide full USB init code that you are using. Device and all.

[James-4DSystems]

We have made our own device, but you can replicate it with the ESP32S3 DEV module in the Arduino IDE.
Here is what is selected for the board:

The code is literally:

`#include "USB.h"

void setup()
{
USB.PID(0x8209);
USB.VID(0x303a);
USB.productName("PRODUCT NAME");
USB.manufacturerName("MANUFACTURER NAME");
USB.usbClass(02);
USB.usbSubClass(02);
USB.begin();
}

void loop()
{
// put your main code here, to run repeatedly:

delay(5000);
}`

My colleague is playing around with the USB.cpp file, and added the following:

and

into the version of the 2.0.9 core, that is the only change.

But the USB methods in the sketch code I put above, does not change the Class and Subclass.
However he put the Class and Subclass into the variant folder arduino_pins.h and it did change the Class and Subclass.
he has gone home for the day, so I will retreive what he did tomorrow and advise.

Is there a reason the USB methods in the sketch do not work though? Is the USB.h / .cpp files preventing that from happening?
Can we have variant versions of this to make those work?

Hopefully this is the information you were after.
Very new to modifying these things so its all a bit of an unknown.

[me-no-dev]

You did not create any device actually. This is an empty USB device. Why not start CDC at least? It is very much possible that Windows sees an empty device and is not assigning anything to it.

[James-4DSystems]

Sorry @me-no-dev, got that fixed now, and working, but it makes it clear there are two issues (1 and 2 below) with the ESP Arduino core

1. When USB Mode is set to ‘USB OTG (Tiny USB)’ the ‘USB.begin’ actually occurs before the start of setup(). This can be observed/proved in three ways (a to c below):
   a) you can remove the USB.begin from setup and the USB Still works, and
   b) using USB.VID and USB.PID both fail with 0, because (from the source) USB is ‘started’.
   c) using an arduino_pins.h variant file, VID and PID can be set with relevant defines, unlike the USB methods in the sketch.

2. Once this USB is running, after loading another program requires that the board is ‘Hard Reset’.
   We have traced this to the “Performing 1200-bps touch reset on serial port COMxx”.
   As well as resetting the processor it puts the processor into ‘download mode’ until such time as the hard reset is done. This can be seen from logs like this:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x15 (USB_UART_CHIP_RESET),boot:0x0 (DOWNLOAD(USB/UART0))
Saved PC:0x40041a76
waiting for download
>>>> Hard resetting via RTS pin... occurs here
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x15 (USB_UART_CHIP_RESET),boot:0x0 (DOWNLOAD(USB/UART0))
Saved PC:0x40041a79
waiting for download

This is also reported in a very long Issue thread here: #6762

Whilst I think the first issue should be an easy fix, for someone with the required knowledge, the second, I’m not sure about, it might be part of the ESP32’s OTP flash.

The second issue becomes a show stopper for us, as our products dont have a reset button on them as it has been deemed not required (and shouldnt be). In USB CDC mode, this issue is not present as after programming the modules it resets and runs the application immediately - as you would expect. However in CDC mode we are left with the defult VID/PID arrangement, which we were trying to customise to give our product a different customer 'face', if you will.

At this stage we can see no option other than to use the defaults, as this reset issue is overwhelming and becomes more of a negative than any positive of customising VID/PID can overcome.

Please advise.

[kutukvpavel]

1. I'm not big on Arduino core, I prefer the IDF, but still, to me, the first issue actually makes sense. Why would you want to change VID/PID dynamically from code? And judging from you putting relevant calls in setup() anyway, I believe you don't. Then it makes sense to hard-code VID/PID, and initialize USB IF and stack "under the hood". The "hard-code" part involves you making your own board variant with relevant definitions, which, again, to my mind is one obvious thing to do for a production environment. I, for one, never needed to fiddle with VID/PID in code with IDF, just set menuconfig entry once and never touch it again.

2. That said, out of curiosity I searched this entire repo and found no references to USB.begin() (except for example sketches). So, it is not called before setup by the core (not even by static USB class constructor). Maybe there are some third-party libraries and their constructors involved?

3. As far as I understand, you've found that USBCDC class constructor is responsible for CDC endpoint setup, the device now advertises that it belongs to class 02 and corresponding drivers are installed automatically? So the whole driver/INF issue is now out of the question?

4. Not familiar with arduino loader situation on ESP32. I've glanced at esp32-s3 does not reset after upload #6762, but it's too much to dive into at the moment. As for me, I've implemented firmware update in the past using native OTA API (same context, data link over TinyUSB CDC), it's not that hard. You slice firmware binary into chunks your serial buffer can handle and then send them through (preferably with some CRC), then you provide feedback to the PC once chunk has been written, rinse and repeat. Yes, it will take some effort to wrap serial traffic into some kind of protocol (I rolled my own HDLC-inspired but byte-oriented one), and write your own script for PC, but it's still a more viable solution than chasing a ROM-code bug. And then whether and when to reset and/or switch OTA partitions is entirely under your control.

[me-no-dev]

USB is started before setup only if you select CDC_ON_BOOT. You can start that yourself later and change VID/PID that way

[rafaelgerges]

Hi @James-4DSystems, Im in the same fight than you. I was able to change de pid and vid using USB API (USB.cpp/USB.h) but was unable to set de class, subclass and protocol thefore my windows do not recognize the device. I read the you comment #8299 (comment) but did not understant what change you did on arduino_pins.h. Can you give more details? Thanks in advance.

[James-4DSystems]

Hi @James-4DSystems, Im in the same fight than you. I was able to change de pid and vid using USB API (USB.cpp/USB.h) but was unable to set de class, subclass and protocol thefore my windows do not recognize the device. I read the you comment #8299 (comment) but did not understant what change you did on arduino_pins.h. Can you give more details? Thanks in advance.

Hi @rafaelgerges
Yeah its certainly all a bit unclear and not really well documented I feel.
We have had to back track as there were just too many issues to deal with at the moment.

But if you look above at this post, #8299 (comment)
I put 2 screenshots of mods done to the USB.cpp file, which triggered the Class and Subclass to be able to be changed, which seems to be what was required for the drivers to be recognised.

In the arduino_pins.h, we started off by just tweaking the ones in the:
C:\Users[USERNAME]\AppData\Local\Arduino15\packages\esp32\hardware\esp32\2.0.9\variants\esp32s3
folder, which is the default for the ESP32-S3 Dev module, depending if you have it set that way in your boards.txt file. You set the PID and VID in there. Setting it in your sketch we found not to work very well.
If you look at some of the Adafruit board examples in the same variant folder as I listed above, then you can see how they set the VID and PID. So as long as you are using the boards.txt entry and points to the right variant folder, and edit that, you should be able to get some sort of result.
Basically if you do the following in there you should get something
#define USB_VID 0x????
#define USB_PID 0x????
#define USB_CLASS 2

I really wish this was better explained, as unless you already know how this works or are an IT guru, its a right pain to get working. Seems to be a lot of assumed knowledge, which is quite frustrating.
The issues with reset were too overwhelming for us to ignore though, so we cannot do this at the moment until that is fixed.

I hope that is somewhat helpful. I hope I have the details right, as I backtracked everything we did.

[VojtechBartoska]

is this still valid @James-4DSystems?

[James-4DSystems]

is this still valid @James-4DSystems?

I havent checked, but I dont recall seeing anything written about things being changed, so can only assume its still valid.

[me-no-dev]

I am going to close this because:

1. You can customize VID/PID at runtime by the methods you already found
2. You can customize VID/PID on board level, by editing pins_arduino.h for the board variant, as it is done here: https://github.com/espressif/arduino-esp32/blob/master/variants/um_feathers2/pins_arduino.h#L6-L10

I think this about exhausts all possible ways to do it :)

As for the Windows driver, this is out of scope here.