Device failure diagnostics in stream callback needed (AEGHB-439)

[anton-malakhov]

Environment

• Development Kit: [none]
• Module or chip used: [ESP32-S2-Mini]
• IDF version (run git describe --tags to find it):
  v5.1.1
• Build System: [CMake]
• Compiler version (run xtensa-esp32-elf-gcc --version to find it):
  toolchain-xtensa-esp32s2 @ 8.4.0+2021r2-patch5
• Operating System: [Linux]
• Power Supply: [external 5V]

ESPHome build system environment

HARDWARE: ESP32S2 240MHz, 320KB RAM, 4MB Flash

• framework-espidf @ 3.40405.230623 (4.4.5)
• tool-cmake @ 3.16.4
• tool-ninja @ 1.7.1
• toolchain-esp32ulp @ 2.35.0-20220830
• toolchain-riscv32-esp @ 8.4.0+2021r2-patch5
• toolchain-xtensa-esp32s2 @ 8.4.0+2021r2-patch5

Problem Description

I created https://github.com/anton-malakhov/esphome_webcam component for ESPHome based on this esp-iot-solution repository and its usb_camera_mic_spk example. Testing it in real application in long run, I found that USB 2.0 AV Capture dongle enters some failure state when it can just occasionally generate valid frame, all the time I'm getting repeated messages like:

[20:14:40][V][esp-idf:000]: W (9776) USB_STREAM: bad packet: error bit set

[20:14:40][V][esp-idf:000]: W (9780) USB_STREAM: bad packet: error bit set

[20:14:40][V][esp-idf:000]: W (9929) USB_STREAM: DROP NO EOF, got data=2793 B

The first two are repeated multiple times before the last message appears.
The problem is that I need 1) a way to detect device failure state 2) a way to hard-reset it. If the last cannot be done via USB drivers, ok, I can power-cycle it. But I need somehow to detect this failure first.

Expected Behavior

usb_streaming_state_register(&stream_state_changed_cb, NULL) registers callback where failures are reported.

Actual Behavior

The verbose messages about failures are coming from usb_stream component and are not passed to user-visible callbacks.

Steps to reproduce

1. Setup working usb_camera_mic_spk example
2. Wait/provoke/simulate USB device failure state
3. Show how to get this state on the user callback side

[leeebo]

Hi @anton-malakhov for the questions you mentioned

I found that USB 2.0 AV Capture dongle enters some failure state when it can just occasionally generate valid frame,

Can it resume to the normal state after this case happens?

usb_streaming_state_register(&stream_state_changed_cb, NULL) registers callback where failures are reported.

Yes, we can add such failure events to the callback.

[20:14:40][V][esp-idf:000]: W (9776) USB_STREAM: bad packet: error bit set
[20:14:40][V][esp-idf:000]: W (9780) USB_STREAM: bad packet: error bit set
[20:14:40][V][esp-idf:000]: W (9929) USB_STREAM: DROP NO EOF, got data=2793 B

The warning message shows the camera frame header mark a error, which may be that the camera has an internal fault, or the USB Fullspeed bandwidth is not enough to transmit the current frame.

[anton-malakhov]

I found that USB 2.0 AV Capture dongle enters some failure state when it can just occasionally generate valid frame,

Can it resume to the normal state after this case happens?

Yes. Further investigation shows that failure state is provoked by night mode of the camera, which is not working well with this USB dongle. However, after color mode is restored, USB device still needs the reset in order to start operating normally.

Yes, we can add such failure events to the callback.

Thank you!

The warning message shows the camera frame header mark a error, which may be that the camera has an internal fault, or the USB Fullspeed bandwidth is not enough to transmit the current frame.

The later is unlikely because sometimes correct frames are still generated and their size is regular.

[leeebo]

Hi @anton-malakhov Please try https://github.com/espressif/esp-usb/tree/master/host/class/uvc/usb_host_uvc if you still have issues with the camera under the usb_stream driver