Issue working with esp_cam_sensor component (AEGHB-939)

[Ncouch64]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate.
• Provided a clear description of your suggestion.
• Included any relevant context or examples.

Issue or Suggestion Description

Hello, I am working with a new esp32p4 dev board and am trying to figure out the new methods for image capture. I am familiar with esp32 and esp32s3 using the esp32-camera component and have a few projects using it.

I am having issues getting the camera capture working with the P4 board, I though I was doing it correctly before realizing that there is a new driver for the camera sensor. Is there examples that show how to properly capture the camera images using the new component? The board came with example code but the example for the camera doesn't make this very clear.

Basically, I am wondering if there is a clear example for using this component. I would like to just figure out how to capture a single image and potentially save that to SD for viewing. If anyone can point me in the correct direction I would greatly appreciate it!

Edit:
ESP Board: ESP32-P4-Nano Dev Board
Camera Sensor: OV5647

[Ncouch64]

I adjusted the detect test example of esp_cam_sensor adding:

#elif CONFIG_CAMERA_OV5647
#include "ov5647.h"
#define SCCB0_CAM_DEVICE_ADDR OV5647_SCCB_ADDR
#else
&
#elif CONFIG_CAMERA_0V5647
    printf("CONFIG_CAMERA_0V5647\n");
    esp_cam_sensor_device_t *cam0 = ov5647_detect(&cam0_config);
    TEST_ASSERT_MESSAGE(cam0 != NULL, "detect fail");

    TEST_ESP_OK(esp_cam_sensor_del_dev(cam0));
#endif

To correspond with the sensor that I am using. I set OV5647 to be the only sensor in menuconfig and during the build phase it fails:

warning: unused variable 'cam0_config' [-Wunused-variable]
   87 |     esp_cam_sensor_config_t cam0_config = {
      |                             ^~~~~~~~~~~
[3/9] Performing build step for 'bootloader'

I am not sure why exactly this would be happening.

[WangYuxin-esp]

@Ncouch64 Hi, The recommended framework for ESP32-P4 video development currently is esp-video-components. It provides management capabilities for devices such as MIPI-CSI, ISP, JPEG, H264, etc.
We are making it work like V4L2 APIs. Here are some ready-made examples for you to use.
The usage of these APIs is roughly as follows.

[Ncouch64]

@Ncouch64 Hi, The recommended framework for ESP32-P4 video development currently is esp-video-components. It provides management capabilities for devices such as MIPI-CSI, ISP, JPEG, H264, etc. We are making it work like V4L2 APIs. Here are some ready-made examples for you to use. The usage of these APIs is roughly as follows.

Awesome, thank you for this information! I will begin digging into this.

[Ncouch64]

@WangYuxin-esp I am trying to get one of the examples working, specifically the UVC example (Also tried the capture stream example but couldnt get the wifi to connect). I have the correct camera sensor selected in the IDF menu config. Builds and flashes ok, on run I am getting this issue:

I (33) boot: ESP-IDF v5.4 2nd stage bootloader
I (33) boot: compile time Jan  7 2025 11:56:04
I (33) boot: Multicore bootloader
I (34) boot: chip revision: v0.1
I (36) boot: efuse block revision: v0.1
I (39) boot.esp32p4: SPI Speed      : 80MHz
I (43) boot.esp32p4: SPI Mode       : DIO
I (47) boot.esp32p4: SPI Flash Size : 2MB
I (51) boot: Enabling RNG early entropy source...
I (55) boot: Partition Table:
I (58) boot: ## Label            Usage          Type ST Offset   Length
I (64) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (71) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (77) boot:  2 factory          factory app      00 00 00010000 00100000
I (84) boot: End of partition table
I (87) esp_image: segment 0: paddr=00010020 vaddr=40050020 size=17778h ( 96120) map
I (109) esp_image: segment 1: paddr=000277a0 vaddr=30100000 size=0002ch (    44) load
I (110) esp_image: segment 2: paddr=000277d4 vaddr=3010002c size=0003ch (    60) load
I (113) esp_image: segment 3: paddr=00027818 vaddr=4ff00000 size=08800h ( 34816) load
I (127) esp_image: segment 4: paddr=00030020 vaddr=40000020 size=4bb94h (310164) map
I (171) esp_image: segment 5: paddr=0007bbbc vaddr=4ff08800 size=09ea4h ( 40612) load
I (179) esp_image: segment 6: paddr=00085a68 vaddr=4ff12700 size=029c4h ( 10692) load
I (186) boot: Loaded app from partition at offset 0x10000
I (187) boot: Disabling RNG early entropy source...
I (197) hex_psram: vendor id    : 0x0d (AP)
I (198) hex_psram: Latency      : 0x01 (Fixed)
I (198) hex_psram: DriveStr.    : 0x00 (25 Ohm)
I (199) hex_psram: dev id       : 0x03 (generation 4)
I (203) hex_psram: density      : 0x07 (256 Mbit)
I (208) hex_psram: good-die     : 0x06 (Pass)
I (212) hex_psram: SRF          : 0x02 (Slow Refresh)
I (217) hex_psram: BurstType    : 0x00 ( Wrap)
I (221) hex_psram: BurstLen     : 0x03 (2048 Byte)
I (225) hex_psram: BitMode      : 0x01 (X16 Mode)
I (230) hex_psram: Readlatency  : 0x04 (14 cycles@Fixed)
I (235) hex_psram: DriveStrength: 0x00 (1/1)
I (239) MSPI DQS: tuning success, best phase id is 2
I (426) MSPI DQS: tuning success, best delayline id is 11
I (426) esp_psram: Found 32MB PSRAM device
I (427) esp_psram: Speed: 200MHz
I (428) hex_psram: psram CS IO is dedicated
I (431) cpu_start: Multicore app
I (1383) esp_psram: SPI SRAM memory test OK
I (1392) cpu_start: Pro cpu start user code
I (1393) cpu_start: cpu freq: 360000000 Hz
I (1393) app_init: Application information:
I (1393) app_init: Project name:     uvc
I (1396) app_init: App version:      1
I (1400) app_init: Compile time:     Jan  7 2025 11:54:43
I (1405) app_init: ELF file SHA256:  43156b744...
I (1409) app_init: ESP-IDF:          v5.4
I (1413) efuse_init: Min chip rev:     v0.1
I (1417) efuse_init: Max chip rev:     v1.99 
I (1421) efuse_init: Chip rev:         v0.1
I (1425) heap_init: Initializing. RAM available for dynamic allocation:
I (1431) heap_init: At 4FF17140 len 00023E80 (143 KiB): RAM
I (1437) heap_init: At 4FF3AFC0 len 00004BF0 (18 KiB): RAM
I (1442) heap_init: At 4FF40000 len 00060000 (384 KiB): RAM
I (1447) heap_init: At 50108080 len 00007F80 (31 KiB): RTCRAM
I (1453) heap_init: At 30100068 len 00001F98 (7 KiB): TCM
I (1458) esp_psram: Adding pool of 32768K of PSRAM memory to heap allocator
I (1465) spi_flash: detected chip: generic
I (1468) spi_flash: flash io: dio
W (1471) spi_flash: Detected size(16384k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (1484) main_task: Started on CPU0
I (1514) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1514) main_task: Calling app_main()
E (1514) i2c.master: I2C transaction unexpected nack detected
E (1524) i2c.master: s_i2c_synchronous_transaction(918): I2C transaction failed
E (1524) i2c.master: i2c_master_transmit_receive(1214): I2C transaction failed
E (1534) sccb_i2c: s_sccb_i2c_transmit_receive_reg_a16v8(112): faled to transmit receive
E (1544) ov5647: ov5647_get_sensor_id(246): read pid_h failed
E (1544) ov5647: Get sensor ID failed
E (1554) esp_video_init: failed to detect MIPI-CSI camera sensor with address=36

The camera is properly connected and the light shows that it is active. Any idea what could cause this issue? Is it because it needs to be further configured in some way? The specific dev board I am using is the esp32p4-nano from waveshare.

Thanks for the help, I have been having a lot of difficulty getting things to work with this P4 board. This is the first time working with a mipi camera like this and am entirely unfamiliar with it.

[wujiangang]

@Ncouch64 The bottom lines show that the sensor is not detected properly. According to the ESP32-P4-NANO schematic, the I2C SCL and SDA pins are GPIO8 and GPIO7 on this board. However, this example uses GPIO34 and GPIO31. Please configure these settings in menuconfig under Example Configuration, try again and feed back, thanks.

@WangYuxin-esp I suggest updating the README to clarify that when using a different board, the I2C pins may vary and must be configured in menuconfig accordingly.

[Ncouch64]

@Ncouch64 The bottom lines show that the sensor is not detected properly. According to the ESP32-P4-NANO schematic, the I2C SCL and SDA pins are GPIO8 and GPIO7 on this board. However, this example uses GPIO34 and GPIO31. Please configure these settings in menuconfig under Example Configuration, try again and feed back, thanks.

@WangYuxin-esp I suggest updating the README to clarify that when using a different board, the I2C pins may vary and must be configured in menuconfig accordingly.

I really appreciate the assistance here, like I said I am still trying to get familiar with all this. I was attempting to parse through the schematic but fear I don't have the best background for that sort of thing. I am working on shaping up!

Regardless, I really do appreciate the assistance. I will be back to testing soon!

[Ncouch64]

@wujiangang You were entirely correct about that, I am now able to flash and run this example (UVC) with just one final issue that I am trying to figure out. Here is my output:

I (25) boot: ESP-IDF v5.4 2nd stage bootloader
I (25) boot: compile time Jan  7 2025 11:56:04
I (26) boot: Multicore bootloader
I (27) boot: chip revision: v0.1
I (28) boot: efuse block revision: v0.1
I (32) boot.esp32p4: SPI Speed      : 80MHz
I (36) boot.esp32p4: SPI Mode       : DIO
I (40) boot.esp32p4: SPI Flash Size : 2MB
I (43) boot: Enabling RNG early entropy source...
I (48) boot: Partition Table:
I (50) boot: ## Label            Usage          Type ST Offset   Length
I (57) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (63) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (70) boot:  2 factory          factory app      00 00 00010000 00100000
I (77) boot: End of partition table
I (80) esp_image: segment 0: paddr=00010020 vaddr=40050020 size=17778h ( 96120) map
I (104) esp_image: segment 1: paddr=000277a0 vaddr=30100000 size=0002ch (    44) load
I (106) esp_image: segment 2: paddr=000277d4 vaddr=3010002c size=0003ch (    60) load
I (109) esp_image: segment 3: paddr=00027818 vaddr=4ff00000 size=08800h ( 34816) load
I (124) esp_image: segment 4: paddr=00030020 vaddr=40000020 size=4bb94h (310164) map
I (177) esp_image: segment 5: paddr=0007bbbc vaddr=4ff08800 size=09ea4h ( 40612) load
I (186) esp_image: segment 6: paddr=00085a68 vaddr=4ff12700 size=029c4h ( 10692) load
I (194) boot: Loaded app from partition at offset 0x10000
I (194) boot: Disabling RNG early entropy source...
I (204) hex_psram: vendor id    : 0x0d (AP)
I (204) hex_psram: Latency      : 0x01 (Fixed)
I (205) hex_psram: DriveStr.    : 0x00 (25 Ohm)
I (205) hex_psram: dev id       : 0x03 (generation 4)
I (210) hex_psram: density      : 0x07 (256 Mbit)
I (214) hex_psram: good-die     : 0x06 (Pass)
I (218) hex_psram: SRF          : 0x02 (Slow Refresh)
I (223) hex_psram: BurstType    : 0x00 ( Wrap)
I (227) hex_psram: BurstLen     : 0x03 (2048 Byte)
I (232) hex_psram: BitMode      : 0x01 (X16 Mode)
I (236) hex_psram: Readlatency  : 0x04 (14 cycles@Fixed)
I (241) hex_psram: DriveStrength: 0x00 (1/1)
I (246) MSPI DQS: tuning success, best phase id is 2
I (433) MSPI DQS: tuning success, best delayline id is 11
I (433) esp_psram: Found 32MB PSRAM device
I (433) esp_psram: Speed: 200MHz
I (434) hex_psram: psram CS IO is dedicated
I (437) cpu_start: Multicore app
I (1390) esp_psram: SPI SRAM memory test OK
I (1399) cpu_start: Pro cpu start user code
I (1399) cpu_start: cpu freq: 360000000 Hz
I (1399) app_init: Application information:
I (1399) app_init: Project name:     uvc
I (1403) app_init: App version:      1
I (1407) app_init: Compile time:     Jan  8 2025 09:01:16
I (1412) app_init: ELF file SHA256:  e64ff7b1a...
I (1416) app_init: ESP-IDF:          v5.4
I (1420) efuse_init: Min chip rev:     v0.1
I (1424) efuse_init: Max chip rev:     v1.99 
I (1428) efuse_init: Chip rev:         v0.1
I (1432) heap_init: Initializing. RAM available for dynamic allocation:
I (1438) heap_init: At 4FF17140 len 00023E80 (143 KiB): RAM
I (1443) heap_init: At 4FF3AFC0 len 00004BF0 (18 KiB): RAM
I (1449) heap_init: At 4FF40000 len 00060000 (384 KiB): RAM
I (1454) heap_init: At 50108080 len 00007F80 (31 KiB): RTCRAM
I (1459) heap_init: At 30100068 len 00001F98 (7 KiB): TCM
I (1465) esp_psram: Adding pool of 32768K of PSRAM memory to heap allocator
I (1472) spi_flash: detected chip: generic
I (1475) spi_flash: flash io: dio
W (1478) spi_flash: Detected size(16384k) larger than the size in the binary image header(2048k). Using the size in the binary image header.
I (1491) main_task: Started on CPU0
I (1521) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1521) main_task: Calling app_main()
I (1521) ov5647: Detected Camera sensor PID=0x5647
I (1591) example: version: 0.8.0
I (1591) example: driver:  MIPI-CSI
I (1591) example: card:    MIPI-CSI
I (1591) example: bus:     esp32p4:MIPI-CSI
I (1591) example: capabilities:
I (1591) example:       VIDEO_CAPTURE
I (1591) example:       STREAMING
I (1601) example: device capabilities:
I (1601) example:       VIDEO_CAPTURE
I (1601) example:       STREAMING
I (1611) example: version: 0.8.0
I (1611) example: driver:  H.264
I (1611) example: card:    H.264
I (1611) example: bus:     esp32p4:H.264
I (1621) example: capabilities:
I (1621) example:       STREAMING
I (1621) example: device capabilities:
I (1631) example:       STREAMING
I (1631) example: Format List
I (1631) example:       Format(1) = H.264
I (1631) example: Frame List
I (1641) example:       Frame(1) = 1280 * 720 @30fps
I (1641) usbd_uvc: UVC Device Start, Version: 1.1.2
I (1651) main_task: Returned from app_main()

So based on this I believe everything is working as it should, only problem is that I am not able to get anything to display in the pot-player app. No cameras are detected in the app itself. I am going to be digging through settings to see if I am maybe missing something that needs to be configured in pot-player. Anything I could be missing?

[Ncouch64]

Update for this. I have managed to get the simple_video_server example working and at the very least was able to verify that the camera is working. I still cannot get the uvc or capture_stream examples working. I am wondering if there is an IDF setting that I am missing to get it to stream to pot-player or if there is a pot-player setting I am missing.

I will need to spend some time trying to figure out the new methods that are done here. I was using esp32-cam component with the new opencv component to do some image processing tasks and was interested in testing it on the P4 but it seems it will be more of a task than originally anticipated.

[WangYuxin-esp]

For capture_stream example, configure the correct pins and enable the correct camera options.
For uvc example, make sure you are connected to the correct USB port (it is not the one that provides power). The correct USB port may be labeled USB-OTG.

[wujiangang]

For waveshare board, slot 5 is used for power & flash & debug, you need to use slot 13, it's high speed USB port.

[Ncouch64]

@wujiangang @WangYuxin-esp Ah I understand this is making sense. I am going to try and source a USB A to USB A cable and test this out again. Thanks again for the assistance!

[Ncouch64]

Alright, I have sourced a cable and have been able to successfully run the UVC example. I am impressed with the results so far, the P4 seems much more capable for image processing tasks than even the S3.

I did have an additional question. I looked at the specifications for the sensor on my P4 board, OV5647, and it lists a few supported resolutions including 640x480 @ 90fps which I am specifically interested in. Are there limitations to supported resolutions and framerates for this component? I have been trying to get some resolutions working but have encountered issues.

Edit: It seems that the idf has these formats for the OV5647 and trying a resolution outside of these fails:

But the OV5647 supposedly supports:

Are these resolutions possible or are they currently limited to what is shown in the IDF? This may be because of the mipi interface which admittedly I am almost entirely unfamiliar with.

Finally, something interesting that I am noticing is that when I do a vertical flip on the image it seems to be in grayscale now. I may need to do more research on V4L2.