The view has added support for the IMX219 sensor. I have encountered some issues and need some assistance

[18506711722]

Checklist

• Checked the issue tracker for similar issues to ensure this is not a duplicate.
• Described the feature in detail and justified the reason for the request.
• Provided specific use cases and examples.

Feature description

I (1813) imx219: Detected Camera sensor PID=0x219
I (1873) video: version: 2.0.1
I (1873) video: 设备: MIPI-CSI
I (1873) video: 卡片: MIPI-CSI
I (1873) video: 总线信息: esp32p4:MIPI-CSI
I (1873) video: 功能:
I (1883) video: 视频捕捉
I (1883) video: 流媒体
I (1883) video: 设备功能:
I (1883) video: 视频捕获
I (1893) video: 视频流
I (1893) imx219: imx219_priv_ioctl = 393222
I (1893) video: chip id: 0x219
E (1903) esp_video_cam: sensor description type=1341371724 is not supported
E (1903) esp_video: video->ops->set_ext_ctrl=106
W (1913) video: failed to mirror the frame horizontally and skip this step
I (1913) imx219: imx219_set_mirror
I (1923) video: Capture format3: RAW8 BGGR, frame size: 1280x720, FPS: 60.0, for 3 seconds:

Use cases

Scenarios that require higher-quality image sensors

Alternatives

No response

Additional context

No response

[WangYuxin-esp]

The IMX219 is a RAW sensor. Please refer to SUPPORT_POLICY document and contact our sales team to initiate a commissioning request.

[18506711722]

IMX219 是一个 RAW 传感器。请参阅SUPPORT_POLICY文档，并联系我们的销售团队发起调试请求。

I referred to the porting manual and implemented the porting of part of the imx219, but it seems that there are problems with the work.
Apart from ov5647, I only have imx219 in my hand. So, I want to learn how to transplant other image sensors in video to facilitate their use in later projects

[WangYuxin-esp]

Even just activating the camera sensor is not an easy task. It typically requires the support of the sensor's original manufacturer.
Since I don't have this sensor, I'm unable to conduct certain tests. If you are familiar with the MIPI protocol or can reach out to the FAE from the sensor's original manufacturer, you can attempt to verify the MIPI-related configurations.

[18506711722]

甚至只是激活摄像头传感器也不是件容易的事。通常需要传感器原厂的支持。因为我没有这个传感器，无法进行某些测试。如果您熟悉MIPI协议，或能联系传感器原厂的FAE，可以尝试验证MIPI相关的配置。

#if CONFIG_CAMERA_IMX219_MIPI_RAW10_1280X720_60FPS
{
.name = "MIPI_2lane_24Minput_RAW10_1280x720_60fps",
.format = ESP_CAM_SENSOR_PIXFORMAT_RAW10,
.port = ESP_CAM_SENSOR_MIPI_CSI,
.xclk = 24000000,
.width = 1280,
.height = 720,
.regs = imx219_mipi_2lane_24Minput_1280x720_raw10_60fps,
.regs_size = ARRAY_SIZE(imx219_mipi_2lane_24Minput_1280x720_raw10_60fps),
.fps = 60,
.isp_info = &imx219_isp_info[1],
.mipi_info = {
.mipi_clk = IMX219_MIPI_CSI_LINE_RATE_1280x720_60FPS,
.lane_num = 2,
.line_sync_en = CONFIG_CAMERA_IMX219_CSI_LINESYNC_ENABLE ? true : false,
},
.reserved = NULL,
},
#endif

/* 720p 60fps 模式：MIPI 链路频率 456MHz，像素时钟约 91.2MHz */
#define IMX219_PIXEL_CLOCK_RATE_1280x720_60FPS (91200000ULL)
#define IMX219_MIPI_CSI_LINE_RATE_1280x720_60FPS (IMX219_PIXEL_CLOCK_RATE_1280x720_60FPS * 5)

It seems that AI will find the MIPI parameters I need in the linux driver

It's indeed not an easy task. I obtained some MIPI parameters through Ai and put them into use. I tried to obtain image data, but it seems that it doesn't work properly. The driver is rather complex. I hope to get support from the official website. Here are some DEBUG logs:

D (11454) i2c.common: new bus(0) at 0x48000db4
D (11454) i2c.common: bus clock source frequency: 40000000hz
D (11464) intr_alloc: Connected src 44 to int 5 (cpu 0)
D (11464) sccb_i2c: new io_i2c: 0x48000f6c
I (11474) imx219: imx219_detect
I (11474) imx219: imx219_power_on
I (11474) imx219: imx219_readreg=0*
I (11484) imx219: imx219_readreg=1*
I (11484) imx219: Detected Camera sensor PID=0x219
I (11494) imx219: imx219_set_format
D (11504) imx219: count=4
D (11534) imx219: count=52
I (11534) imx219: imx219_readreg=12343*
I (11534) imx219: imx219_readreg=12342*
I (11534) imx219: imx219_readreg=12341*
I (11534) imx219: imx219_readreg=12340*
I (11544) imx219: imx219_readreg=12550*
I (11544) imx219: imx219_readreg=14348*
I (11554) imx219: imx219_readreg=14349*
I (11554) imx219: imx219_readreg=14350*
I (11564) imx219: imx219_readreg=14351*
I (11574) imx219: imx219_set_stream
D (11574) imx219: Stream=0
I (11574) imx219: imx219_readreg=15361*
I (11584) imx219: imx219_readreg=15372*
D (11574) imx219: light freq=0x3c
D (11584) esp_video: alignments=64
D (11594) ISP: version: 2.0.1
D (11594) ISP: driver: MIPI-CSI
D (11594) ISP: card: MIPI-CSI
D (11604) ISP: bus: esp32p4:MIPI-CSI
D (11604) ISP: capabilities:
D (11604) ISP: VIDEO_CAPTURE
D (11604) ISP: STREAMING
D (11614) ISP: device capabilities:
D (11614) ISP: VIDEO_CAPTURE
D (11614) ISP: STREAMING
I (11624) imx219: imx219_query_para_desc=196631
I (11624) imx219: ESP_CAM_SENSOR_GAIN
D (11624) esp_video_cam: failed to query sensor id=30017
D (11634) esp_video: video->ops->query_ext_ctrl=102
D (11634) ISP: V4L2_CID_GAIN is not supported
I (11644) imx219: imx219_query_para_desc=196610
I (11644) imx219: ESP_CAM_SENSOR_EXPOSURE_VAL
I (11644) imx219: imx219_query_para_desc=196610
I (11654) imx219: ESP_CAM_SENSOR_EXPOSURE_VAL
D (11664) ISP: Exposure time:
D (11664) ISP: tline: 0
D (11664) ISP: min: 2
D (11664) ISP: max: 235
D (11664) ISP: step: 1
D (11674) ISP: current: 80
I (11674) imx219: imx219_query_para_desc=196632
I (11674) imx219: ESP_CAM_SENSOR_STATS
D (11684) esp_video_cam: failed to query sensor id=30018
D (11684) esp_video: video->ops->query_ext_ctrl=102
D (11694) ISP: V4L2_CID_CAMERA_STATS is not supported
I (11694) imx219: imx219_query_para_desc=196634
I (11704) imx219: default
D (11704) imx219: id=3001a is not supported
D (11704) esp_video_cam: failed to query sensor id=3001a
D (11714) esp_video: video->ops->query_ext_ctrl=102
D (11714) ISP: V4L2_CID_CAMERA_GROUP is not supported
I (11724) imx219: imx219_query_para_desc=196630
I (11724) imx219: ESP_CAM_SENSOR_AE_LEVEL
D (11724) esp_video_cam: failed to query sensor id=30016
D (11734) esp_video: video->ops->query_ext_ctrl=102
D (11734) ISP: V4L2_CID_CAMERA_AE_LEVEL is not supported
D (11744) ISP: version: 2.0.1
D (11744) ISP: driver: ISP
D (11744) ISP: card: ISP
D (11754) ISP: bus: esp32p4:ISP
D (11754) ISP: capabilities:
D (11754) ISP: STREAMING
D (11754) ISP: device capabilities:
D (11764) ISP: STREAMING
D (11764) esp_ipa_adn: BF index=0
D (11764) esp_ipa_acc: saturation index=0
D (11774) esp_ipa_acc: ccm index=0
D (11774) esp_ipa_af: AF edge_thresh=32
D (11774) esp_ipa_af: period_in_us=0, codes_per_step=0
I (11784) video: version: 2.0.1
I (11784) video: 设备: MIPI-CSI
I (11784) video: 卡片: MIPI-CSI
I (11794) video: 总线信息: esp32p4:MIPI-CSI
I (11794) video: 功能:
I (11794) video: 视频捕捉
I (11804) video: 流媒体
I (11804) video: 设备功能:
I (11804) video: 视频捕获
I (11814) video: 视频流
I (11814) imx219: ESP_CAM_SENSOR_IOC_G_CHIP_ID
I (11814) imx219: imx219_readreg=0*
I (11824) imx219: imx219_readreg=1*
I (11824) video: chip id: 0x219
I (11824) imx219: imx219_query_para_desc=131083
I (11834) imx219: ESP_CAM_SENSOR_VFLIP
I (11834) imx219: imx219_set_vflip
I (11844) imx219: imx219_readreg=369*
I (11844) imx219: imx219_query_para_desc=131082
I (11844) imx219: ESP_CAM_SENSOR_HMIRROR
I (11854) imx219: imx219_set_mirror
I (11854) imx219: imx219_readreg=369*
D (11864) esp_video: video->ops->enum_framesizes=NULL
I (11864) video: Capture format3: RAW8 BGGR, frame size: 1280x720, FPS: 60.0, for 3 seconds:
D (11874) intr_alloc: Connected src 104 to int 5 (cpu 0)
D (11874) intr_alloc: Connected src 105 to int 5 (cpu 0)
D (11884) intr_alloc: Connected src 106 to int 5 (cpu 0)
D (11884) intr_alloc: Connected src 107 to int 5 (cpu 0)
D (11894) intr_alloc: Connected src 108 to int 5 (cpu 0)
I (11894) video: 设置格式
D (11904) esp_video: alignments=64
I (11904) video: 请求缓冲区
I (11904) video: 查询缓冲区
I (11914) video: 缓冲区入队
I (11914) video: 查询缓冲区
I (11914) video: 缓冲区入队
I (11924) video: 启动数据传输
I (11924) imx219: GET ESP_CAM_SENSOR_DATA_SEQ
D (11924) CSI: config->queue_items: 1
D (11934) CSI: ctlr->h_res: 0d 1280
D (11934) CSI: ctlr->v_res: 0d 720
D (11934) CSI: ctlr->in_bpp: 0d 10
D (11944) CSI: ctlr->out_bpp: 0d 8
D (11944) CSI: ctlr->fb_size_in_bytes=921600
D (11944) csi_hal: CSI-DPHY lane_rate: 456000000 Hz, hs_freq: 0x6, lane_num: 0x2
D (11954) dw-gdma: new group (0) at 0x48001394
D (11954) dw-gdma: new channel (0,0) at 0x480e2484
D (11964) CSI: csi_transfer_size: 0d 144000
D (11964) intr_alloc: Connected src 24 to int 5 (cpu 0)
D (11974) dw-gdma: install interrupt service for channel (0,0)
D (11974) CSI: trans.buffer: 0x48001480, trans.buflen: 921600
D (11984) intr_alloc: Connected src 100 to int 7 (cpu 0)
I (11984) imx219: ESP_CAM_SENSOR_IOC_S_STREAM
I (11994) imx219: imx219_set_stream
D (11994) imx219: Stream=1
I (11994) video: 缓存出列
D (14904) CSI: new_trans.buffer: 0x480e24c0, new_trans.buflen: 921600
D (14904) csi_video: size=921600
I (14904) video: 缓存重新入列
I (14904) video: 缓存出列

[ddB0515]

@18506711722 don't bother as IMX219 is RAW sensors, I have similar issue with new GC2083 sensor(Issue #78 ) and issue is that you need to configure ESP ISP via JSON and that require 24 colour chart, good lightning and colour meter

Currently for us (community members) ever if we managed to get all to get sensor working from connecting side/logic we still need ISP to configure for colour accuracy and reliability and this is NOT yet made for us to be easy
For official support probably Espressif will charge adding new sensors and current sensors (at least FHD) are not easy obtainable :(
Take SC2336 for example(can't get it/buy) but you can get SC3336 that is NOT supported

So Espressif needs to make this easier for us and provide how to get new sensors working as is in our mutual interest to get it supported/working....

[18506711722]

别费心了，IMX219是RAW传感器，我用新的GC2083传感器（问题#78）也遇到类似问题，问题是你需要通过JSON配置ESP ISP，这需要24色图表、良好的光照和色度计

目前我们（社区成员）如果能从连接端/逻辑端让传感器正常工作，我们还需要ISP来配置色彩准确和可靠性，而这还不是为我们设计的简单方案。官方支持可能会收费，Espressif可能会收费，新增传感器和现有传感器（至少是FHD）不容易获得:(比如SC2336（买不到也买不到），但你可以买到SC3336。不支持的

所以Espressif需要让我们更容易做到这一点，并提供如何让新传感器正常工作，符合我们共同利益，让它得到支持和正常工作......

Theoretically, the ISP should not affect my acquisition of the original raw data from the sensor through mipi.
Regarding the issue of Espressif charging in this regard, I would like to express a different opinion. If the connection of new sensors is not provided with assistance, it will inevitably greatly restrict the promotion of P4. Such a thing will only lead to the destruction of all the efforts made by this company before. My understanding is that Espressif is essentially a chip company, and its main profit is from selling chips. All the efforts made in the software ecosystem are aimed at better promoting their chips, just like Espressif has ESP-IDF and ESP-ADF, and ST has stm32cubeMX.

[18506711722]

@WangYuxin-esp I don't quite understand that as an esp32 dedicated library, helping open-source libraries add new image sensors is related to business. Promoting mipi cameras can help increase the adoption rate of ESP32P4, and there's no reason why SONY's image sensor reference is not included in the supported sensors due to its market share in SONY's image sensors.

[WangYuxin-esp]

There are simply too many camera sensors on the market. We and the developers face the same dilemmas:

1. How to obtain hardware modules compatible with the official development board for testing?

2. How to continuously obtain technical support from the sensor manufacturer? When the sensor malfunctions, can you provide debugging suggestions?

3. Replacing the sensor, or even just the lens, requires re-tuning the IPA parameters. This not only requires the input of professional personnel but also the application of equipment such as multi-color temperature light boxes.

We have been exploring how to better conduct open source work, just as we have been developing framework source code and improving various documentation.

Thank you again for your attention. If you require commercial cooperation, please contact our business team and mail us the hardware module before we can proceed with a tuning plan.

Regarding the IMX219, although it is discontinued and technical support is no longer available from SONY, thank God I was able to purchase it online, thus giving me another opportunity to try it out. Please read our support policy. Thank you.

[18506711722]

市场上的摄像头传感器实在太多了。我们和开发者面临同样的两难：

1. 如何获得与官方开发板兼容的硬件模块进行测试？
2. 如何持续获得传感器制造商的技术支持？传感器故障时，你能提供调试建议吗？
3. 更换传感器，甚至仅更换镜头，都需要重新调节IPA参数。这不仅需要专业人员的参与，还需要多色温度灯箱等设备的应用。

我们一直在探索如何更好地开展开源工作，就像我们开发框架源代码和改进各种文档一样。

再次感谢大家的关注。如果您需要商业合作，请联系我们的业务团队并将硬件模块邮寄给我们，我们才能继续进行调校计划。

关于IMX219，虽然它已经停产，索尼也不再提供技术支持，但谢天谢地我能在线购买，这给了我另一个试用的机会。请阅读我们的支持政策。谢谢。

I fully understand your hardships and am grateful for the significance ESP32 has brought to this world.
The mipi driver is not as easy to debug as other embedded peripherals (at least I don't quite understand how to debug it).
I referred to the linux driver and Ai of IMX219 for reference and read part of the V4L2 code in the video section. It seems that set_stream was successful and i2c communication was also successful, but I still couldn't find the specific error location during the process of porting IMX219.
My suggestion is to add some reference and guide for the connection of image sensors from certain manufacturers, especially for the isp debugging part, which should focus more on guiding the debugging methods. Different image sensors and different modules and lenses should have many differences. If all of them need to be adjusted by the official Espressif team, the workload will undoubtedly be huge. The advantages of ESP32P4 over S3, such as its higher main frequency and MIPI interface, should be fully demonstrated. How can the usage be increased?