Any support for ov5640 possible?

[searchingforcode]

Hi is it possible to integrate ov5640 driver? I found drivers of ov5640 for linux. Can esp32 process ov5640 images?
Thanks

[Niek]

+1 for this

I know there is OV3660 support in the corresponding branch (https://github.com/espressif/esp32-camera/tree/ov3660-support), but the OV5640 is more widely available on Taobao/AliExpress and supports 30fps @ 1080p resolution.

Linux driver for reference: https://github.com/torvalds/linux/blob/master/drivers/media/i2c/ov5640.c

ArduCAM code: https://github.com/ArduCAM/Arduino/blob/master/ArduCAM/ov5640_regs.h

[me-no-dev]

don't believe frame per second numbers like this :) while the camera might be able to do it, it does not mean that the ESP can handle it :) this is mainly where we hit a wall.
Also Adding a new driver takes time and effort that should have some sort of positive outcome in therms of business and OV5640 does not look like a prime time candidate. If any of you is willing to put some time into it, I can guide you through getting the driver to work.

Plus, I think we will have enough ov3660 cameras on the market soon :P

[Niek]

OK, that makes sense - looking forward to see those OV3660 sensors/updated ESP-EYE board 👍

[searchingforcode]

Hi great. Waiting for the ov3660 modules. And more thing, there is an micro-rtsp implementation for esp32 on platformio. Can it be ported/implemented to esp-idf? Thanks a lot :)

[Niek]

@searchingforcode ADF will (very?) soon have SIP/VoIP added: espressif/esp-adf#106 / espressif/esp-adf#81 - I guess it will be possible to add a MJPEG video stream in there as well, which will be more powerful than RTSP.

[me-no-dev]

MJPEG we already do :) example is in ESP-WHO repository.

[Niek]

I am aware of that, I meant MJPEG + (microphone) audio combined over SIP so it's possible to create a true "webcam" video+audio stream.

[me-no-dev]

Out SIP engineer has left the company, but I will poke arround and see if I can finish it. That said, SIP uses h263 and h264 for video transmission. Voice options are a couple, but all feasible. h263/h264 on ESP32 .... not a thing yet and maybe never a thing :)

[searchingforcode]

I dont think h263/h264 is feasible on esp32. Any idea about streaming to a cloud of this mjpeg streaming?

[Niek]

Out SIP engineer has left the company, but I will poke arround and see if I can finish it. That said, SIP uses h263 and h264 for video transmission. Voice options are a couple, but all feasible. h263/h264 on ESP32 .... not a thing yet and maybe never a thing :)

I thought SIP allowed different video codecs such as MJPEG as well, but I might be mistaken. Anyway, having the SIP support in ADF would be awesome.

[me-no-dev]

for actual streaming to a server, MJPEG should be encapsulated in some sort of video container in order to keep the timing between frames correct, else you will watch a video with speed comparable to the speed of the connection to your server :)

h263/h264 and other good video compressions work by having multiple frames at the same time and transmitting the differences (simplified). The ESP32 does not have enough memory or speed to acquire and keep a few frames in RGB format and then calculate and transmit their differences. Maybe some really low res thing can be done... but not 100% sure.

[me-no-dev]

@Niek honestly, I think that SIP should be it's own component. We will not use the compressions used by ADF, because 90% of SIPs out there use G711 audio compression (100% doable on ESP32).

[asmalldev]

Hi @me-no-dev are there any higher resolution cameras that may be better candidates than OV5640? I am looking to add more resolution and may have time to work on a driver :)

[me-no-dev]

@asmalldev honestly I am not sure if it makes sense to look at higher res cameras than 5MP. We do not have enough RAM available :) The sensor must support JPEG compression or there is no chance at all even for 3MP sensors :)

[asmalldev]

@me-no-dev thanks for the input :) If the limiting factor is the RAM space, would it be possible to interface with higher resolution cameras by adding more SPI/PSRAM to the hardware? I think this could be interesting for projects that need high resolution + low power but don't have frame rate requirements/ only need to take still images.

[me-no-dev]

You are limited to 4MB PSRAM access at a time, so that is the maximum.

[asmalldev]

Ah right, bank switching stuff is needed to access the higher memory. Thanks for the info, @me-no-dev
I think I will try writing a driver for OV5640 to get 5MP w/ JPEG compression. If you know of any other 5MP sensors that would be good candidates, I'd love to hear about them :)

[xxxajk]

I would be happier with any camera doing autofocus myself.
I only need VGA resolution anyway.
I have a bunch OV5648 I pulled from dead tablets, but it has a different connector.
I hope to be able to make my own board for it soon.
Another thought on using higher-res cameras, why not send the raw data and use javascript on the PC side to process it? :-)
Also, it seems that if the ESP is sending data there is corruption, and I get streaks in the image.
If I delay 500ms before each frame, these don't happen. 2FPS is probabbly OK for my project too.

[me-no-dev]

Autofocus is a feature of the camera itself. If it can do it and provides control, it can be added.
You can use "raw" data if you want (check the BMP sending in the README) but that means many times more data that needs to go through the network. Therefore it's not good for streaming.
Please show the corruption in an image and post your camera config.

[xxxajk]

My setup is:
CameraWebServer Arduino example.
ESP32-CAM board with OV2640 fixed focus camera.

I hope that the bare-bones SDK will work better, as it just could be an outdated version in Arduino-ESP32. I've recently downloaded the SDK, etc and will try the bare metal example as this issue is possibly already solved.

Project goal is to use a couple of these for vision. It will be nice to have them wireless v.s. adding more cabling.

The main processing and computation will be using OpenCV on an Android tablet or Linux PC. I haven't decided which yet.

[xxxajk]

Adding example screen captures from tablet.

[xxxajk]

Happens on the bare-metal variant as well, providing I turn off the debugging. Has to be a DMA conflict.

[me-no-dev]

What XCLK frequency are you setting?

[xxxajk]

20MHz.
Also tried 10, 12, and a few others. Result is the same.
From what I can tell, somehow, there is a race condition or DMA conflict.
I wonder if it might be a good idea to pause I2S between getting the FB and returning it back to the pool? According to the ESP32 RM, you shouldn't try to DMA and operate on a bank at the same time if you want 100% speed. Since I tried much lower speeds, that shouldn't be happening either way.
Checked the power supply and voltages on my 'scope, and they are solid. No power problems that could perturb the camera module, but...
Seems to happen only if there is WiFi traffic. I am about to try tests using high-speed serial. If that works, then I could use an ESP8266 to do the actual transmitting. Would be awesome if it worked out-of the box though.

[me-no-dev]

I have not seen any issues with 20MHz XCLK and OV2640. Have tested quite a few samples. Some were bad, but presented in another way. You can set the frame buffers to 1, which will start I2S and DMA when you request a frame and then stop it once done. That will limit the FPS by half though.

[xxxajk]

Not worried about FPS at this point.
I'll try that and let you know how it works out in a few minutes.

[xxxajk]

Still happens with a singular fb. :-( Any other good ideas?

[me-no-dev]

try another sensor?

[xxxajk]

Have 4 of them, tried 2, they both the same thing. :-/
I'll try the other two.

[xxxajk]

Tried 4 cams, on 4 boards, all showing same issue.

[jjsch-dev]

@tototrix If you are going to use the camera demo application, you need to install the idf together with the bin tools.
Espressif has a guide with the steps to install the idf on windows, linux and mac os.
Before compiling the application, you need to copy the esp32-camera driver to the idf component directory.

[tototrix]

@jjsch-dev Thank you. Regarding to ESP version, which one should i use ?

[me-no-dev]

@tototrix you need to use a supported ESP32-Camera board (there are a few out there). ESP8266 can not work with cameras.

[tototrix]

@tototrix you need to use a supported ESP32-Camera board (there are a few out there). ESP8266 can not work with cameras.

I have the ESP-EYE board wich work pretty well. For my needs i want to get 5 MP photos with the OV5640 sensor. I have the OV5640 waveshare board.

[Celppu]

Is ov5642 compatible? Can exposure be controlled from the library to increase framerate?

[jjsch-dev]

@me-no-dev I Download your new March 11, 2020 version of OV5640.

Below I show you the captures in JPG format in VGA size.

This is a capture in RGB565 format in QQVGA size, with problems in the PLL.

[me-no-dev]

@jjsch-dev thanks for testing! All is fixed and support is now in master :)
closing!!!

[tototrix]

Hi,
I'am sorry but i'am still stucked with compilation.

With ESP-IDF41

• I have ESP-IDF 4.1.
• The ESP-CAMERA library was copied to COMPONENTS folder.
• I downloaded your ESP32-CAM-DEMO into the EXAMPLES folder of the ESP-IDF folder.

First issue: the "CMakeLists.txt" file is missing from the project. I added it with the following content:

When i tried ty compile i have the following error:

So i decided to test with ESP-IDF40. My issues with ESP-IDF40:

First issue: the "CMakeLists.txt" file is missing from the project. I added it with the following content:

Second issue: the app_main function is missing

I added the file "CMakeLists.txt" into the main folder ...

..with the following content

This time the file "bitmap.h" is missing. I can find the file into the component folder (of the project) but i don't know how to solve the issue.

Thanks in advance for your help.
Best regards.

[me-no-dev]

something wrong with your project setup. Don't know what cam demo you are talking about... anyway, not an issue with the driver :)

[tototrix]

I speak about the following project:
https://github.com/jjsch-dev/esp32-cam-demo

Have you an other one ?

[me-no-dev]

maybe you should ask there :) please do not create issues just to ask for examples ;) this is an issue tracker for issues with the driver and not with user issues.

[jjsch-dev]

Hi @tototrix sorry for the late answer. Today I updated the esp32-cam-demo repository with the CMakeLists.txt files.
Until now I used make to compile in Ubuntu.

[misghna]

@me-no-dev well I got my board finally, do you mind if you can direct me to any doc or info on how I can add this driver to Arduino IDE (I see the repo [https://github.com/espressif/arduino-esp32/releases ] is last updated on Oct 2019) . Sorry for the hand holding here I am new to this ESP32 stuff :)

[me-no-dev]

@misghna if you are novice, you should wait for me to update the Arduino repo with the new driver. If you can not wait, https://github.com/espressif/esp32-arduino-lib-builder

[misghna]

novice

Okay, I will wait. I hope you will update it soon :)
Thanks man

[misghna]

@me-no-dev, hi I have played with the ov5640 this week but I see vertical lines in the images(still-JPEG picture),and the problem magnifies as the resolution increases. I have tried two cameras, but unfortunately, the result is the same.
Any idea why this is happening?
the attached pictures are for UXGA and QSXGA (camera is 120 degrees wide)

[me-no-dev]

5640 really do not like less than perfect light :D Drop the XCLK to 10/15 MHz and they will dissapear

[zell0323]

Hi, we have ported the OV5640 on our esp32-wrover-b.
But we are suffering from a lot of noise, which doesn't appear on OV5642 with the same board.
Any idea how to fix it? thanks!

[xxxajk]

Add 100uF tantalum caps on +5 and +3.3v, that solved it for me.

[NathanKuo0821]

Hi
I used the esp-eye + ov5640 to capture , i see the vertical lines and noise ,
Adjust the XCLK to 5/10/15/20 MHz has no effect.
Does anyone know how to solve it?

Extra info:
.xclk_freq_hz = 10000000,
.pixel_format = PIXFORMAT_JPEG,
.frame_size = FRAMESIZE_QSXGA,

Thanks .

[misghna]

Hi
I used the esp-eye + ov5640 to capture , i see the vertical lines and noise ,
Adjust the XCLK to 5/10/15/20 MHz has no effect.
Does anyone know how to solve it?

Extra info:
.xclk_freq_hz = 10000000,
.pixel_format = PIXFORMAT_JPEG,
.frame_size = FRAMESIZE_QSXGA,

Thanks .

I have lowered the clock Freq to 5MHz and the lines reduced considerably but not totally.

[me-no-dev]

In the web server example you can adjust quite a few things. Maybe try to adjust the gamma sliders slightly.

[NathanKuo0821]

@me-no-dev

Hi
Thanks for your reply.
Adjust the gamma value seems to have no effect, because the vertical lines and noise are still there.
There are some improvements in reducing the xClkto 5MHz, but the effect is limited.
Do you have any idea for this issue?

Nathan Kuo

[me-no-dev]

@NathanKuo0821 there is really nothing definitive that I can recommend. Do you have other OV5460 sensors there to try? I do not see that same effect on the sensors I currently have, but I did see them while developing the driver.

[NathanKuo0821]

@me-no-dev
Hi
Which OV5460 sensor are you using now? Can you share the appearance of the OV5640 sensor, specifications or web links?
Many thanks

Nathan Kuo

[me-no-dev]

I have a total of 3 sensors here. They look identical and came from a customer (no idea where they got them from). Sensors look just like any other OV2460 or OV3360 sensors that I have seen. Marked as 5460 on the flat flex cable

[misghna]

I have a total of 3 sensors here. They look identical and came from a customer (no idea where they got them from). Sensors look just like any other OV2460 or OV3360 sensors that I have seen. Marked as 5460 on the flat flex cable

@me-no-dev well I have tried with several sensors as well but the result is the same, so maybe its the board some capacitor value like they mentioned above? if you can share the board you are using I can try and compare my circuit with the one which is working and report back.
Thanks for your support.

[me-no-dev]

I am using ESP-EYEs, M5Stack cameras and a board that we use for camera development. I did not find a difference between the boards I personally have and we have poked at power before. Honestly I have no idea what is causing the artefacts.

[xxxajk]

Oscilloscope showed me that it was a power problem. It looked similar, with the stripes, but even worse.
Lots of noise on the LDO output sides from WiFi and during captures, which was solved for me by adding extra capacitors.
Perhaps the one that I have here only needed 3.3v and 5v, but it could be the other voltage (1.8v?) on yours. The ESP32 draws more power when WiFi is active, and causes just enough of a disturbance.
LDOs do not react quickly enough, that's why you add the capacitors. The MLCC capacitors are known to be pretty poor in this regard, so use tantalum, or, at least electrolytic. I avoid electrolytic when I can, because they are prone to other problems.