-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for OV5640 camera #3063
base: rolling
Are you sure you want to change the base?
Conversation
what is the impact on the RAM usage? |
@caco3 There's no impact on RAM usage at all. It works the same way as ov2640 where cropping is done on the camera module itself. |
Process:
However, further changes to the implementation are necessary, e.g. at the sharpness.... Ablauf:
Allerdings sind noch weitere Änderungen an der Implementierung notwendig, z.B. bei sharpness..... |
@SybexX May I know what you would like to change? |
I made a few quick adjustments, but I don't know if it works because I don't have an OV5640. |
@SybexX I've adjusted the sharpness handling to your version and it's working fine on my OV5640 system. I changed the sharpness range because the officially supported sharpness range is from -3 to +3. Your |
I specifically set the sharpnessLevel to 2. I know that the OV5640 can do more, but we should adapt everything to the OV2640, Sometimes less is more^^ |
@SybexX Thank you for reviewing. I've refactored my code based on your sample code but had to make a few changes to make it work with the OV5640.
My apologies, it is not clear to me that OV3660 (which is the other supported camera) behaves the same way as OV2640 and parts of the code are hitting the OV2640 registers directly (presumably the OV3660 has the same registers as OV2640?). Personally, I prefer to be more explicit in the code when it comes to handling hardware specific behaviour, like
Either way, I am not particular with how the code is written as long as it is well documented via comments or code itself so other developers understand the intention of the author. |
The registers don't matter, the properties of the camera are more important. |
That was an oversight on my part as well. We should probably test sharpness against OV3660 and update the code accordingly? |
If something is not available on a camera model, the esp32-camera library actually returns a -1 https://github.com/espressif/esp32-camera/blob/master/sensors/ov2640.c |
I propose this sharpness handling code: void CCamera::SetCamSharpness(bool _autoSharpnessEnabled, int _sharpnessLevel)
{
sensor_t *s = esp_camera_sensor_get();
if (s != NULL)
{
_sharpnessLevel = min(2, max(-2, _sharpnessLevel));
camera_sensor_info_t *sensor_info = esp_camera_sensor_get_info(&(s->id));
if (sensor_info != NULL)
{
if (sensor_info->model == CAMERA_OV5640 || sensor_info->model == CAMERA_OV3660)
{
if (_autoSharpnessEnabled)
{
// autoSharpness is not supported, default to zero
s->set_sharpness(s, 0);
}
else
{
s->set_sharpness(s, _sharpnessLevel);
}
}
else if (sensor_info->model == CAMERA_OV2640)
{
// The OV2640 does not officially support sharpness, so the detour is made with the ov2640_sharpness.cpp.
if (_autoSharpnessEnabled)
{
s->set_sharpness(s, 0);
ov2640_enable_auto_sharpness(s);
}
else
{
ov2640_set_sharpness(s, _sharpnessLevel);
}
}
}
}
else
{
LogFile.WriteToFile(ESP_LOG_ERROR, TAG, "SetCamSharpness, Failed to get Cam control structure");
}
}
|
@SybexX Looks like OV3660 |
or so, then the ESP is spared the second comparison ^^ void CCamera::SetCamSharpness(bool _autoSharpnessEnabled, int _sharpnessLevel)
} |
|
@SybexX I have updated the code to your suggested sharpness handling and also updated zoom handling code to better deal with OV3660 but I don't have an OV3660 so can't test. I have only tested with OV5640 so far. |
|
@SybexX The new I've also increased the OV5640 full frame size to match the datasheet. |
still necessary adjustment in the edit_reference.html:
still necessary adjustment in the edit_config_template.html:
|
@SybexX I'm not sure what else we can do to track down the root cause of your camera red blue colour swap.
I have shared my firmware here for you to test to help isolate the problem. Other than this, I'm not sure what else we can do to track this issue down, then the next step is to probably have a red blue colour swap workaround flag that can be controlled via the web interface? |
me too^^ I assume it's due to the camera's firmware, your number has an 8 and mine has a B. Since there are no differences when querying the version and revision, the only way to do this is via the settings page. maybe like this: AI-on-the-edge-device-ov5640_support_changes.zip |
@SybexX I've pushed your changes, however I did not comment out the code that limits the jpeg quality range. Was it commented out on purpose? if (CCstatus.CamSensor_id == OV5640_PID)
{
qual = min(63, max(18, qual)); // Limit quality from 18..63 (values lower than 20 tent to be unstable)
}
else
{
qual = min(63, max(8, qual)); // Limit quality from 8..63 (values lower than 8 tent to be unstable)
} |
We already query the validity of the value in MainFlowControl.cpp and ClassFlowTakeImage.cpp and my OV5640 works with a value of 8 without any problems(even in poor lighting conditions), so I commented it out.
|
@SybexX OK, I have updated the jpeg limits for our 2 variants of OV5640 and removed the OV2640 jpeg limit. Do you know whether OV3660 has a limit? I guess we just update the OV3660 limit when someone can actually test it? For now we just handle it the same as OV2640? |
So far I don't know anyone who has the OV3660, and if anyone does, to my knowledge there have never been any complaints about problems with the camera. |
@SybexX Is there anymore changes or testing that we want to do on this PR before we merge it in? |
Everything works as it should with my camera, so I don't have any other changes that need to be made. |
I think I have found the main problem of swapping the colors. |
I've tried it like this now: void CCamera::SetCamWindow(sensor_t *s, int frameSizeX, int frameSizeY, int xOffset, int yOffset, int xTotal, int yTotal, int xOutput, int yOutput)
{
if (CCstatus.CamSensor_id == OV2640_PID)
{
s->set_res_raw(s, 0, 0, 0, 0, xOffset, yOffset, xTotal, yTotal, xOutput, yOutput, false, false);
}
else
{
// for CAMERA_OV5640 and CAMERA_OV3660
bool scale = !(xOutput == xTotal && yOutput == yTotal);
bool binning = (xTotal >= (frameSizeX >> 1));
if (CCstatus.ImageVflip == true)
{
s->set_res_raw(s, xOffset, yOffset, xOffset + xTotal - 1, yOffset + yTotal - 1, 0, 0, frameSizeX, frameSizeY, xOutput, yOutput, scale, binning);
}
else
{
s->set_res_raw(s, xOffset, yOffset, xOffset + xTotal, yOffset + yTotal, 0, 0, frameSizeX, frameSizeY, xOutput, yOutput, scale, binning);
}
}
} Now is "xOffset + xTotal - 1" instead of "xOffset + xTotal + 1", because a strange line appeared in the image at "+" Remember that this is only for testing and you have to save first before the change is applied, since CFstatus.ImageVflip is set when creating the reference image and not CCstatus.ImageVflip. you then had to expand it like this: void CCamera::SetCamWindow(sensor_t *s, int frameSizeX, int frameSizeY, int xOffset, int yOffset, int xTotal, int yTotal, int xOutput, int yOutput, int imageVflip)
{
if (CCstatus.CamSensor_id == OV2640_PID)
{
s->set_res_raw(s, 0, 0, 0, 0, xOffset, yOffset, xTotal, yTotal, xOutput, yOutput, false, false);
}
else
{
// for CAMERA_OV5640 and CAMERA_OV3660
bool scale = !(xOutput == xTotal && yOutput == yTotal);
bool binning = (xTotal >= (frameSizeX >> 1));
if (imageVflip == true)
{
s->set_res_raw(s, xOffset, yOffset, xOffset + xTotal - 1, yOffset + yTotal - 1, 0, 0, frameSizeX, frameSizeY, xOutput, yOutput, scale, binning);
}
else
{
s->set_res_raw(s, xOffset, yOffset, xOffset + xTotal, yOffset + yTotal, 0, 0, frameSizeX, frameSizeY, xOutput, yOutput, scale, binning);
}
}
} |
@SybexX I have 2 different lots of OV5640 camera modules and both behave the same way as yours where vertical flip causes the image to turn red. I suggest we get rid of vertical flip altogether because vertical flip can be achieved by mirror + 180 degrees rotation. |
I would reluctantly waive Vflip because the Hmirror function then doesn't make much sense anymore. |
@SybexX I've removed the colour swap config and implemented the workaround based on vflip. I'm not sure why the build on github action is failing. |
https://github.com/jasaw/AI-on-the-edge-device/actions/runs/9218953628/job/25363354780 components/jomjol_controlcamera/ClassControllCamera.cpp:468:6: error: no declaration matches 'void CCamera::SetQualityZoomSize(int, framesize_t, bool, int, int, int)' |
in components/jomjol_controlcamera/ClassControllCamera.cpp |
Interesting PR 👍 Just a small side note: To my knowledge OV3660 is not supported up to now. It is not even compiled in the latest rolling. https://github.com/jomjol/AI-on-the-edge-device/blob/rolling/code%2Fsdkconfig.defaults#L137 |
@jasaw Can you try out whether the following changes eliminate the red coloring when you have vertical flip activated? change in ClassControllCamera.cpp; esp_err_t CCamera::setSensorDatenFromCCstatus(void)
{
sensor_t *s = esp_camera_sensor_get();
if (s != NULL)
{
s->set_framesize(s, CCstatus.ImageFrameSize);
s->set_quality(s, CCstatus.ImageQuality); // 0 - 63
s->set_brightness(s, CCstatus.ImageBrightness); // -2 to 2
s->set_contrast(s, CCstatus.ImageContrast); // -2 to 2
s->set_saturation(s, CCstatus.ImageSaturation); // -2 to 2
// s->set_sharpness(s, CCstatus.ImageSharpness); // auto-sharpness is not officially supported, default to 0
SetCamSharpness(CCstatus.ImageAutoSharpness, CCstatus.ImageSharpness);
s->set_denoise(s, CCstatus.ImageDenoiseLevel); // The OV2640 does not support it, OV3660 and OV5640 (0 to 8)
s->set_special_effect(s, CCstatus.ImageSpecialEffect); // 0 to 6 (0 - No Effect, 1 - Negative, 2 - Grayscale, 3 - Red Tint, 4 - Green Tint, 5 - Blue Tint, 6 - Sepia)
s->set_wb_mode(s, CCstatus.ImageWbMode); // 0 to 4 - if awb_gain enabled (0 - Auto, 1 - Sunny, 2 - Cloudy, 3 - Office, 4 - Home)
s->set_ae_level(s, CCstatus.ImageAeLevel); // -2 to 2
s->set_aec_value(s, CCstatus.ImageAecValue); // 0 to 1200
s->set_agc_gain(s, CCstatus.ImageAgcGain); // 0 to 30
s->set_gainceiling(s, CCstatus.ImageGainceiling); // Image gain (GAINCEILING_x2, x4, x8, x16, x32, x64 or x128)
s->set_lenc(s, CCstatus.ImageLenc); // 0 = disable , 1 = enable
s->set_gain_ctrl(s, CCstatus.ImageAgc); // 0 = disable , 1 = enable
s->set_exposure_ctrl(s, CCstatus.ImageAec); // 0 = disable , 1 = enable
s->set_hmirror(s, CCstatus.ImageHmirror); // 0 = disable , 1 = enable
s->set_vflip(s, CCstatus.ImageVflip); // 0 = disable , 1 = enable
s->set_aec2(s, CCstatus.ImageAec2); // 0 = disable , 1 = enable
s->set_bpc(s, CCstatus.ImageBpc); // 0 = disable , 1 = enable
s->set_wpc(s, CCstatus.ImageWpc); // 0 = disable , 1 = enable
s->set_raw_gma(s, CCstatus.ImageRawGma); // 0 = disable , 1 = enable
s->set_awb_gain(s, CCstatus.ImageAwbGain); // 0 = disable , 1 = enable
s->set_whitebal(s, CCstatus.ImageAwb); // 0 = disable , 1 = enable
s->set_dcw(s, CCstatus.ImageDcw); // 0 = disable , 1 = enable
TickType_t xDelay2 = 1000 / portTICK_PERIOD_MS;
vTaskDelay(xDelay2);
return ESP_OK;
}
else
{
return ESP_FAIL;
}
} change in MainFlowControl.cpp; esp_err_t setCFstatusToCam(void)
{
sensor_t *s = esp_camera_sensor_get();
if (s != NULL)
{
s->set_framesize(s, CFstatus.ImageFrameSize);
s->set_quality(s, CFstatus.ImageQuality); // 0 - 63
s->set_brightness(s, CFstatus.ImageBrightness); // -2 to 2
s->set_contrast(s, CFstatus.ImageContrast); // -2 to 2
s->set_saturation(s, CFstatus.ImageSaturation); // -2 to 2
// s->set_sharpness(s, CFstatus.ImageSharpness); // auto-sharpness is not officially supported, default to 0
Camera.SetCamSharpness(CFstatus.ImageAutoSharpness, CFstatus.ImageSharpness);
s->set_denoise(s, CFstatus.ImageDenoiseLevel); // The OV2640 does not support it, OV3660 and OV5640 (0 to 8)
s->set_special_effect(s, CFstatus.ImageSpecialEffect); // 0 to 6 (0 - No Effect, 1 - Negative, 2 - Grayscale, 3 - Red Tint, 4 - Green Tint, 5 - Blue Tint, 6 - Sepia)
s->set_wb_mode(s, CFstatus.ImageWbMode); // 0 to 4 - if awb_gain enabled (0 - Auto, 1 - Sunny, 2 - Cloudy, 3 - Office, 4 - Home)
s->set_ae_level(s, CFstatus.ImageAeLevel); // -2 to 2
s->set_aec_value(s, CFstatus.ImageAecValue); // 0 to 1200
s->set_agc_gain(s, CFstatus.ImageAgcGain); // 0 to 30
s->set_gainceiling(s, CFstatus.ImageGainceiling);
s->set_lenc(s, CFstatus.ImageLenc); // 0 = disable , 1 = enable
s->set_gain_ctrl(s, CFstatus.ImageAgc); // 0 = disable , 1 = enable
s->set_exposure_ctrl(s, CFstatus.ImageAec); // 0 = disable , 1 = enable
s->set_hmirror(s, CFstatus.ImageHmirror); // 0 = disable , 1 = enable
s->set_vflip(s, CFstatus.ImageVflip); // 0 = disable , 1 = enable
s->set_aec2(s, CFstatus.ImageAec2); // 0 = disable , 1 = enable
s->set_bpc(s, CFstatus.ImageBpc); // 0 = disable , 1 = enable
s->set_wpc(s, CFstatus.ImageWpc); // 0 = disable , 1 = enable
s->set_raw_gma(s, CFstatus.ImageRawGma); // 0 = disable , 1 = enable
s->set_awb_gain(s, CFstatus.ImageAwbGain); // 0 = disable , 1 = enable
s->set_whitebal(s, CFstatus.ImageAwb); // 0 = disable , 1 = enable
s->set_dcw(s, CFstatus.ImageDcw); // 0 = disable , 1 = enable
TickType_t xDelay2 = 100 / portTICK_PERIOD_MS;
vTaskDelay(xDelay2);
return ESP_OK;
}
else
{
return ESP_FAIL;
}
} |
@jasaw I changed something regarding gainceiling because it didn't work properly with the OV5640 |
The config order changes does not fix the vertical flip red colour problem for me. Both my normal OV5640 and OV5640 autofocus behave the same way. From I can tell, the autofocus version behaves exactly the same way as the normal OV5640 in every way, except it has the autofocus VCM circuit attached at the side.
Good catch. Gain ceiling is a little hard to test because it only kicks in after it has reached maximum exposure level and the datasheet is not exactly clear that the gain ceiling registers take real value rather than enum. The espressif library gets it wrong, so would be nice to upstream your fix. |
@jasaw It might be an advantage if you put something like this in the AI-on-the-edge-device/param-docs/parameter-pages/TakeImage/CamVflip.md: !!! On some OV5640 Cameras, the image becomes reddish when Vflip is used in conjunction with the zoom function! To avoid countless inquiries/reports. |
@SybexX I've updated the vflip param page and resolved merge conflict. |
I've been running my OV5640 for some time now. It is working well for me. Can we merge this in? |
Add support for OV5640 5MP camera. The camera has a resolution of 2560 x 1920 but the output will be scaled down to the configured resolution e.g. 640 x 480. When zoom is enabled, a 640 x 480 window is cropped from the full 2560 x 1920 frame, therefore giving double the magnification compared to OV2640.
This is a zoom comparison between OV5640 and OV2640.
OV5640:
![ov5640](https://private-user-images.githubusercontent.com/721280/328171528-dea3f8b2-83b4-412d-9046-9d4aa85037ad.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjE2OTUwNjcsIm5iZiI6MTcyMTY5NDc2NywicGF0aCI6Ii83MjEyODAvMzI4MTcxNTI4LWRlYTNmOGIyLTgzYjQtNDEyZC05MDQ2LTlkNGFhODUwMzdhZC5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzIzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcyM1QwMDMyNDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zODFhMDRhM2E3OGVhOGEyZGU5ZTRjMWNhMjU1NjVmNGZkMDg0MTM0NzQ2ODQ2NjhkMzJhN2E3OThhYzEzYmU4JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.zScKEJTybipGeN0oV5LWNZoBQ8BvizwMRi9Ih_hiZok)
OV2640:
![ov2640](https://private-user-images.githubusercontent.com/721280/328171552-1e2c3c6f-e532-48e5-96f1-be66a6b1fdf1.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjE2OTUwNjcsIm5iZiI6MTcyMTY5NDc2NywicGF0aCI6Ii83MjEyODAvMzI4MTcxNTUyLTFlMmMzYzZmLWU1MzItNDhlNS05NmYxLWJlNjZhNmIxZmRmMS5qcGc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzIzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcyM1QwMDMyNDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iYjhmMDVmYzE2OTMwMDNhOTE4OTkxM2NkYjQ2NzkxYWIzNWU1ZWQ3YWUxNjA0ZDVlZDZkZTA0ODUxY2ZjOTYyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.BNtTnsl40kByJSl7HFrjDRbyXaahNm3gEbfQvjFkdPk)