Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OV9782 camera: after being ISP scaled it's not divisible by 2 anymore #961

Open
ostepan2006 opened this issue Mar 7, 2023 · 20 comments
Open

OV9782 camera: after being ISP scaled it's not divisible by 2 anymore #961

ostepan2006 opened this issue Mar 7, 2023 · 20 comments

Comments

@ostepan2006
Copy link

Hello. I had been trying to calibrate my OAK-FFC-3P device. And i got an error (see print screen).

image

If i replace OAK-FFC-3P by OAK-D and run camera_preview.py I got another error : File "camera_preview.py", line 36, in
device.startPipeline(pipeline)
RuntimeError: Camera(4) - 'video' width or height (-1, -1) must be bigger than (32, 32).

I have tried cam_test.py. I got this output
image
Luxonis-Erik posted this message:
https://discord.com/channels/790680891252932659/924798475030392922/1082672913032949921

Please help me calibrate my OAK-FFC-3P device
@Erol444
Copy link
Member

Erol444 commented Mar 7, 2023

Hi @ostepan2006 ,
The culprit of this error is here; so these lines:

        if not self.args.disableRgb:
            rgb_cam = pipeline.createColorCamera()
            rgb_cam.setResolution(
                dai.ColorCameraProperties.SensorResolution.THE_4_K)
            rgb_cam.setInterleaved(False)
            rgb_cam.setBoardSocket(dai.CameraBoardSocket.RGB)
            rgb_cam.setIspScale(1, 3)
            rgb_cam.initialControl.setManualFocus(self.focus_value)
            rgb_cam.setFps(self.args.fps)

So it tries to use 4K resolution, fails as it's OV9782, then downscales the (default) 800P resolution by 1/3 => 427x267, which isn't supported by opencv (nv12->rgb). As a workaround, you can just remove the rgb_cam.setIspScale(1, 3) and it should work, but @njezersek we should update this so it works for users, just as we did for factory calib. Thoughts?
Thanks, Erik

@njezersek
Copy link
Contributor

@ostepan2006 please try switching to modular-calibration branch. This branch has the latest version of the calibration script and I think the issue is already fixed there.

@ostepan2006
Copy link
Author

ostepan2006 commented Mar 9, 2023 •
edited
Loading

Hi @ostepan2006 , The culprit of this error is here; so these lines:

        if not self.args.disableRgb:
            rgb_cam = pipeline.createColorCamera()
            rgb_cam.setResolution(
                dai.ColorCameraProperties.SensorResolution.THE_4_K)
            rgb_cam.setInterleaved(False)
            rgb_cam.setBoardSocket(dai.CameraBoardSocket.RGB)
            rgb_cam.setIspScale(1, 3)
            rgb_cam.initialControl.setManualFocus(self.focus_value)
            rgb_cam.setFps(self.args.fps)

So it tries to use 4K resolution, fails as it's OV9782, then downscales the (default) 800P resolution by 1/3 => 427x267, which isn't supported by opencv (nv12->rgb). As a workaround, you can just remove the rgb_cam.setIspScale(1, 3) and it should work, but @njezersek we should update this so it works for users, just as we did for factory calib. Thoughts? Thanks, Erik

Thanks for your reply. After doing that i got another error.
image

@ostepan2006
Copy link
Author

ostepan2006 commented Mar 9, 2023 •
edited
Loading

@ostepan2006 please try switching to modular-calibration branch. This branch has the latest version of the calibration script and I think the issue is already fixed there.

Thanks for your reply. Should I delete main branch Depthai and clone modular calibration branch? Or it might be possible use only calibrate.py script from modular calibration branch?

@Erol444
Copy link
Member

Erol444 commented Mar 9, 2023

Hi @ostepan2006 best to try the modular-calibration branch.

@ostepan2006
Copy link
Author

ostepan2006 commented Mar 10, 2023 •
edited
Loading

Hi @ostepan2006 best to try the modular-calibration branch.

I had swithed to modular-calibration branch.
After running command python calibrate.py -s 2.5 -db -brd ACME01.json -ih i got this output (see screenshot)
image

I had used this board configuration (see screenshot):
image
Please , give me a tip how can i calibrate my OAK-FFC-3P device.

@njezersek
Copy link
Contributor

@ostepan2006 you have to specify the cameras you have connected. I suggest you modify this file to fit your needs.

@ostepan2006
Copy link
Author

ostepan2006 commented Mar 10, 2023 •
edited
Loading

@ostepan2006 you have to specify the cameras you have connected. I suggest you modify this file to fit your needs.

I have made my board configuration from your documentation (see screenshot below)
image
If my board configuration is wrong give me please a tip how board configuration have to be for OAK-FFC-3P device.
I have two Global Shutter Grayscale OV9282 cameras and one Global Shutter Color OV9782 camera in my OAK-FFC-3P device.
Thanks

@njezersek
Copy link
Contributor

@ostepan2006 I see the link I wanted to post in the previous comment didn't display correctly. Please use this file for reference and just change the extrinsics and fovs of the cameras.

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

@ostepan2006
Copy link
Author

@ostepan2006 I see the link I wanted to post in the previous comment didn't display correctly. Please use this file for reference and just change the extrinsics and fovs of the cameras.

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 3.75,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

Am i right in thinking that extrinsics should be filled before calibration? I thought that extrinsics will be calculated during calibration.

@njezersek
Copy link
Contributor

These are so called spec extrinsics and are sometimes more accurate than the calculated ones. For example the StereoDepth node uses spec extrinsics to convert disparity to depth by default (you can change this behaviour with setDisparityToDepthUseSpecTranslation but you need to use the latest develop of DepthAI). That's why you need to provide accurate manual measurements as well.

@ostepan2006
Copy link
Author

These are so called spec extrinsics and are sometimes more accurate than the calculated ones. For example the StereoDepth node uses spec extrinsics to convert disparity to depth by default (you can change this behaviour with setDisparityToDepthUseSpecTranslation but you need to use the latest develop of DepthAI). That's why you need to provide accurate manual measurements as well.

As I understand spec extrinsics is not mundatory to do calibration of OAK-FFC-3P device. Right? Could I live rotation fields empty (equal zero) in the board_config file? If not do you have more detailed explanation of the procedure of formation of the board_config file? In particalar it is iteresting how can I calculate all extrinsics. Thanks

@njezersek
Copy link
Contributor

You can set all rotation fields to zero. I'm not sure what happens if you leave them empty. The camera without the extrinsics field set is considered to be at (0,0,0) with rotation (0,0,0). Extirnsics of other cameras must be specified relative to a camera specified in the to_cam field.

I assume your cameras are placed in the following way:

(RIGHT)------(CENTER)-------(LEFT)

and the distance between left and right camera is 5.2 cm and the distance between center and left camera is 2.6 cm.

In that case, the board config file should look something like that:

{
    "board_config":
    {
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -5.2,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 2.6,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }

@ostepan2006
Copy link
Author

Hi @njezersek . I have created board_config.json as you had recommended. The file board_config.json is shown bellow. And I had run command python calibrate.py -s 2.5 -db -brd board_config.json -ih . `
Calibration procedure had been performing (a file calibration_output.txt is attached). But there was no calibration data on the device afterwards calibration procedure. Concretely I had had that error at the end of calibration procedure -

Device closed in exception..
'name'
Traceback (most recent call last):
  File "calibrate.py", line 925, in calibrate
    calibration_handler.setBoardInfo(self.board_config['name'], self.board_config['revision'])
KeyError: 'name'

What is a problem I had while calibration procedure? Please give me a help. Thanks

board_config.json:

{
    "board_config":
    {
        "cameras":{
            "CAM_A": {
                "name": "rgb",
                "hfov": 68.7938,
                "type": "color"
            },
            "CAM_B": {
                "name": "left",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_C",
                    "specTranslation": {
                        "x": -5.2,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            },
            "CAM_C": {
                "name": "right",
                "hfov": 71.86,
                "type": "mono",
                "extrinsics": {
                    "to_cam": "CAM_A",
                    "specTranslation": {
                        "x": 2.6,
                        "y": 0,
                        "z": 0
                    },
                    "rotation":{
                        "r": 0,
                        "p": 0,
                        "y": 0
                    }
                }
            }
        },
        "stereo_config":{
            "left_cam": "CAM_B",
            "right_cam": "CAM_C"
        }
    }
}

calibration_output.txt

@Erol444
Copy link
Member

Erol444 commented Apr 12, 2023

@ostepan2006 your json needs these 2 fields as well inside board_config: name and revision, just like in json above:

{
    "board_config":
    {
        "name": "OAK-D-PRO",
        "revision": "R3M1E3",
        "cameras":{
...

@ostepan2006
Copy link
Author

Thanks. I will fix my config file and redo calibration.

@ostepan2006
Copy link
Author

@Erol444 The previous error have not arised while calibration. But I got new error:

Reprojection error threshold -> 3
rgb Reprojection Error: 55.234071
800
Reprojection error threshold -> 1.1111111111111112
left Reprojection Error: 55.754000
800
Reprojection error threshold -> 1.1111111111111112
right Reprojection Error: 55.477004
['high Reprojection Error', 'high Reprojection Error', 'Epiploar validation failed between left and right', 'high Reprojection Error', 'Epiploar validation failed between right and rgb']

I think that there is a mismatch between cameras placement and info in configuration file. Do you have tutorial about:
how do right cameras placement on the tripod relatively each other;
how do right cameras placement on the tripod relatively up and down.

@Erol444
Copy link
Member

Erol444 commented Apr 12, 2023

@ostepan2006 It could be the camera placement / configuration file. What's your current setup? Could you maybe take a photo of all 3 cameras, and draw the distances between them?
Thanks, Erik

@ostepan2006
Copy link
Author

ostepan2006 commented Apr 12, 2023 •
edited
Loading

@Erol444

Thanks for the quick reply. The image of my installation is below. As you can see a distance between neighbor cameras is about 26 mm. All cameras have rotated on 180 degrees. The color camera is in between grayscale ones. Maybe I have mismatch between the grayscale cameras in the config file?
Cameras_placement

@Erol444
Copy link
Member

Erol444 commented Jun 9, 2023

@ostepan2006 based on the error it does look like the config file (json) is incorrect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@njezersek @Erol444 @ostepan2006