Using dynamic calibration tool underwater

[neilharrison]

Required Info
Camera Model	D435i
Firmware Version	5.14.0
Operating System & Version	Linux (Ubuntu 20.04)
Kernel Version (Linux Only)	(e.g. 4.14.13)
Platform	PC/Raspberry Pi

Issue Description

Hi, I am trying to calibrate a realsense D435 underwater using one of the intel tools. I have my realsense camera connected to a Raspberry Pi 4 in a waterproof housing.

I've installed the free version of the dynamic calibration tool on the raspberry pi and I'm able to run the calibration process with the target in air with no issues. When trying to calibrate underwater, the first stage of the calibration works (the part where you move the target around the screen to erase the boxes) but the second stage fails (where target is tracked in the images - the image on screen shows a green box around the target but it says that it cannot find the target in both images).

I realise that the free version of the tool only calibrates the extrinsics of the cameras and not the intrinsics which will be important to correct distortion underwater, for that I need to buy the OEM target and use the OEM version of the software. I have also tried the on-chip calibration using the realsense viewer which apparently calibrates the intrinsics but that doesn't seem to correct the distortion very well unfortunately.

Before I go ahead and buy the OEM target, do you know if the OEM version of the tool uses a similar process to track the target as the free one and will I end up with the same problem as using the free one?

Also, the OEM target is very large, do you know if it would be possible for me to make my own smaller version of the target to use with the OEM tool?

Thanks,
Neil

[MartyG-RealSense]

Hi @neilharrison Is it the Scale phase of the calibration after the blue squares have been removed that your Dynamic Calibration is failing on, please?

If it is, are you able to increase the level of light illumination around the target to test whether the problem is related to insufficient lighting in the scene?

The main features that the OEM version of the Dynamic Calibration software adds are calibration of intrinsics and the ability to perform calibration over a remote Linux server connection. If you have difficulties with calibration using the free version of the tool then I would expect similar issues to be experienced with the OEM version of the tool.

There is a small PDF document version of the calibration chessboard image available at the link below for use with single cameras. Page 1 of the document is printed off and then attached to a flat surface.

https://dev.intelrealsense.com/docs/dynamic-calibration-print-target

In regard to the OEM target board product, I have previously enquired about the possibility of creating a custom-made version of it and was advised by my Intel RealSense colleagues that it is not possible to do so due to the special materials that the board is constructed from.

A RealSense team member confirms at IntelRealSense/realsense-ros#1723 (comment) that calibration can be carried out underwater using the calibration tool. They tested in a large fish tank in a lab but said that if a larger underwater area such as a pool was being used then the OEM tool was the one to use.

The On-Chip tool calibrates both intrinsics and extrinsics, though you have to perform intrinsic and extrinsic calibration separately as it does not carry out both types of calibration at the same time.

[neilharrison]

Hi Marty, yeah it is the scale phase that I'm having issues with - I'm getting the green rectangle around the target but not the blue rectangles as it's moved around the view. I don't think the issue is to do with illumination, I was seeing the target in the image at a similar brightness to your image above. I also tried increasing the exposure in the camera settings and didn't see any change in results. My theory was that the distortion in the image prevented the targets from being matched in both the images but I don't really know.

Thanks for that link, it's useful to know someone has tried to calibrate these underwater!

About the custom target, I assume the target just has to be perfectly flat and rigid? Is there any other requirements, for example do you know if the target recognition is looking for a certain number of rows and columns on the checkerboard? There looks to be a feature in the centre of the target but apart from that it seems to be regular.

As for the On-Chip tool, do you know how to change whether the camera is calibrating the intrinsics or the extrinsics? When I ran it from the viewer I didn't seem to get any options.

Thanks,
Neil

[MartyG-RealSense]

The On-Chip calibration options to choose intrinsic or extrinsic calibration can be missing from the interface if the realsense-viewer version is newer than 2.49.0. The image below from the 2.49.0 viewer show the options that are meant to be displayed by the On-Chip interface. These options may be absent on viewer versions 2.50.0, 2.51.1 and 2.53.1. If this occurs on your computer then there is not a solution for accessing the absent options other than to install the 2.49.0 SDK and calibrate using the 2.49.0 realsense-viewer.

If you have access to a Windows computer then an easier method of dealing with this issue than installing the entire SDK on Linux is to download and run version 2.49.0 of the standalone Intel.RealSense.Viewer.exe Viewer from the 'Assets' file list of the 2.49.0 Releases page.

The standalone Viewer should be launched from the folder on the Windows computer that it was downloaded to and not with the Viewer launch shortcut icon on the Windows desktop.

https://github.com/IntelRealSense/librealsense/releases/tag/v2.49.0

Yes, the target should be flat and rigid.

As you are using a custom housing, you should ensure that the housing's outer window has an anti-reflective coating on both sides in order to avoid negative consequences for the image from causes such as the laser being reflected back, as described at #2566 (comment)

Section 2.4.1 of the programmer's version of the Dynamic Calibration user guide on pages 14-16 of the the PDF document discusses the processes that a targeted calibration is carrying out.

https://www.intel.com/content/www/us/en/support/articles/000026724/emerging-technologies/intel-realsense-technology.html

To quote the guide:

---

As part of the process, the application acquires images of the calibration target and feed into dynamic calibration algorithm.
The process is divided into two consecutive steps: rectification phase and scale phase. The rectification phase focuses on fixing rectification issues so the device stream is corrected with sufficient depth quality for the next phase to fix scaling issues. The process requires images from simultaneous depth, rectified left and right images at 1280x720 resolution.

---

Please confirm in the realsense-viewer side panel options that Infrared 2 (the right-side infrared stream) is available as an option, as there could be problems with calibration if the Infrared 2 option is not available on your computer.

If calibration with a target is not successful then you could try targetless calibration, as described in section 2.4.2 on page 17 of the guide.

[MartyG-RealSense]

Hi @neilharrison Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.