When two cameras are used, there will be a discrepancy between the image of one camera and the separation of the object.

[lopesnb]

Required Info
Camera Model	D435
Firmware Version	05.12.07.150
Operating System & Version	Win10
Kernel Version (Linux Only)
Platform	PC
SDK Version	2..
Language	C++/opencv
Segment	others

Hai!
I modified exaple's multicam and algin-advance programs to create a program that separates objects by depth. There is no problem when using one camera. When two cameras are used, there will be a discrepancy between the image of one camera and the separation of the object. Do you have any countermeasures?

source code
multicam

[MartyG-RealSense]

Hi @lopesnb The only other case I know of regarding performing depth-color alignment with multicam code is at #1833

May I ask whether changing line 56 to use the default depth scale of 0.001 for the 400 Series cameras (except the D405 model) as a hard-coded value instead of retrieving the depth scale from the camera corrects the alignment issue with the second camera?

depth_scale = 0.001f;

[lopesnb]

Hi! MartyG-RealSense

Thank You!

I changed it, but it doesn't change. Originally, the value read by both cameras was 0.00100000005.

[MartyG-RealSense]

Could you provide further information please about what the 'discrepancy' is (which part of the image you think is wrong and why).

[lopesnb]

Hi! MartyG-RealSense
In the camera image on the left, the line separated by the object and the depth is shifted in the X direction

[MartyG-RealSense]

Does this offset still occur if you unplug the camera with the brown object so that only the camera with the offset is displayed when you run the program? If it does then it may be worth performing a recalibration of the camera to eliminate the possibility that the offset is caused by a mis-calibration in that particular camera.

[lopesnb]

When only one camera is connected, no offset will occur in the image.

[MartyG-RealSense]

Please open the Windows Task Manager interface when using two cameras by using the Ctrl-Alt-Delete keyboard shortcut and then go to the 'Performance' category. In the CPU information display, does it show a very high percentage value when two cameras are being accessed? If it does then this could indicate that the computer is struggling to handle alignment of two cameras simultaneously.

Even if the two cameras work normally in the RealSense Viewer, alignment is a processing intensive operation that is twice the normal burden on the CPU in this particular case with two alignments happening at the same time.

[lopesnb]

Hi! MartyG-RealSense
Thank you. I will try tomorrow and report the results.

[lopesnb]

Hi! MartyG-RealSense

I started the task manager.
When the phenomenon occurred, the CPU was 100%.
When I changed the OpenMP setting to no, the CPU became 50%.
However, the phenomenon does not change.
The first recognized D435 will not be offset until it recognizes the second D435, but it will be offset when it recognizes the second.

[MartyG-RealSense]

Hi @lopesnb I see that you closed this case. Can you confirm if you no longer require assistance, please? Thanks!

[lopesnb]

I accidentally closed it. I need support.

[MartyG-RealSense]

When BUILD_WITH_OPENMP is set to true, it allows the SDK to use multiple cores of the CPU to process depth-color alignment, with the disadvantage that it can increase the percentage of the CPU that is utilized.

Fluorescent lights such as the ceiling strip lights in your pictures can cause interference to the images that are output by 400 Series cameras. As your results are correct when only one camera is used though, this suggests that the ceiling lights are not the cause of your particular problem.

Next, could you try covering over the projector of the other camera to eliminate the possibility that the light emissions from its projector are negatively affecting the view of the second camera that has the offset problem. Normally, multiple 400 Series cameras do not interfere with each other. On D435 the projector is the large circle with a green center on the front of the camera.

While standing behind the camera, please try putting your finger over this green sensor on the camera that is looking at the brown object to block its emissions and so see whether the offset on the other camera image disappears.

[lopesnb]

Hi! MartyG-RealSense

Thank You!

The two D435s are 90 degrees in different directions.
I try to cover the projector and stop the light emission.

[lopesnb]

I covered the projector and stopped the light emission. But there was no change.

I tried to find out where in the program the offset occurs.
The offset occurs in the program's aligin.process ().

auto processed = aligin.process (fs);

① No1 No offset
② No2 No offset
③ No1 offset occurs

??????????????????????????????????????????????????????????????????????????????????
In this case, does the program need as many realsense numbers (2) as aline?
??????????????????????????????????????????????????????????????????????????????????

[MartyG-RealSense]

I note that on line 78 of your script, you are nesting the align operation within the instruction if (pipe.try_wait_for_frames(&fs))

wait_for_frames() is not recommended by Intel to be used in multiple camera scripts. poll_for_frames() is the appropriate instruction for muticam scripts such as rs-multicam. A RealSense team member provides advice about this at #2422 (comment)

I see that you originally tried poll_for_frames on line 77 and commented the line out. Is this because you preferred try_wait_for_frames because it only returns a true / false status instead of exiting with an error if a problem occurs? If an exception error message can be produced along with an exit when using poll_for_frames() then adding an error try-catch mechanism may allow the exception error to be registered but allow the program to continue instead of exiting.

[lopesnb]

Hi!　MartyG-RealSense

thank you for the advice. Actually, I didn't know what was different, so I tried various things.

[lopesnb]

Hi!　MartyG-RealSense

thank you for the advice.
The offset issue has been resolved.
I was wrong with the program.
When importing an image by leaving a profile separately for RealSense
It was solved by importing scale, align_to, align for each profile.

multicam_non_offset_error

[MartyG-RealSense]

It's excellent to hear that you achieved a solution. Thanks very much for the update and for sharing your solution with the RealSense community!