Random crash of D405 while capturing frames

[yasmineYK]

Hello everyone,

I'm encountering an issue with the API when I try to communicate with D405 cameras. I have 3 cameras plugged in and I try to capture 1 image from each one, successively.
The problem is that I have a random crash when I capture frames from the cameras with this error message:

"RealSense error calling rs2_pipeline_wait_for_frames(pipe:0x5624b13d7120):
    Frame didn't arrive within 15000"

When this happens, I sometime loose access to the camera until I unplug/plug it.
Worse, I can loose access to all USB ports (including mouse and keyboard).
This only happens with D405 cameras. I don't have this issue with D435.

Here is a script reproducing the bug:
rs-pcl-debug.txt

And here is the file containing this scripts' logs:

logs_pcl_debug.txt

[MartyG-RealSense]

Hi @yasmineYK Have you built librealsense from source code as an x86 (32-bit) version and are using a librealsense version older than 2.51.1, please? There was a D405-specific crash problem when using the camera with the 32-bit version of the librealsense library that was fixed in SDK version 2.51.1.

[yasmineYK]

Hi @MartyG-RealSense ,
I built it from source code but it's a amd64 version of the lib. I use librealsense-dev 2.53.1.
To be more precise, here's what I get when I display librealsense-dev version:

Package: librealsense2-dev
Version: 2.53.1-0~realsense0.8251
Priority: extra
Section: libdevel
Source: librealsense2
Maintainer: Raikhel, Evgeni <evgeni.raikhel@intel.com>
Installed-Size: 16,7 MB
Depends: librealsense2 (= 2.53.1-0~realsense0.8251)
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 7 075 kB
APT-Manual-Installed: yes
APT-Sources: https://librealsense.intel.com/Debian/apt-repo jammy/main amd64 Packages
Description: Intel(R) RealSense(tm) Camera Capture API - development files
 This package contains the development header files for
 for librealsense.

And I'm using D405 cameras with firmware version 05.14.00.

Another thing: when I use realsense-viewer or rs-multicam, I don't have any problem. I guess I am doing something wrong in my script, but I don't get it at all (I followed the PCL example given in source code)

[MartyG-RealSense]

There is not an obvious reason in the script why the D405 in particular would be having an issue. In terms of hardware, the main differences between D435 and D405 are:

1. D405 does not have an IR pattern projector, though this should not prevent streaming from occurring. I can foresee that this may cause problems if the camera was being used in a dark location as it would have no light output to illuminate the scene.

2. D405 does not have a separate RGB color sensor like the D435 does and instead obtains RGB from the depth sensor. This is not usually a problem though so long as a script does not have code that checks for the presence of an RGB sensor.

3. D405's ideal depth sensing range is 7 cm to 50 cm (0.5 m), as it is a model specializing in short range, high quality depth. So depth information of objects / surfaces further away than around a meter will not be rendered.

I am not aware of a previous RealSense case in which D405 has been used with PCL, so it is possible that there is a previously undiscovered D405-specific PCL issue.

Have you tried removing the for(int) frame skip mechanism to see whether it might be causing a conflict?

        // Get frame
        rs2::frameset frameset;
   
  for (int i = 0; i < 50; i++)
        {

            frameset = cameras[device.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER)]->get_frames(); // We drop several frames for auto-exposure
            // D405 will often timeout at this step for no apparent reasons.
 }
    }

So the code would look like this:

        // Get frame
        rs2::frameset frameset;

        frameset = cameras[device.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER)]->get_frames(); 

[yasmineYK]

You're right for the IR signal. And actually, in our script, we check if the camera handles IR signal before enabling it.

• In our case, we are using the cameras in a lighted room so we don't have this darknes problem.

• hmmm , I just tried to remove the loop as you told me, but it didn't change anything...

Could it be a configuration problem when I use D405? In wrappers/opencv/depth-filter I have a JSON file containing camera settings. Maybe some parameters are adapted to D435 and not D405...?

[MartyG-RealSense]

A script run on a D405 should in theory behave the same as being run a D435 if the following components are not being accessed.

• RGB sensor
• Projector
• Inter Cam Sync Mode hardware sync

You could test for whether parameters in a D435 json are not supported by D405 if you export a json from the RealSense Viewer whilst only the D405 is connected and then test loading that json with your program. The json export option is near the top of the Viewer's options side-panel.

[yasmineYK]

Hi @MartyG-RealSense ,

We could find out where was the problem! It wasn't the configuration file (we don't read it in our script).
But in our script, we called enable_stream this way _config.enable_stream(RS2_STREAM_INFRARED, 1280, 720);.

And once we had our frameset, we used to extract from it a video_frame using frameset.get_infrared_frame(0);, which is inappropriate since D405 don't have IR sensor.
This function seems to be able to send back an image anyway but it looks like it doesn't always work (it works 50% of the time), so this is another source of bug.

The reason for all this is that we started working with D435 cameras first. We then added D405 cameras to our scope.

Thanks for everything, you've been a great help!

[MartyG-RealSense]

You are very welcome!

The D405 has left and right infrared sensors. It is a separate RGB sensor that it lacks. The D405 instead obtains RGB from the depth sensor (the left infrared sensor, specifically).

An index number (1 or 2) is used with the left and right infrared sensors when using both of the sensors, so that the program knows which of the sensors to access. If only the left infrared sensor (1) is being used in a program then it is not necessary to use an index number.

[yasmineYK]

Thanks for everything!
I think I can close the discussion now, thanks to you.
Have a nice day!

[MartyG-RealSense]

You are very welcome :)