use Intel RealSense camera D435i as a YARP device #706

jimingda · 2024-06-04T10:06:01Z

jimingda
Jun 4, 2024

Hi,

I'm currently integrating the D435i camera with YARP, and I'm following the guidelines outlined in the "Configure YARP" section. However, I'm encountering a step where I need to enable specific YARP CMake flags, and I'm seeking clarification on how to enable the following YARP CMake flags:

ENABLE_yarpmod_RGBDSensorWrapper  
ENABLE_yarpmod_RGBDSensorClient  
ENABLE_yarpcar_mjpeg  
ENABLE_yarppm_depthimage_to_rgb

My current YARP version is 3.7.0. I appreciate any guidance you can provide on how to enable these flags correctly.

Answered by pattacini

Jun 6, 2024

Hi @jimingda

The failure is reported at this line:

[ERROR] |yarp.os.Property| cannot read from sensors/RealSense_conf.ini

This means that the configuration file is not found. The instructions do refer to robots that are shipped with that file integrated in the robot folder, like it happens for e.g., R1SN001, ergoCubSN001.

Therefore, as a quick test, you could:

Create the configuration file Realsense_conf.ini anew in the icub-head.
From the same directory, launch: yarpdev --from Realsense_conf.ini.

View full answer

pattacini · 2024-06-04T11:09:38Z

pattacini
Jun 4, 2024
Maintainer

Hi @jimingda

Did you build YARP via robotology-superbuild or as an isolated project?

8 replies

jimingda Jun 4, 2024
Author

Thank you @pattacini

I didn't build YARP as a isolated project. I think it's built within robotology-superbuild

traversaro Jun 4, 2024
Maintainer

Those options are already enabled by default if you build YARP in the robotology-superbuild, see https://github.com/robotology/robotology-superbuild/blob/da4b2d20b7ac08db956f9e8f9ec9f683b67eacb9/cmake/BuildYARP.cmake#L74 .

jimingda Jun 5, 2024
Author

Thanks @traversaro

Then i got error when command yarpdev --from sensors/RealSense_conf.ini

icub@icubsrv:~$ yarpdev --from sensors/RealSense_conf.ini
[ERROR] |yarp.os.Property| cannot read from sensors/RealSense_conf.ini
[INFO] |yarp.dev.Drivers| Welcome to yarpdev, a program to create YARP devices
[INFO] |yarp.dev.Drivers| To see the devices available, try:
[INFO] |yarp.dev.Drivers|    yarpdev --list
[INFO] |yarp.dev.Drivers| To create a device whose name you know, call yarpdev like this:
[INFO] |yarp.dev.Drivers|    yarpdev --device DEVICENAME --OPTION VALUE ...
[INFO] |yarp.dev.Drivers|    For example:
[INFO] |yarp.dev.Drivers|    yarpdev --device fakeFrameGrabber --width 32 --height 16 --name /grabber
[INFO] |yarp.dev.Drivers| You can always move options to a configuration file:
[INFO] |yarp.dev.Drivers|    yarpdev [--device DEVICENAME] --from CONFIG_FILENAME
[ERROR] |yarp.dev.Drivers| Unable to find --device option in file sensors/RealSense_conf.ini. Closing.

Shall i initial a RealSense_conf.ini file like below for D435i?

device       RGBDSensorWrapper
subdevice    realsense2withIMU
name         /depthCamera

[SETTINGS]
depthResolution (640 480)    #Note the parentesys
rgbResolution   (640 480)
framerate       30
enableEmitter   true
alignmentFrame  RGB

[HW_DESCRIPTION]
clipPlanes (0.2 10.0)

pattacini Jun 6, 2024
Maintainer

Hi @jimingda

The failure is reported at this line:

[ERROR] |yarp.os.Property| cannot read from sensors/RealSense_conf.ini

This means that the configuration file is not found. The instructions do refer to robots that are shipped with that file integrated in the robot folder, like it happens for e.g., R1SN001, ergoCubSN001.

Therefore, as a quick test, you could:

Create the configuration file Realsense_conf.ini anew in the icub-head.
From the same directory, launch: yarpdev --from Realsense_conf.ini.

Answer selected by pattacini

jimingda Jun 7, 2024
Author

Thanks @pattacini

After logging into icub-head, I added a RealSense_conf.ini file with the following configuration in the home directory:

device       RGBDSensorWrapper
subdevice    realsense2
name         /depthCamera

[SETTINGS]
depthResolution (640 480)    #Note the parentesys
rgbResolution   (640 480)
framerate       30
enableEmitter   true
alignmentFrame  RGB

[HW_DESCRIPTION]
clipPlanes (0.2 10.0)

However, upon attempting to run the command yarpdev --from RealSense_conf.ini, an error occurred indicating that the 'realsense2' plugin could not be found. The error log shows that the system failed to find the "realsense2" plugin and encountered difficulties opening the subdevice. As a result, the RGBDSensorWrapper driver was found but could not be opened. The specific error messages and warnings are as follows:

icub@icub-head:~$ yarpdev --from RealSense_conf.ini
[DEBUG] |yarp.dev.PolyDriver|RGBDSensorWrapper| Parameters are (HW_DESCRIPTION (clipPlanes (0.200000000000000011102 10.0))) (SETTINGS (depthResolution (640 480)) (rgbResolution (640 480)) (framerate 30) (enableEmitter true) (alignmentFrame RGB)) (device RGBDSensorWrapper) (from RealSense_conf.ini) (name "/depthCamera") (single_threaded 1) (subdevice realsense2)
[WARNING] |yarp.devices.RGBDSensorWrapper| The 'RGBDSensorWrapper' device is deprecated in favour of 'rgbdSensor_nws_yarp'.
[WARNING] |yarp.devices.RGBDSensorWrapper| The old device is no longer supported, and it will be deprecated in YARP 3.7 and removed in YARP 4.
[WARNING] |yarp.devices.RGBDSensorWrapper| Please update your scripts.
[INFO] |yarp.devices.RGBDSensorWrapper| Using default 'period' parameter of  0.03 s
[INFO] |yarp.devices.RGBDSensorWrapper| ROS configuration parameters are not set, skipping ROS topic initialization.
[INFO] |yarp.os.Port|/depthCamera/rpc:i| Port /depthCamera/rpc:i active at tcp://10.0.0.2:10042/
[INFO] |yarp.os.Port|/depthCamera/rgbImage:o| Port /depthCamera/rgbImage:o active at tcp://10.0.0.2:10043/
[INFO] |yarp.os.Port|/depthCamera/depthImage:o| Port /depthCamera/depthImage:o active at tcp://10.0.0.2:10044/
[DEBUG] |yarp.devices.RGBDSensorWrapper| Opening IRGBDSensor subdevice
[ERROR] |yarp.os.YarpPluginSettings| Cannot find "realsense2" plugin (not built in, and no .ini file found for it)Check that YARP_DATA_DIRS leads to at least one directory with plugins/realsense2.ini or share/yarp/plugins/realsense2.ini in it
[ERROR] |yarp.dev.PolyDriver|realsense2| Could not find device <realsense2>
[ERROR] |yarp.devices.RGBDSensorWrapper| Opening IRGBDSensor subdevice... FAILED
[ERROR] |yarp.devices.RGBDSensorWrapper| Error while opening subdevice
[ERROR] |yarp.dev.PolyDriver|RGBDSensorWrapper| Driver <RGBDSensorWrapper> was found but could not open
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ===============================================================
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| == Options checked by device:
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ==
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| device=RGBDSensorWrapper
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| id [RGBDSensorWrapper]
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper|     Id assigned to this device
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ==
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ===============================================================
[ERROR] |yarp.dev.Drivers|RGBDSensorWrapper| yarpdev: ***ERROR*** device not available.
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| Suggestions:
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| + Do "yarpdev --list" to see list of supported devices.

And the yarpdev --list also got error as:

icub@icub-head:~$ yarpdev --list
[INFO] |yarp.dev.Drivers| Here are devices listed for your system:
[ERROR] |yarp.os.YarpPluginSettings| Error while opening /usr/local/src/robot/robotology-superbuild/build/install/lib/yarp/yarp_fakeFrameGrabber.so:
  /usr/local/src/robot/robotology-superbuild/build/install/lib/yarp/yarp_fakeFrameGrabber.so: undefined symbol: test_grabber
[ERROR] |yarp.os.YarpPluginSettings| Error while opening /usr/local/src/robot/robotology-superbuild/build/install/lib/yarp/yarp_fakeFrameGrabber.so:
  /usr/local/src/robot/robotology-superbuild/build/install/lib/yarp/yarp_fakeFrameGrabber.so: undefined symbol: test_grabber
[WARNING] |yarp.dev.Drivers| Wrong library name for plugin test_grabber

yarpdev.txt

Shall i install the yarp-device-realsense2 on icub-head?

pattacini Jun 7, 2024
Maintainer

Hi @jimingda

Sorry, I was probably mistaken. The Realsense will be connected to a PC, I imagine, not to icub-head. So, please proceed with those steps on the PC where you were using the camera.

jimingda Jun 7, 2024
Author

Thank you @pattacini

I have added a RealSense_conf.ini file on my laptop, and now it seems to be working fine.
By the way, my camera is D435i. Should I use the parameter subdevice realsense2withIMU instead of subdevice realsense2 in the RealSense_conf.ini file?

icub@icubsrv:~/workspace$ yarpdev --from realsense_conf.ini 
[DEBUG] |yarp.dev.PolyDriver|RGBDSensorWrapper| Parameters are (HW_DESCRIPTION (clipPlanes (0.200000000000000011102 10.0))) (SETTINGS (depthResolution (640 480)) (rgbResolution (640 480)) (framerate 30) (enableEmitter true) (alignmentFrame RGB)) (device RGBDSensorWrapper) (from realsense_conf.ini) (name "/depthCamera") (single_threaded 1) (subdevice realsense2)
[WARNING] |yarp.devices.RGBDSensorWrapper| The 'RGBDSensorWrapper' device is deprecated in favour of 'rgbdSensor_nws_yarp'.
[WARNING] |yarp.devices.RGBDSensorWrapper| The old device is no longer supported, and it will be deprecated in YARP 3.7 and removed in YARP 4.
[WARNING] |yarp.devices.RGBDSensorWrapper| Please update your scripts.
[INFO] |yarp.devices.RGBDSensorWrapper| Using default 'period' parameter of  0.03 s
[INFO] |yarp.devices.RGBDSensorWrapper| ROS configuration parameters are not set, skipping ROS topic initialization.
[INFO] |yarp.os.Port|/depthCamera/rpc:i| Port /depthCamera/rpc:i active at tcp://10.0.0.1:10042/
[INFO] |yarp.os.Port|/depthCamera/rgbImage:o| Port /depthCamera/rgbImage:o active at tcp://10.0.0.1:10043/
[INFO] |yarp.os.Port|/depthCamera/depthImage:o| Port /depthCamera/depthImage:o active at tcp://10.0.0.1:10044/
[DEBUG] |yarp.devices.RGBDSensorWrapper| Opening IRGBDSensor subdevice
[DEBUG] |yarp.dev.PolyDriver|realsense2| Parameters are (HW_DESCRIPTION (clipPlanes (0.200000000000000011102 10.0))) (SETTINGS (depthResolution (640 480)) (rgbResolution (640 480)) (framerate 30) (enableEmitter true) (alignmentFrame RGB)) (device realsense2) (from realsense_conf.ini) (name "/depthCamera") (single_threaded 1) (subdevice realsense2)
[INFO] |yarp.device.realsense2| Sensor warm-up...
[INFO] |yarp.device.realsense2| Device ready!
[INFO] |yarp.device.realsense2| Device consists of 3 sensors. More infos using --verbose option
[INFO] |yarp.dev.PolyDriver|realsense2| Created device <realsense2>. See C++ class realsense2Driver for documentation.
[INFO] |yarp.dev.PolyDriver|RGBDSensorWrapper| Created wrapper <RGBDSensorWrapper>. See C++ class RGBDSensorWrapper for documentation.
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ===============================================================
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| == Options checked by device:
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ==
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| device=RGBDSensorWrapper
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| id [RGBDSensorWrapper]
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper|     Id assigned to this device
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ==
[DEBUG] |yarp.dev.Drivers|RGBDSensorWrapper| ===============================================================
[INFO] |yarp.os.Port|/depthCamera/quit| Port /depthCamera/quit active at tcp://10.0.0.1:10068/
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| device active in background...
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| device active in background...
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| device active in background...
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| device active in background...
[INFO] |yarp.dev.Drivers|RGBDSensorWrapper| device active in background...

pattacini Jun 7, 2024
Maintainer

I have added a RealSense_conf.ini file on my laptop, and now it seems to be working fine.

Great!

By the way, my camera is D435i. Should I use the parameter subdevice realsense2withIMU instead of subdevice realsense2 in the RealSense_conf.ini file?

Yep, indeed 👍🏻

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Robotology

use Intel RealSense camera D435i as a YARP device #706

{{title}}

Replies: 1 comment 8 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Robotology

use Intel RealSense camera D435i as a YARP device #706

jimingda Jun 4, 2024

Replies: 1 comment · 8 replies

pattacini Jun 4, 2024 Maintainer

jimingda Jun 4, 2024 Author

traversaro Jun 4, 2024 Maintainer

jimingda Jun 5, 2024 Author

pattacini Jun 6, 2024 Maintainer

jimingda Jun 7, 2024 Author

pattacini Jun 7, 2024 Maintainer

jimingda Jun 7, 2024 Author

pattacini Jun 7, 2024 Maintainer

jimingda
Jun 4, 2024

Replies: 1 comment 8 replies

pattacini
Jun 4, 2024
Maintainer

jimingda Jun 4, 2024
Author

traversaro Jun 4, 2024
Maintainer

jimingda Jun 5, 2024
Author

pattacini Jun 6, 2024
Maintainer

jimingda Jun 7, 2024
Author

pattacini Jun 7, 2024
Maintainer

jimingda Jun 7, 2024
Author

pattacini Jun 7, 2024
Maintainer