Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so failed to be imported while being present on the system #1144

Open
squizz617 opened this issue Jul 20, 2023 · 27 comments
Labels
bug Something isn't working more-information-needed Further information is required

Comments

@squizz617
Copy link
Contributor

Bug report

Required Info:

  • Operating System: Ubuntu 22.04
  • Installation type: Tried both binary and source installation
  • Version or commit hash:
    • bin: ros-humble-rclpy/jammy,now 3.3.8-2jammy.20230623.051015 amd64 [installed,automatic]
    • src: 76212669
  • DDS implementation: Fast-DDS (default)
  • Client library (if applicable): rclpy

Steps to reproduce issue

  • Binary installation:
$ source /opt/ros/humble/setup.sh
$ python3 -c "import rclpy"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/__init__.py", line 49, in <module>
    from rclpy.signals import install_signal_handlers
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/signals.py", line 15, in <module>
    from rclpy.exceptions import InvalidHandle
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/exceptions.py", line 15, in <module>
    from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy
  File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/impl/implementation_singleton.py", line 32, in <module>
    rclpy_implementation = import_c_library('._rclpy_pybind11', package)
  File "/opt/ros/humble/lib/python3.10/site-packages/rpyutils/import_c_library.py", line 39, in import_c_library
    return importlib.import_module(name, package=package)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: librcl_action.so: cannot open shared object file: No such file or directory
The C extension '/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so' failed to be imported while being present on the system. Please refer to 'https://docs.ros.org/en/{distro}/Guides/Installation-Troubleshooting.html#import-failing-even-with-library-present-on-the-system' for possible solutions

$ file /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so
/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=7a13e46bccf18deb10a3cfdb76b436aa3856fdda, stripped
  • Source installation:
$ source ~/ros2_humble/install/setup.sh
$ python3 -c "import rclpy"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/__init__.py", line 49, in <module>
    from rclpy.signals import install_signal_handlers
  File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/signals.py", line 15, in <module>
    from rclpy.exceptions import InvalidHandle
  File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/exceptions.py", line 15, in <module>
    from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy
  File "/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/impl/implementation_singleton.py", line 32, in <module>
    rclpy_implementation = import_c_library('._rclpy_pybind11', package)
  File "/home/seulbae/ros2_humble/build/rpyutils/rpyutils/import_c_library.py", line 39, in import_c_library
    return importlib.import_module(name, package=package)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: librcl_action.so: cannot open shared object file: No such file or directory
The C extension '/home/seulbae/ros2_humble/install/rclpy/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so' failed to be imported while being present on the system. Please refer to 'https://docs.ros.org/en/{distro}/Guides/Installation-Troubleshooting.html#import-failing-even-with-library-present-on-the-system' for possible solutions

$ file /home/seulbae/ros2_humble/build/rclpy/test_rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so
/home/seulbae/ros2_humble/build/rclpy/test_rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=521156ac1b623401f2b012c6dcece989048e6273, not stripped

Expected behavior

rclpy successfully imported.

Actual behavior

Import failure.

Additional information

I'm not sure what's causing this issue. A ROS 2 scripts on my machine that worked before suddenly stopped working, and it turned out to be a import failure. Also, the trobleshooting guide in the error message seems to be only for Windows.
Thank you.

@mjcarroll
Copy link
Member

What is the output of python3 -VV ?

@squizz617
Copy link
Contributor Author

$ python3 -VV
Python 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]

@mjcarroll
Copy link
Member

Ah, I didn't read closely enough, this is likely the culprit:

ImportError: librcl_action.so: cannot open shared object file: No such file or directory

Do you have ros-humble-rcl-action installed? You should also have the library located at /opt/ros/humble/librcl_action.so from the binaries.

@clalancette
Copy link
Contributor

Further, what does your environment look like?

$ printenv | grep -i ros

@squizz617
Copy link
Contributor Author

I have ros-humble-rcl-action installed.

$ apt list | grep humble-rcl-action                                                                                           

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

ros-humble-rcl-action-dbgsym/jammy 5.3.3-1jammy.20230623.045652 amd64
ros-humble-rcl-action/jammy,now 5.3.3-1jammy.20230623.045652 amd64 [installed,automatic]

Env when using the binary installation:

$ printenv | grep -i ros          
PATH=/opt/ros/humble/bin:/usr/local/texlive/2022/bin/x86_64-linux:/home/seulbae/.local/bin:/usr/local/texlive/2022/bin/x86_64-linux:/home/seulbae/.local/bin:/opt/gcc-arm-none-eabi-9-2020-q2-update/bin:/home/seulbae/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin
AMENT_PREFIX_PATH=/opt/ros/humble
PYTHONPATH=/opt/ros/humble/lib/python3.10/site-packages:/opt/ros/humble/local/lib/python3.10/dist-packages
LD_LIBRARY_PATH=/opt/ros/humble/opt/rviz_ogre_vendor/lib:/opt/ros/humble/lib/x86_64-linux-gnu:/opt/ros/humble/lib
ROS_DISTRO=humble
ROS_LOCALHOST_ONLY=0
ROS_PYTHON_VERSION=3
ROS_VERSION=2

@squizz617
Copy link
Contributor Author

$ file /opt/ros/humble/lib/librcl_action.so 
/opt/ros/humble/lib/librcl_action.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=e1d3f18a4b279ed1c5c6aca4dd4332e5fd3f06e5, stripped

@mjcarroll
Copy link
Member

I'm not seeing anything obvious. Can you also run ldd /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so

@squizz617
Copy link
Contributor Author

seulbae@xps13 ~   [23-07-27 13:37:31]
$ source /opt/ros/humble/setup.zsh                                                                                
seulbae@xps13 ~   [23-07-27 13:37:32]
$ ldd /opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/_rclpy_pybind11.cpython-310-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007fff79b99000)
	librcl_action.so => /opt/ros/humble/lib/librcl_action.so (0x00007f7f3459f000)
	librcl_lifecycle.so => /opt/ros/humble/lib/librcl_lifecycle.so (0x00007f7f34595000)
	libpython3.10.so.1.0 => /lib/x86_64-linux-gnu/libpython3.10.so.1.0 (0x00007f7f33e00000)
	librcl.so => /opt/ros/humble/lib/librcl.so (0x00007f7f34526000)
	librcl_yaml_param_parser.so => /opt/ros/humble/lib/librcl_yaml_param_parser.so (0x00007f7f3451a000)
	librmw_implementation.so => /opt/ros/humble/lib/librmw_implementation.so (0x00007f7f3450b000)
	librcl_logging_interface.so => /opt/ros/humble/lib/librcl_logging_interface.so (0x00007f7f34506000)
	librmw.so => /opt/ros/humble/lib/librmw.so (0x00007f7f344fa000)
	liblifecycle_msgs__rosidl_typesupport_c.so => /opt/ros/humble/lib/liblifecycle_msgs__rosidl_typesupport_c.so (0x00007f7f344f3000)
	librcutils.so => /opt/ros/humble/lib/librcutils.so (0x00007f7f344db000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f7f33a00000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7f344b9000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7f33600000)
	libtracetools.so => /opt/ros/humble/lib/libtracetools.so (0x00007f7f344b4000)
	liblifecycle_msgs__rosidl_generator_c.so => /opt/ros/humble/lib/liblifecycle_msgs__rosidl_generator_c.so (0x00007f7f344a4000)
	librosidl_runtime_c.so => /opt/ros/humble/lib/librosidl_runtime_c.so (0x00007f7f34499000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f7f34468000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7f3444a000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7f33d19000)
	librcl_logging_spdlog.so => /opt/ros/humble/lib/librcl_logging_spdlog.so (0x00007f7f34443000)
	librcl_interfaces__rosidl_typesupport_c.so => /opt/ros/humble/lib/librcl_interfaces__rosidl_typesupport_c.so (0x00007f7f3443a000)
	librcl_interfaces__rosidl_generator_c.so => /opt/ros/humble/lib/librcl_interfaces__rosidl_generator_c.so (0x00007f7f3441e000)
	libyaml.so => /opt/ros/humble/lib/libyaml.so (0x00007f7f343fc000)
	libament_index_cpp.so => /opt/ros/humble/lib/libament_index_cpp.so (0x00007f7f343ef000)
	librcpputils.so => /opt/ros/humble/lib/librcpputils.so (0x00007f7f343e1000)
	librosidl_typesupport_c.so => /opt/ros/humble/lib/librosidl_typesupport_c.so (0x00007f7f343db000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7f34696000)
	libspdlog.so.1 => /lib/x86_64-linux-gnu/libspdlog.so.1 (0x00007f7f33c9e000)
	libbuiltin_interfaces__rosidl_generator_c.so => /opt/ros/humble/lib/libbuiltin_interfaces__rosidl_generator_c.so (0x00007f7f33c99000)
	libfmt.so.8 => /lib/x86_64-linux-gnu/libfmt.so.8 (0x00007f7f33c76000)

@squizz617
Copy link
Contributor Author

The issue persists after upgrading ros-humble apt package to ros-humble-rcl-action/jammy,now 5.3.4-3jammy.20230721.204515 amd64 [installed,automatic]..

@mjcarroll
Copy link
Member

I'm pretty stumped here. I suppose as a next step, you could look at each of the dependent shared libraries and make sure that they can correctly resolve their dependencies via ldd?

@iuhilnehc-ynos
Copy link
Contributor

@squizz617

I'm interested in this issue because it's almost impossible as the ldd shows that the rcl_action.so is found.

Could you run the command strace python3 -c "import rclpy" > /tmp/failed-to-import-rclpy.txt 2>&1, and then upload the log /tmp/failed-to-import-rclpy.txt?

@squizz617
Copy link
Contributor Author

@iuhilnehc-ynos Here's the strace of python3 importing rclpy:
strace_rclpy.txt.

@iuhilnehc-ynos
Copy link
Contributor

iuhilnehc-ynos commented Aug 10, 2023

According to access("/etc/suid-debug", F_OK), I believe that you changed the /usr/bin/python3.10 with '+s' which will erase the LD_LIBRARY_PATH.

Please revert it with sudo chmod -s /usr/bin/python3.10.

@squizz617
Copy link
Contributor Author

I believe that you changed the /usr/bin/python3.10 with '+s'

@iuhilnehc-ynos Thanks for the suggestion but I haven't set the setuid bit..

$ ls -alh /usr/bin/python3.10 
-rwxr-xr-x 1 root root 5.7M Jun 11 01:26 /usr/bin/python3.10

@iuhilnehc-ynos
Copy link
Contributor

Wow, so there are other cases can make the __libc_enable_secure get true and then call unsetenv the LD_LIBRARY_PATH.

Besides the manually setting suid for the /usr/bin/python3.10, another case I can think of is the mount option.

@squizz617 Could you show the mount options for the device?

@DRAGON-BOOSTERR
Copy link

I also have the same problem, for me the command import rclpy was not working in pycharm and after some messing around I found that we need to add the interpreter paths in pycharm. after that I ended up this error and by following this beautiful thread, Im looking for a way forward. but when I run the code with just the python command python3 ros_test.py, the program runs.

PS: I was following a udemy tutorial on ROS2

@chenyenru
Copy link

@DRAGON-BOOSTERR May you elaborate on how you solved the problem? I'm having the same problem too 😭

@DRAGON-BOOSTERR
Copy link

@chenyenru yeah here is the stackover flow page that I raised the issue and answered myself
https://stackoverflow.com/questions/76983995/python-package-is-installed-but-it-doesnt-get-detected-in-a-ide-ros2/77009205#77009205

@chenyenru
Copy link

Thank you! I was able to solve it by deactivating my anaconda environment (don't know why but that way my Python interpreter finds rclpy package).

@DRAGON-BOOSTERR
Copy link

@chenyenru @squizz617 Hey guys I think i found a short term fix for this issue! As said before I was having problems with IDEs not recognising the rclpy library of ROS2. As by the comment yesterday I mentioned how I solved the problem by adding the interpreter paths in Pycharm IDE. The Pycharm IDE tries to run the file inside the IDE or something like that and received the same error as the title of this thread. Today I tried the code with visual studio code, surprisingly VS code recognised that rclpy library and showed all the auto complete and other things of that library. Since VS code runs every file in terminal inside the IDE, so the code works without any issues. I think this might fix your problem with that error?

@chenyenru
Copy link

I was able to include extra paths in the settings.json file, where the interpreter would successfully autosuggest functions from the rclpy library.

However, when I tried to compile it on the IDE, it still showed that rclpy was not found. I was able to resolve this problem by not running it on conda environment and instead the base file while installing required packages with pip. Hope this result would help you too!

@clalancette clalancette added bug Something isn't working more-information-needed Further information is required labels Sep 14, 2023
@fferri
Copy link

fferri commented Sep 18, 2023

I got the exact issue reported by OP.
Turned out I was overriding LD_LIBRARY_PATH for the process responsible for spawning python3.
Appending/prepending paths to LD_LIBRARY_PATH -instead of overriding- fixed the issue for me.

@asukiaaa
Copy link

I could resolve this problem by removing all packages of using ros2 distribution and install again.

sudo apt remove ros-rolling-*
sudo apt install ros-rolling-desktop-full

@2019lelexia
Copy link

According to this,
https://docs.ros.org/en/rolling/How-To-Guides/Installation-Troubleshooting.html#import-failing-without-library-present-on-the-system
I think we cannot use different version of Python because the Python of ROS2 build different binary file of C extension from the Python of another

@yerkgb
Copy link

yerkgb commented Nov 23, 2023

I had a same problem. These two lines are helped me out:
$sudo apt install --reinstall ros-humble-rclpy
$source /opt/ros/humble/setup.bash
Try your luck!

Also, on your .bashrc file try to source ros directory at the end after all lines.

@sungchan1
Copy link

I finally managed to solve the problem and wrote a blog post about it in Korean.

https://du-sungchan-24k.tistory.com/37

In summary, you need to directly inject this into your pycharm.sh script.

. /opt/ros/humble/setup.sh'
If installed via Jetbrain Toolbox, your pycharm.sh will be located at

'/home/{USER_NAME}/.local/share/JetBrains/Toolbox/apps/{pycharm_version}/bin/pycharm.sh'.
For Jetbrain Remote Development, pycharm.sh is in

'/home/{USER_NAME}/.cache/JetBrains/RemoteDev/dist/{cached_pycharm_version}/bin/pycharm.sh'.
The same applies to CLion and IntelliJ.

@nixrunner
Copy link

nixrunner commented Mar 4, 2024

nixrunner@fedora:/usr/lib64/ros2-iron$ ros2 Traceback (most recent call last): File "/usr/lib64/ros2-iron/bin/ros2", line 33, in <module> sys.exit(load_entry_point('ros2cli==0.25.4', 'console_scripts', 'ros2')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/ros2-iron/bin/ros2", line 25, in importlib_load_entry_point return next(matches).load() ^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/metadata/__init__.py", line 205, in load module = import_module(match.group('module')) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/ros2cli/cli.py", line 22, in <module> from rclpy.executors import ExternalShutdownException File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/__init__.py", line 49, in <module> from rclpy.signals import install_signal_handlers File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/signals.py", line 15, in <module> from rclpy.exceptions import InvalidHandle File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/exceptions.py", line 15, in <module> from rclpy.impl.implementation_singleton import rclpy_implementation as _rclpy File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/impl/implementation_singleton.py", line 32, in <module> rclpy_implementation = import_c_library('._rclpy_pybind11', package) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/ros2-iron/lib/python3.12/site-packages/rpyutils/import_c_library.py", line 39, in import_c_library return importlib.import_module(name, package=package) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ImportError: librosidl_typesupport_c.so: cannot open shared object file: No such file or directory The C extension '/usr/lib64/ros2-iron/lib/python3.12/site-packages/rclpy/_rclpy_pybind11.cpython-312-x86_64-linux-gnu.so' failed to be imported while being present on the system. Please refer to 'https://docs.ros.org/en/{distro}/Guides/Installation-Troubleshooting.html#import-failing-even-with-library-present-on-the-system' for possible solutions

I am having a similar error. I faced with same error on Fedora 39 and Ubuntu 24.04 Noble

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working more-information-needed Further information is required
Projects
None yet
Development

No branches or pull requests