Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find ROS1 message packages #67

Merged
merged 17 commits into from
Jun 13, 2017
Merged

Find ROS1 message packages #67

merged 17 commits into from
Jun 13, 2017

Conversation

mikaelarguedas
Copy link
Member

Currently the bridge doesnt compile if you have an overlay ROS1 workspace. The messages are found and added to the factories but then fail to compile because we never find_package them.

the first commits rename a variable to make it obvious it's referring to ros2 packages.
the second commit call rosmsg and rossrv to get the list of ROS1 message packages and find_package them
the third one makes the shared lib link against these packages
commit 3 and 4 are just fixups/ cleanup

@mikaelarguedas mikaelarguedas added in progress Actively being worked on (Kanban column) in review Waiting for review (Kanban column) and removed in progress Actively being worked on (Kanban column) labels May 26, 2017
CMakeLists.txt Outdated
@@ -34,6 +34,32 @@ endif()

find_ros1_package(std_msgs REQUIRED)

# find all known ROS1 message/service packages
find_ros1_package(rosmsg REQUIRED)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this line needed for?

CMakeLists.txt Outdated
# find all known ROS1 message/service packages
find_ros1_package(rosmsg REQUIRED)
execute_process(
COMMAND bash -c "rosmsg list"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use bash here? This should be platform agnostic.

Same below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I previously tried just the command and it resulted in an empty string that's why I added the bash -c here. Did investigate the why though

CMakeLists.txt Outdated
)
set(ros1_interfaces_string "${rosmsg_output}\n${rossrv_output}")

string(REGEX REPLACE "\n" ";" ros1_interfaces_list "${ros1_interfaces_string}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the pattern doesn't use a regex this should be calling string(REPLACE ...).

@mikaelarguedas
Copy link
Member Author

@Kukanani FYI

@mikaelarguedas mikaelarguedas self-assigned this May 30, 2017
CMakeLists.txt Outdated
ERROR_VARIABLE rossrv_error
)
if(NOT rosmsg_error STREQUAL "" OR NOT rossrv_error STREQUAL "")
message(FATAL_ERROR "${rosmsg_error}\n${rossrv_error}\nFailed to find ROS 1 message packages\nPlease make sure to source your ROS2 workspace BEFORE sourcing your ROS1 workspace")
Copy link
Member

@dirk-thomas dirk-thomas May 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand how this message is specific to the error here. I would expect that the commands fail when the user didn't source ROS 1. Sourcing (or not) ROS 2 as well as the order shouldn't make any difference.

And the CMake code got that far in the first place the user must have sourced something from ROS 1. So maybe the message should just mention that rosmsg / rossrv failed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason here is that ROS1 and ROS2 message packages have been found. But the ros2 ones are the first ones on the python path. So the invocation of rosmsg / rossrv will fail because these are python2 scripts trying to import python3 (ROS2) messages. I can make the error message more explicit to reflect what happens exactly.

I'm still not sure why the problem didnt appear when I was calling out bash explicitely though ...

@Kukanani
Copy link

Kukanani commented May 30, 2017

When trying to build using this branch, I get CMake warnings.
I have a ROS1 workspace with nothing but a custom message package in it, built with catkin_make_isolated. I have a full ROS2 workspace with the ROS2 version of my custom message package, built with ament build --isolated. I then run

# (/opt/ros/kinetic/setup.bash not sourced)
$ source /home/adam/osrf/ros2_ws/install_isolated/local_setup.bash
$ source /home/adam/osrf/ros1_vision_msgs_ws/install_isolated/setup.bash 
$ cd /home/adam/osrf/ros2_ws
$ ament build --isolated --force-cmake --only ros1_bridge

And get the following output:

...

-- Found example_interfaces: 0.0.0 (/home/adam/osrf/ros2_ws/install_isolated/example_interfaces/share/example_interfaces/cmake)
-- Configuring done
CMake Warning at CMakeLists.txt:116 (add_executable):
  Cannot generate a safe runtime search path for target static_bridge because
  there is a cycle in the constraint graph:

    dir 0 is [/home/adam/osrf/ros2_ws/install_isolated/vision_msgs/lib]
    dir 1 is [/home/adam/osrf/ros2_ws/install_isolated/trajectory_msgs/lib]
    dir 2 is [/home/adam/osrf/ros2_ws/install_isolated/tf2_msgs/lib]
    dir 3 is [/home/adam/osrf/ros2_ws/install_isolated/test_communication/lib]
    dir 4 is [/home/adam/osrf/ros2_ws/install_isolated/stereo_msgs/lib]
    dir 5 is [/home/adam/osrf/ros2_ws/install_isolated/std_srvs/lib]
    dir 6 is [/home/adam/osrf/ros2_ws/install_isolated/shape_msgs/lib]
    dir 7 is [/home/adam/osrf/ros2_ws/install_isolated/sensor_msgs/lib]
    dir 8 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp_components/lib]
    dir 9 is [/home/adam/osrf/ros2_ws/install_isolated/console_bridge/lib]
    dir 10 is [/home/adam/osrf/ros2_ws/install_isolated/class_loader/lib]
      dir 32 must precede it due to runtime library [libclass_loader.so]
    dir 11 is [/home/adam/osrf/ros2_ws/install_isolated/rcutils/lib]
    dir 12 is [/home/adam/osrf/ros2_ws/install_isolated/rmw/lib]
    dir 13 is [/home/adam/osrf/ros2_ws/install_isolated/rmw_fastrtps_cpp/lib]
    dir 14 is [/home/adam/osrf/ros2_ws/install_isolated/rcl/lib]
    dir 15 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp/lib]
    dir 16 is [/home/adam/osrf/ros2_ws/install_isolated/composition/lib]
    dir 17 is [/home/adam/osrf/ros2_ws/install_isolated/nav_msgs/lib]
    dir 18 is [/home/adam/osrf/ros2_ws/install_isolated/diagnostic_msgs/lib]
    dir 19 is [/home/adam/osrf/ros2_ws/install_isolated/geometry_msgs/lib]
    dir 20 is [/home/adam/osrf/ros2_ws/install_isolated/actionlib_msgs/lib]
    dir 21 is [/home/adam/osrf/ros2_ws/install_isolated/std_msgs/lib]
    dir 22 is [/home/adam/osrf/ros2_ws/install_isolated/rcl_interfaces/lib]
    dir 23 is [/home/adam/osrf/ros2_ws/install_isolated/pendulum_msgs/lib]
    dir 24 is [/home/adam/osrf/ros2_ws/install_isolated/lifecycle_msgs/lib]
    dir 25 is [/home/adam/osrf/ros2_ws/install_isolated/example_interfaces/lib]
    dir 26 is [/home/adam/osrf/ros2_ws/install_isolated/builtin_interfaces/lib]
    dir 27 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_c/lib]
    dir 28 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_cpp/lib]
    dir 29 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_generator_c/lib]
    dir 30 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_c/lib]
    dir 31 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_cpp/lib]
    dir 32 is [/opt/ros/kinetic/lib]
      dir 10 must precede it due to runtime library [libclass_loader.so]
    dir 33 is [/home/adam/osrf/ros2_ws/build_isolated/ros1_bridge]

  Some of these libraries may not be found correctly.
Call Stack (most recent call first):
  CMakeLists.txt:160 (custom_executable)


CMake Warning at CMakeLists.txt:116 (add_executable):
  Cannot generate a safe runtime search path for target dynamic_bridge
  because there is a cycle in the constraint graph:

    dir 0 is [/home/adam/osrf/ros2_ws/install_isolated/vision_msgs/lib]
    dir 1 is [/home/adam/osrf/ros2_ws/install_isolated/trajectory_msgs/lib]
    dir 2 is [/home/adam/osrf/ros2_ws/install_isolated/tf2_msgs/lib]
    dir 3 is [/home/adam/osrf/ros2_ws/install_isolated/test_communication/lib]
    dir 4 is [/home/adam/osrf/ros2_ws/install_isolated/stereo_msgs/lib]
    dir 5 is [/home/adam/osrf/ros2_ws/install_isolated/std_srvs/lib]
    dir 6 is [/home/adam/osrf/ros2_ws/install_isolated/shape_msgs/lib]
    dir 7 is [/home/adam/osrf/ros2_ws/install_isolated/sensor_msgs/lib]
    dir 8 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp_components/lib]
    dir 9 is [/home/adam/osrf/ros2_ws/install_isolated/console_bridge/lib]
    dir 10 is [/home/adam/osrf/ros2_ws/install_isolated/class_loader/lib]
      dir 32 must precede it due to runtime library [libclass_loader.so]
    dir 11 is [/home/adam/osrf/ros2_ws/install_isolated/rcutils/lib]
    dir 12 is [/home/adam/osrf/ros2_ws/install_isolated/rmw/lib]
    dir 13 is [/home/adam/osrf/ros2_ws/install_isolated/rmw_fastrtps_cpp/lib]
    dir 14 is [/home/adam/osrf/ros2_ws/install_isolated/rcl/lib]
    dir 15 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp/lib]
    dir 16 is [/home/adam/osrf/ros2_ws/install_isolated/composition/lib]
    dir 17 is [/home/adam/osrf/ros2_ws/install_isolated/nav_msgs/lib]
    dir 18 is [/home/adam/osrf/ros2_ws/install_isolated/diagnostic_msgs/lib]
    dir 19 is [/home/adam/osrf/ros2_ws/install_isolated/geometry_msgs/lib]
    dir 20 is [/home/adam/osrf/ros2_ws/install_isolated/actionlib_msgs/lib]
    dir 21 is [/home/adam/osrf/ros2_ws/install_isolated/std_msgs/lib]
    dir 22 is [/home/adam/osrf/ros2_ws/install_isolated/rcl_interfaces/lib]
    dir 23 is [/home/adam/osrf/ros2_ws/install_isolated/pendulum_msgs/lib]
    dir 24 is [/home/adam/osrf/ros2_ws/install_isolated/lifecycle_msgs/lib]
    dir 25 is [/home/adam/osrf/ros2_ws/install_isolated/example_interfaces/lib]
    dir 26 is [/home/adam/osrf/ros2_ws/install_isolated/builtin_interfaces/lib]
    dir 27 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_c/lib]
    dir 28 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_cpp/lib]
    dir 29 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_generator_c/lib]
    dir 30 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_c/lib]
    dir 31 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_cpp/lib]
    dir 32 is [/opt/ros/kinetic/lib]
      dir 10 must precede it due to runtime library [libclass_loader.so]
    dir 33 is [/home/adam/osrf/ros2_ws/build_isolated/ros1_bridge]

  Some of these libraries may not be found correctly.
Call Stack (most recent call first):
  CMakeLists.txt:167 (custom_executable)


CMake Warning at CMakeLists.txt:145 (add_library):
  Cannot generate a safe runtime search path for target ros1_bridge because
  there is a cycle in the constraint graph:

    dir 0 is [/opt/ros/kinetic/lib]
      dir 11 must precede it due to runtime library [libclass_loader.so]
    dir 1 is [/home/adam/osrf/ros2_ws/install_isolated/vision_msgs/lib]
    dir 2 is [/home/adam/osrf/ros2_ws/install_isolated/trajectory_msgs/lib]
    dir 3 is [/home/adam/osrf/ros2_ws/install_isolated/tf2_msgs/lib]
    dir 4 is [/home/adam/osrf/ros2_ws/install_isolated/test_communication/lib]
    dir 5 is [/home/adam/osrf/ros2_ws/install_isolated/stereo_msgs/lib]
    dir 6 is [/home/adam/osrf/ros2_ws/install_isolated/std_srvs/lib]
    dir 7 is [/home/adam/osrf/ros2_ws/install_isolated/shape_msgs/lib]
    dir 8 is [/home/adam/osrf/ros2_ws/install_isolated/sensor_msgs/lib]
    dir 9 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp_components/lib]
    dir 10 is [/home/adam/osrf/ros2_ws/install_isolated/console_bridge/lib]
    dir 11 is [/home/adam/osrf/ros2_ws/install_isolated/class_loader/lib]
      dir 0 must precede it due to runtime library [libclass_loader.so]
    dir 12 is [/home/adam/osrf/ros2_ws/install_isolated/composition/lib]
    dir 13 is [/home/adam/osrf/ros2_ws/install_isolated/nav_msgs/lib]
    dir 14 is [/home/adam/osrf/ros2_ws/install_isolated/diagnostic_msgs/lib]
    dir 15 is [/home/adam/osrf/ros2_ws/install_isolated/geometry_msgs/lib]
    dir 16 is [/home/adam/osrf/ros2_ws/install_isolated/actionlib_msgs/lib]
    dir 17 is [/home/adam/osrf/ros2_ws/install_isolated/std_msgs/lib]
    dir 18 is [/home/adam/osrf/ros2_ws/install_isolated/pendulum_msgs/lib]
    dir 19 is [/home/adam/osrf/ros2_ws/install_isolated/lifecycle_msgs/lib]
    dir 20 is [/home/adam/osrf/ros2_ws/install_isolated/example_interfaces/lib]
    dir 21 is [/home/adam/osrf/ros2_ws/install_isolated/builtin_interfaces/lib]
    dir 22 is [/home/adam/osrf/ros2_ws/install_isolated/rcl_interfaces/lib]
    dir 23 is [/home/adam/osrf/ros2_ws/install_isolated/rcutils/lib]
    dir 24 is [/home/adam/osrf/ros2_ws/install_isolated/rmw/lib]
    dir 25 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_c/lib]
    dir 26 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_introspection_cpp/lib]
    dir 27 is [/home/adam/osrf/ros2_ws/install_isolated/rmw_fastrtps_cpp/lib]
    dir 28 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_generator_c/lib]
    dir 29 is [/home/adam/osrf/ros2_ws/install_isolated/rcl/lib]
    dir 30 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_c/lib]
    dir 31 is [/home/adam/osrf/ros2_ws/install_isolated/rosidl_typesupport_cpp/lib]
    dir 32 is [/home/adam/osrf/ros2_ws/install_isolated/rclcpp/lib]

  Some of these libraries may not be found correctly.


-- Generating done
-- Build files have been written to: /home/adam/osrf/ros2_ws/build_isolated/ros1_bridge
==> '. /home/adam/osrf/ros2_ws/build_isolated/ros1_bridge/cmake__build.sh && /usr/bin/make -j8 -l8' in '/home/adam/osrf/ros2_ws/build_isolated/ros1_bridge'
[  5%] Built target simple_bridge_2_to_1
[ 17%] Built target simple_bridge
[ 17%] Built target simple_bridge_1_to_2
Scanning dependencies of target ros1_bridge
[ 22%] Building CXX object CMakeFiles/ros1_bridge.dir/src/convert_builtin_interfaces.cpp.o

...

@mikaelarguedas
Copy link
Member Author

thanks @Kukanani for the report. looks like cmake is not happy about finding twice class_loader.

Can you confirm that despite the warning, this does fix the original issue and you can use the bridge for the custom message package?

I'll try to reproduce the warning locally and comment here.

@mikaelarguedas
Copy link
Member Author

tested this again in a fresh docker container using the same message package as @Kukanani and various configs and could not reproduce the Cmake warning. Without a reproducible example I'll move forward and ask for this to be reviewed.
The error message has been changed in df0799a to address #67 (comment)

CMakeLists.txt Outdated
ERROR_VARIABLE rossrv_error
)
if(NOT rosmsg_error STREQUAL "" OR NOT rossrv_error STREQUAL "")
message(FATAL_ERROR "${rosmsg_error}\n${rossrv_error}\nFailed to call rosmsg/rossrv\nPlease make sure to source your ROS2 workspace BEFORE sourcing your ROS1 workspace")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current recommendation is sourcing ROS 2 after ROS 1. That is to find CMake config files of ROS 2 packages over ROS 1 packages in case of the same package name. With this inverted order how is that supposed to work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify why this is necessary in the first place? (In the context of the bridge being built separately). Is it to find the mapping_rules? or is it for another reason that it's necessary to find the ROS2 cmake config files?

The problem I'm facing is that given that the message packages have the same names on both side, the ROS1 python scripts (rosmsg and rossrv) need the ros ones to be the first ones on the sys.path so that when they import std_msgs they get the ROS1 module. I'm happy to explore an alternative solution to the rosmsg/rossrv approach if you have a pointer to what would be the best way to handle this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ROS 2 packages can only be found via CMake, ROS 1 on the other hand can additionally be found via pkg-config. In order to be able to find both kinds. ROS 2 must be sourced last.

The problem with the Python code for messages is that both ROS 1 as well as 2 are using the exact same naming scheme. Maybe that is another reason why we should consider using a different naming in ROS 2 in order to distinguish them.

@Kukanani Kukanani mentioned this pull request Jun 10, 2017
@mikaelarguedas
Copy link
Member Author

a266513 ensures that we can keep the current order of sourcing worskpaces. This addreses #67 (comment) allow us to keep the instructions as is and fixes the original problem of ros1 workspace overlays

@mikaelarguedas
Copy link
Member Author

CI:
Build Status

CMakeLists.txt Outdated
@@ -34,6 +34,59 @@ endif()

find_ros1_package(std_msgs REQUIRED)

# ROS1 message packages need to be first in the PYTHONPATH
# So we remove every ROS2 path from the python path for now
set(__AMENT_PREFIX_PATH $ENV{AMENT_PREFIX_PATH})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why two underscores as a prefix? One should be sufficient.

CMakeLists.txt Outdated
string(REPLACE ":" ";" __PYTHONPATH ${__PYTHONPATH})
foreach(ament_path ${__AMENT_PREFIX_PATH})
foreach(py_path ${__PYTHONPATH})
if(py_path MATCHES "^${ament_path}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ament_path doesn't have a trailing slash this might match for unrelated paths. Therefore the pattern should end with a slash / backslash.

CMakeLists.txt Outdated
set(__PYTHONPATH_STRING "")
foreach(py_path ${__PYTHONPATH})
string(CONCAT __PYTHONPATH_STRING "${__PYTHONPATH_STRING}" "${py_path}" ":")
endforeach()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of building the Python path in a separate loop could this be populated in the loops above? Also I would suggest to use a CMake list and afterwards replace the semicolons with colons (but not on Windows!).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I tried but couldn't figure out how to do that (Convert a list to string and replace semi-colon with colons).
I'm didn't put much thought about the windows case figuring that windows users won't have ROS1 installed so this cmakelists would return before hitting this section.

CMakeLists.txt Outdated
OUTPUT_VARIABLE rossrv_output
ERROR_VARIABLE rossrv_error
)
set(ENV{PYTHONPATH} ${__ORIG_PYTHONPATH})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of changing the environment and resetting it later it would be cleaned to just set the environment variable for the two executed processes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh good, I didn't know cmake had a shell + platform agnostic way to do that, I'll look into it then

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't find a way to do it from CMake so I kept the current logic of replacing and restoring it after calling the scripts

@mikaelarguedas
Copy link
Member Author

I addressed all review comments except #67 (comment).
Ready for another review

@dirk-thomas
Copy link
Member

dirk-thomas commented Jun 12, 2017

After some offline conversation I refactored the code a bit to hopefully make it work across all platforms and be little bit shorter / more readable. See 0157853

Build Status

@mikaelarguedas
Copy link
Member Author

Looks like it fail, I think that the latest change leaves us with an empty pythonpath. Looking into it now

Copy link
Member

@dirk-thomas dirk-thomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mikaelarguedas
Copy link
Member Author

all good: Build Status
Merging

@mikaelarguedas mikaelarguedas merged commit 449a08b into master Jun 13, 2017
@mikaelarguedas mikaelarguedas deleted the find_ros1_packages branch June 13, 2017 00:45
@mikaelarguedas mikaelarguedas removed the in review Waiting for review (Kanban column) label Jun 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants