rosdep and ROS2

[clydemcqueen]

It appears that rosdep check depends on ROS_PACKAGE_PATH, which is set by the ROS1 build system (catkin).

The ROS2 build system (ament) does not set ROS_PACKAGE_PATH. The workaround is for the user to export (and update) ROS_PACKAGE_PATH.

Is the ROS2 behavior working-as-intended?

See also: https://answers.ros.org/question/315296/crystal-rosdep-cannot-find-all-required-resources/

Thanks.

[dirk-thomas]

Just a nipick about the terms and part involved: the build tool (like colcon) has nothing to do with it. The build system (catkin vs. ament) determine the functionality and ament_cmake / ROS 2 doesn't have the concept of a ROS_PACKAGE_PATH. Hence I removed colcon from the title.

[NikolausDemmel]

So what ament/ros2 mechanism should rosdep be using instead to find packages?

[dirk-thomas]

ROS 2 uses the environment variable AMENT_PREFIX_PATH for a list of prefixes where packages are located. In each prefix path the packages can be retrieved from the resource index (rather than crawling through the file system).

[ruffsl]

In each prefix path the packages can be retrieved from the resource index

@dirk-thomas , what do you mean by "resource index"? Is this a file or something at the root of one of the paths listed in AMENT_PREFIX_PATH? After sourcing a dashing install, I only see:

AMENT_PREFIX_PATH=/opt/ros/dashing
$ ls -alh /opt/ros/dashing/
total 112K
drwxr-xr-x   9 root root 4.0K Jul  9 13:50 .
drwxr-xr-x   5 root root 4.0K Jul  9 13:49 ..
drwxr-xr-x   2 root root 4.0K Aug  9 19:55 bin
drwxr-xr-x   2 root root 4.0K Jul  9 13:49 cmake
drwxr-xr-x 104 root root 4.0K Aug  9 19:55 include
drwxr-xr-x  62 root root  40K Aug  9 19:55 lib
-rw-r--r--   1 root root  371 Apr  8 15:01 local_setup.bash
-rw-r--r--   1 root root 2.1K May 29 22:03 local_setup.sh
-rw-r--r--   1 root root  644 Apr  8 15:01 local_setup.zsh
drwxr-xr-x   4 root root 4.0K Jul  9 13:50 opt
-rw-r--r--   1 root root 2.5K Apr  8 15:01 _order_packages.py
-rw-r--r--   1 root root  347 Apr  8 15:01 setup.bash
-rw-r--r--   1 root root 4.0K May 29 22:03 setup.sh
-rw-r--r--   1 root root  620 Apr  8 15:01 setup.zsh
drwxr-xr-x 198 root root  12K Aug 12 21:38 share
drwxr-xr-x   4 root root 4.0K Jul  9 13:49 src

[dirk-thomas]

The resource index is a file base storage under share/ament_index/resource_index. Please see the docs for more information: https://github.com/ament/ament_cmake/blob/be483fdcf62de7d89ea4107de2ea594804860ec4/ament_cmake_core/doc/resource_index.md

[ruffsl]

Thanks, that was very informative.
Looks like we just need to add some logic to include package results from something like:

def list_prefix_of_packages(prefixes):
    return list_prefix_of_packages_by_resource('packages', prefixes)

@dirk-thomas , I'm assuming you wouldn't want to rope in ament_package as a python dependency for rosdep. Would there be an existing implementation of the function above I could use for a PR?

Or should I just use the template from ament_package as a pattern for extending rosdep's lookup?

https://github.com/ament/ament_package/blob/a6c73aeff77c188597211fee9f9e34508487b531/ament_package/template/prefix_level/_order_packages.py

Are there any special considerations when looking up isolated packages as well? I see there is a _order_isolated_packages.py template as well in the ament_package repo.

After sourcing an isolated workspace, does the AMENT_PREFIX_PATH env simply contain a list of <root>/<pkg_name> paths that each contain a separate 'share/ament_index/resource_index/packages resource folder? Thus no additional logic on our side is needed other than pre-existing capability to iteration over multiple prefix paths in the AMENT_PREFIX_PATH env.

[dirk-thomas]

I'm assuming you wouldn't want to rope in ament_package as a python dependency for rosdep.

That is simply not possible. rosdeb is a plain Python package which is not related to a specific ROS distro. Since ament_package is a ROS package released into specific ROS distros it can't be a dependency of rosdeb.

Would there be an existing implementation of the function above I could use for a PR?

The ament_index_* packages implement an API to access the resource index in C++ and Python. The same limitation applies. So they can only serve as a template to create your own function.

After sourcing an isolated workspace, does the AMENT_PREFIX_PATH env simply contain a list of <root>/<pkg_name> paths that each contain a separate share/ament_index/resource_index/packages resource folder?

Correct.

I see there is a _order_isolated_packages.py template as well in the ament_package repo.

I am not sure why you would need the order for something like rosdep check?

[ruffsl]

I am not sure why you would need the order for something like rosdep check?

I figured, just want to check if it had any cause to be a special case.

So they can only serve as a template to create your own function.

Ok, sounds good. I'll read through the rosdep code a bit more and try an open a PR.

[ruffsl]

Please see: #699