Having Issues Bundling Ros1 Project #13

mr337 · 2019-02-16T18:02:29Z

Hi,

New to colcon and trying to wrap my head for ROS1 (Kinetic) project. I am trying to bundle a ROS1 (Kinetic) with colcon bundle that built fine with colcon build.

$ colcon bundle              
Bundling workspace...
Reading package lists... Done
Building dependency tree... Done
Collecting dependency information...
[7.788s] WARNING:colcon.colcon_bundle.verb:No task extension to bundle a cmake package
                     
Summary: 0 packages finished [7.24s]
Fetching and installing dependencies...
[9.643s] ERROR:colcon:colcon bundle: [Errno 2] No such file or directory: '/home/ubuntu/Dev/bundle/bundle_staging/setup.sh'
Traceback (most recent call last):
  File "/home/ubuntu/.local/lib/python3.5/site-packages/colcon_core/command.py", line 436, in verb_main
    rc = context.args.main(context=context)
  File "/home/ubuntu/.local/lib/python3.5/site-packages/colcon_bundle/verb/bundle.py", line 142, in main
    generate_archive_v1(install_base, staging_path, bundle_base)
  File "/home/ubuntu/.local/lib/python3.5/site-packages/colcon_bundle/verb/_archive_generators.py", line 28, in generate_archive_v1
    shutil.copy2(shellscript_path, os.path.join(staging_path, 'setup.sh'))
  File "/usr/lib/python3.5/shutil.py", line 251, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.5/shutil.py", line 115, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/Dev/bundle/bundle_staging/setup.sh'

I am guessing that the bundle_staging/setup.sh file missing is due to the missing of extension to handle cmake packages. Looks like in version 0.0.6 CMake handling was added in #6 and #7.

Here are my current versions of colcon and related modules

$ pip3 freeze | grep colcon
colcon-argcomplete==0.3.2
colcon-bash==0.3.1
colcon-bundle==0.0.11
colcon-cmake==0.2.8
colcon-common-extensions==0.2.0
colcon-core==0.3.17
colcon-defaults==0.2.1
colcon-devtools==0.2.1
colcon-library-path==0.2.1
colcon-metadata==0.2.1
colcon-notification==0.2.6
colcon-output==0.2.3
colcon-package-information==0.2.2
colcon-package-selection==0.2.3
colcon-parallel-executor==0.2.3
colcon-pkg-config==0.1.0
colcon-powershell==0.3.3
colcon-python-setup-py==0.2.1
colcon-recursive-crawl==0.2.0
colcon-ros==0.3.7
colcon-ros-bundle==0.0.11
colcon-test-result==0.3.1
colcon-zsh==0.3.1

If it is better to report this to colcon-bundle project let me know!

The text was updated successfully, but these errors were encountered:

murphm8 · 2019-02-18T19:23:15Z

Thanks for the bug report! The WARNING hints at the problem, but we should definitely have some better messaging, I've filed colcon/colcon-bundle#34 for this.

There are two possible causes of WARNING:colcon.colcon_bundle.verb:No task extension to bundle a cmake package.

Do you have CMakeLists.txt in intermediate directories? For example:

src/
├── package_1/
│   ├── package.xml
|   └── CMakeLists.txt (OK)
└── intermediate_directory/
    ├── package_2
    |   ├── package.xml
    |   └── CMakeLists.txt (OK)
    ├── package_3
    |   ├── package.xml
    |   └── CMakeLists.txt (OK)
    └── CMakeLists.txt (REMOVE)

If you do please remove the intermediate CMakeLists.txt they are unecessary and cause colcon to recognize the intermediate directory as a cmake package and skip all the ROS sub-packages, where we need to get the dependency information from.

The other reason why this warning might have been emitted is because you have a cmake package instead of a ros.cmake package.

A ros.cmake package contains a package.xml with <buildtool_depend>cmake</buildtool_depend>. Here is the package.xml from a ros.cmake package in our integration test suite. To fix the warning you could place a similar package.xml in the root of your pure CMake package. colcon-bundle does not currently support pure CMake packages, I don't expect converting to ros.cmake will cause any issues.

Please let me know if either of those solutions fixes your issue. I'll be happy to help troubleshoot further.

mr337 · 2019-02-19T00:31:39Z

Looks like I had all of those problems!

Problems:

I had an intermediary CMakeLists.txt from the original project creation.
For a package I wasn't using format v2 in my package.xml
For a package I updated the CMakeLists.txt to remove the use of catkin_package

Once the above were done I now have a bundle file!

murphm8 · 2019-02-19T01:35:34Z

That's great to hear! One note from your reply:

For a package I updated the CMakeLists.txt to remove the use of catkin_package

You shouldn't need to remove usages of catkin_package from your CMakeLists.txt in ROS packages. colcon build uses catkin_make_isolated to build catkin packages (link). Just want to clarify that, in case removing this causes other build or linking problems for you.

Edit: Strikethrough see Dirk's reponse below.

dirk-thomas · 2019-02-19T02:45:43Z

colcon build uses catkin_make_isolated to build catkin packages.

This statements isn't correct. colcon doesn't use catkin_make_isolated anywhere. Also the referenced documentation doesn't state that - it says:

... Using colcon instead of the recommended tool catkin_make_isolated only changes a couple of the steps. ...

mr337 · 2019-02-19T02:46:08Z

That is what I was thinking at first since I am using other catkin helpers such as catkin_add_gmock and catkin_install_python.

Unfortunately adding back catkin_package throws this error:

$ colcon build
Starting >>> packagename
--- stderr: packangename                         
CMake Error at CMakeLists.txt:27 (catkin_package):
  Unknown CMake command "catkin_package".


---
Failed   <<< packagename   [ Exited with code 1 ]

This is part of my cmake file that is throwing the error

catkin_package(
  INCLUDE_DIRS include
  LIBRARIES rosfmt
  CATKIN_DEPENDS  rospy std_msgs sensor_msgs geometry_msgs....
)

murphm8 · 2019-02-19T02:57:22Z

@dirk-thomas Should catkin_package() be available to builds using colcon build?

dirk-thomas · 2019-02-19T03:32:44Z

Should catkin_package() be available to builds using colcon build?

This has nothing to do with colcon. If you want to call the CMake function catkin_package provided by the package catkin you need to call find_package(catkin REQUIRED) in your CMakeLists.txt file before.

mr337 · 2019-02-19T03:45:00Z

I wonder if my CMakeLists.txt file isn't setup properly. Reverting back to #13 (comment) and adding find_package(catkin REQUIRED) before any executable or libraries are added results in this error:

$ colcon build
Starting >>> packagename
--- stderr: packagename                         
CMake Error at /opt/ros/kinetic/share/catkin/cmake/catkin_package.cmake:116 (message):
  catkin_package() 'catkin' must be listed as a buildtool dependency in the
  package.xml
Call Stack (most recent call first):
  /opt/ros/kinetic/share/catkin/cmake/catkin_package.cmake:102 (_catkin_package)
  CMakeLists.txt:29 (catkin_package)


---
Failed   <<< packagename   [ Exited with code 1 ]

murphm8 · 2019-04-19T17:27:09Z

@mr337 Sorry, I missed your update. This looks like you're missing <buildtool_depend>catkin</buildtool_depend> in your package.xml

Ref: http://wiki.ros.org/catkin/package.xml
Ref: http://wiki.ros.org/catkin/CMakeLists.txt

Rohita83 · 2021-03-24T10:25:04Z

@murphm8 @dirk-thomas @mr337 I got the following error same error as below when I execute as colcon bundle --apt-sources-list ~/test_bundle_apt/source.list where source.list having local apt packages path I.e. deb [trusted=yes] file:///home/user/test_bundle_apt/alto_apt_pkgs ./
Note: I had downloaded all apt packages using apt download <packages list> under this folder 'alto_apt_pkgs' and also generate a Packages details file using dpkg-scanpackages . /dev/null > Packages that was required for colcon bundle command.

Error as below:

Get:191 file:/home/user/test_bundle_apt/alto_apt_pkgs ./ ros-melodic-ros-core 1.4.1-0bionic.20210112.183722 [2176 B]                         
Get:192 file:/home/user/test_bundle_apt/alto_apt_pkgs ./ ros-melodic-ros-base 1.4.1-0bionic.20210112.183921 [1924 B]                         
Fetched 0 B in 0s (0 B/s)                                                                                                                    
Extracting apt packages...
Creating bundle archive V2...
[6.795s] ERROR:colcon:colcon bundle: [Errno 2] No such file or directory: '/home/user/temp/AltoRosNodes/host_app_ws/bundle/cache/bundle_staging/setup.sh'
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/colcon_core/command.py", line 528, in verb_main
    rc = context.args.main(context=context)
  File "/usr/local/lib/python3.6/dist-packages/colcon_bundle/verb/bundle.py", line 143, in main
    dependencies_changed)
  File "/usr/local/lib/python3.6/dist-packages/colcon_bundle/verb/_archive_generators.py", line 111, in generate_archive_v2
    dependencies_overlay_path)
  File "/usr/local/lib/python3.6/dist-packages/colcon_bundle/verb/_overlay_utilities.py", line 81, in create_dependencies_overlay
    _CONTEXT_VAR_SH
  File "/usr/local/lib/python3.6/dist-packages/colcon_bundle/verb/_overlay_utilities.py", line 136, in _render_template
    with script_dest.open('w') as file:
  File "/usr/lib/python3.6/pathlib.py", line 1183, in open
    opener=self._opener)
  File "/usr/lib/python3.6/pathlib.py", line 1037, in _opener
    return self._accessor.open(self, flags, mode)
  File "/usr/lib/python3.6/pathlib.py", line 387, in wrapped
    return strfunc(str(pathobj), *args)
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/temp/AltoRosNodes/host_app_ws/bundle/cache/bundle_staging/setup.sh'

I have also attached entirely the logs below for reference. my local apt packages folder i.e. alto_apt_pkgs is ~580 but I am getting this error after 192 apt packages.
colcon-bundle-error.log

Could you please assist me to crack this error?

Rohita83 · 2021-04-02T07:08:28Z

@murphm8 @dirk-thomas @mr337 I got the following error same error as below when I execute as colcon bundle --apt-sources-list ~/test_bundle_apt/source.list where source.list having local apt packages path I.e. deb [trusted=yes] file:///home/user/test_bundle_apt/alto_apt_pkgs ./
Note: I had downloaded all apt packages using apt download <packages list> under this folder 'alto_apt_pkgs' and also generate a Packages details file using dpkg-scanpackages . /dev/null > Packages that was required for colcon bundle command.

Error as below:

@murphm8 @dirk-thomas @mr337
Could you guys please assist me to crack the above error?

mr337 · 2021-04-02T17:56:39Z

I don't think I have anything else to add except for what is on this issue. Once resolved all my problems went away.

murphm8 · 2021-04-06T23:24:46Z

@Rohita83 Have you tried deleting the bundle cache? Please rm -rf /home/user/temp/AltoRosNodes/host_app_ws/bundle and try again, it might be an inconsistency in the cache.

Theoretically doing what you are doing should work if you have downloaded all the packages that are in the dependency closure.

I see these errors in the logs:

[3.087s] ERROR:colcon.colcon_ros_bundle.task:Could not find key for OpenCV
[3.087s] ERROR:colcon.colcon_ros_bundle.task:Could not find key for scikit-image

Are you using the proper rosdep names in your package xml for these apt packages?

Here's what you might be able to use for scikit: python-skimage-pip - https://index.ros.org/search/?term=scikit&section=deps
If you want Python's OpenCV library then you should use: python-opencv - https://index.ros.org/search/?term=OpenCV&section=deps

murphm8 added the bug Something isn't working label Feb 18, 2019

murphm8 self-assigned this Feb 18, 2019

murphm8 mentioned this issue Feb 18, 2019

Bundling should exit with an error message if no ROS packages are found colcon/colcon-bundle#34

Closed

mr337 closed this as completed Feb 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Having Issues Bundling Ros1 Project #13

Having Issues Bundling Ros1 Project #13

mr337 commented Feb 16, 2019

murphm8 commented Feb 18, 2019 •

edited

Loading

mr337 commented Feb 19, 2019 •

edited

Loading

murphm8 commented Feb 19, 2019 •

edited

Loading

dirk-thomas commented Feb 19, 2019

mr337 commented Feb 19, 2019

murphm8 commented Feb 19, 2019

dirk-thomas commented Feb 19, 2019

mr337 commented Feb 19, 2019

murphm8 commented Apr 19, 2019 •

edited

Loading

Rohita83 commented Mar 24, 2021 •

edited by emersonknapp

Loading

Rohita83 commented Apr 2, 2021

mr337 commented Apr 2, 2021

murphm8 commented Apr 6, 2021 •

edited

Loading

Having Issues Bundling Ros1 Project #13

Having Issues Bundling Ros1 Project #13

Comments

mr337 commented Feb 16, 2019

murphm8 commented Feb 18, 2019 • edited Loading

mr337 commented Feb 19, 2019 • edited Loading

murphm8 commented Feb 19, 2019 • edited Loading

dirk-thomas commented Feb 19, 2019

mr337 commented Feb 19, 2019

murphm8 commented Feb 19, 2019

dirk-thomas commented Feb 19, 2019

mr337 commented Feb 19, 2019

murphm8 commented Apr 19, 2019 • edited Loading

Rohita83 commented Mar 24, 2021 • edited by emersonknapp Loading

Rohita83 commented Apr 2, 2021

mr337 commented Apr 2, 2021

murphm8 commented Apr 6, 2021 • edited Loading

murphm8 commented Feb 18, 2019 •

edited

Loading

mr337 commented Feb 19, 2019 •

edited

Loading

murphm8 commented Feb 19, 2019 •

edited

Loading

murphm8 commented Apr 19, 2019 •

edited

Loading

Rohita83 commented Mar 24, 2021 •

edited by emersonknapp

Loading

murphm8 commented Apr 6, 2021 •

edited

Loading