Use catkin_tools on top of colcon workspace

[paulbovbel]

Error when trying to have one catkin workspace (via catkin_tools) extend another catkin workspace built by colcon, due to a missing env.sh file.

error: could not configure catkin workspace: Cannot load environment from resultspace "/opt/locus/colcon-test/bot" because the required setup file "/opt/locus/colcon-test/bot/env.sh" does not exist.

It appears that colcon does not generate an env.sh file in the install space. Is this intentional?

[dirk-thomas]

Yes, colcon intentionally suppresses the generation of the prefix-level files from catkin (

colcon-ros/colcon_ros/task/ros.py

Line 90 in c1a5214

args.cmake_args += ['-DCATKIN_INSTALL_INTO_PREFIX_ROOT=0']

) since those would collide with the ones provided by colcon and wouldn't work with other packages in the workspaces.

What are you trying to do where you rely on the env.sh file to be present?

[paulbovbel]

catkin config --extend /colcon_ws/install will fail when env.sh is not present.

[dirk-thomas]

To support using catkin_tools on top of a colcon workspace:

• catkin would need to provide an option to still generate the env.sh script while suppressing the generation of setup files or
• colcon-ros could generate the file itself (I haven't checked if that feasible though)

Since this seems to be an uncommon case it might not be something the maintainers will likely work on.

[paulbovbel]

I don't know if it's so uncommon given that developers are likely to continue using catkin-tools (for devel support), while colcon is great from a CI/release point of view. Happy to take a crack at the first option.

[dirk-thomas]

I don't know if it's so uncommon given that developers are likely to continue using catkin-tools (for devel support) ...

Sure, but why would they mix two build tools?

[paulbovbel]

I'm using colcon for packaging workspaces as 'fat debs', while consumers of these packages are likely to continue using catkin-tools in the near future.

[paulbovbel]

Adding _setup_util.py to this list - when missing, you tend to end up with a mangled CMAKE_PREFIX_PATH and PYTHONPATH when extending a colcon resultspace with catkin-tools.

While it makes sense not to install any of the setup.* scripts since that interfers with colcon, it may make sense to continue installing env.sh and _setup_util.py even when CATKIN_INSTALL_INTO_PREFIX_ROOT=0.

[dirk-thomas]

The CATKIN_INSTALL_INTO_PREFIX_ROOT option prevents any installation into the root of the install prefix. That is necessary to avoid concurrently trying to write these files from multiple packages since it poses a race condition.

[paulbovbel]

Agreed that it's unpleasant. If you think this issue is too esoteric (catkin-tools building on top of a colcon resultspace), I can close it out. I'm working around it for now, and it's probably just a matter of adding a few usability improvements that I'd be able to drop catkin_tools in favour of colcon wholesale.

[dirk-thomas]

If you think this issue is too esoteric (catkin-tools building on top of a colcon resultspace), I can close it out.

No, absolutely not. It would be great if they can coexist / their results inter operate.

[mikepurvis]

Rather than coercing colcon into generating the env.sh file, another option might be teaching catkin_tools how to source the environment of a colcon workspace. I'm still working this out for myself (see colcon/colcon-core#262), but it might be a more fruitful path forward.

[paulbovbel]

I've done some terrible things here (https://github.com/locusrobotics/tailor-distro/blob/master/tailor_distro/debian_templates/rules.j2#L52) to work around this issue for artifacts generated by our CI, but would love a 'better' way.

To be fair, I think colcon/colcon-core#168 is the only outstanding feature that would allow us to move off catkin-tools onto colcon for development wholesale, and drop the whole issue. I can't count the number of user faults we avoid by using the catkin config --extend mechanism to prevent misconfigured/poisoned developer environments.

[dirk-thomas]

I think colcon/colcon-core#168 is the only outstanding feature that would allow us to move off catkin-tools onto colcon for development wholesale, and drop the whole issue.

It would be great if you could post as specific as possible workflow examples for this in the referenced ticket (what are the steps / commands you commonly use). That will help to make sure your use case works if anyone is working on this feature.

I can't count the number of user faults we avoid by using the catkin config --extend mechanism to prevent misconfigured/poisoned developer environment.

I wouldn't consider this to be part of the profile feature. Also depending on what the user passes it requires to "rollback" environment changed of other not chained workspaces which isn't a feature supported by ament (simply because it can only work partially even in catkin). So we probably have to consider was is actually possible in a reliable way and what information would be helpful to the user if it isn't.

[paulbovbel]

I wouldn't consider this to be part of the profile feature.

That makes sense. I guess that warrants a separate ticket then.

[dirk-thomas]

Since it seems unlikely that anyone will spend effort on this feature (and catkin_tools development is anyway in "hibernation") I will go ahead and close this ticket for now. Please feel free to continue commenting on it.