Unable to include headers from workspace packages that do not install the headers, when install is enabled

[ajthompson]

System Info

• Operating System: Linux odin 4.4.0-59-generic Metadata Files and Associated Verbs #80~14.04.1-Ubuntu SMP Fri Jan 6 18:02:02 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
• Python Version: Python 2.7.6
• Version of catkin_tools: catkin_tools 0.4.3 (C) 2014-2017 Open Source Robotics Foundation catkin_tools is released under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0) --- Using Python 2.7.6 (default, Oct 26 2016, 20:30:19) [GCC 4.8.4]
• ROS Distro: indigo

Build / Run Issue

Headers from packages that export their include folder, but do not install it, cannot be found when install is enabled, and using linked or isolated develspaces. Using a merged develspace still works.

• Works with catkin_make
• Works with catkin_make install
• Works with catkin_make_isolated --merge
• Works with catkin build (linked develspace)
• Works with catkin build (isolated develspace)
• Works with catkin build (merged develspace)
• Works with catkin build -p1 (linked develspace)
• Works with catkin build -p1 (isolated develspace)
• Works with catkin build -p1 (merged develspace)

Expected Behavior

Headers from packages that export their include directory should be accessible to dependent packages within the workspace when the workspace is configured to install, even if the package does not install the headers.

Actual Behavior

The project fails to build, because the secondary

Log gist: https://gist.github.com/ajthompson/e6f958a979de86254eb29e914966a0bf

Steps to Reproduce the Issue

cd /tmp
git clone https://github.com/ajthompson/catkin_build_include_failure.git test_ws
cd test_ws

catkin clean -y
catkin config --link-devel --install
catkin build -p1
# build fails

catkin clean -y
catkin config --no-install
catkin build -p1
# build succeeds

catkin clean -y
catkin config --isolate-devel --install
catkin build -p1
# build fails

catkin clean -y
catkin config --no-install
catkin build -p1
# build succeeds

catkin clean -y
catkin config --merge-devel --install
catkin build -p1
# build succeeds

catkin clean -y
catkin config --no-install
catkin build -p1
# build succeeds

catkin clean -y
catkin_make -DCMAKE_INSTALL_PREFIX=/tmp/test_ws/install install
# build succeeds

The workspace provided uses two simple packages - the first package, test, provides a simple header file defining a macro in include/test/test.h, and exports the include directory in the CMakeLists.txt. The second package, test2, has a build dependency for test, and contains a single C++ source file, consisting of a main function that returns the macro defined in test.h.

[wjwwood]

Expected Behavior

Headers from packages that export their include directory should be accessible to dependent packages within the workspace when the workspace is configured to install, even if the package does not install the headers.

I would expect the exact opposite. If you do the --install option, then the process is build A, install A, "source" A's install space, build B (which uses A), and so on. The devel spaces are not shared between packages. Therefore, if you do not install the headers, then they should not be found when using --install. In fact this is what you want if you're looking for issues with your package that might occur on the build farm where the same, build-install-source-build-next, process occurs.

[gavanderhoorn]

Also:

> - [x] Works with catkin_make

But surely not when using catkin_make install?

[ajthompson]

@gavanderhoorn
It works when using catkin_make install. I've added a step to the steps to reproduce the issue to demonstrate that.

@wjwwood
The particular use case here is a large workspace with both a full and minimal install configuration. Our normal usage is the minimal install, where many packages that produce only static libraries are omitted from the installation. These libraries are used to link against executables in many packages, but are not necessary for the installation.

[gavanderhoorn]

@gavanderhoorn
It works when using catkin_make install. I've added a step to the steps to reproduce the issue to demonstrate that.

O wait, of course it works: I assumed you were then building other packages on-top of the installed workspace. If the headers weren't present in the underlying workspace, things wouldn't have worked. But that is not the case here.

[wjwwood]

The particular use case here is a large workspace with both a full and minimal install configuration. Our normal usage is the minimal install, where many packages that produce only static libraries are omitted from the installation. These libraries are used to link against executables in many packages, but are not necessary for the installation.

I understand your use case, but it's not something we intended to support. AFAIK, the idea has always been that the only "right" way to share resources between two packages is by installing them. The fact that you can access them when using a merged devel space (which catkin_make always is because it is a single CMake instance) is an implementation detail, which happens to not hold when using catkin_make_isolated (without the --merge option) or catkin build (without the --merge-devel or --linked-devel options, which emulate a merged devel space, or when using the --install option).

I'm sorry, as I expect this isn't what you wanted to hear, but I'd currently classify this as "wont-fix".