Failure passing MAKEFLAGS to catkin_make_isolated on FreeBSD

[dodsonmg]

I'm attempting to install ROS Kinetic on FreeBSD 12.1 (at my peril).

I'm following the 'from source' instructions at:
http://wiki.ros.org/kinetic/Installation/Source

I am performing step 2.1.3 (Building the catkin workspace), which directs invoking catkin_make_isolated as follows:
./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release

I get the following output:

==> Processing catkin package: 'catkin'
...
==> make -j4 -l4 in '$PATH/ros_catkin_ws/build_isolated/catkin'
...
<== Failed to process package 'catkin':
Command '['make', '-j4', '-l4']' returned non-zero exit status 2

This seems to be due to FreeBSD make not having the loading average flag -l.

I have tried both setting the MAKEFLAGS and ROS_PARALLEL_JOBS environment variables to avoid setting the -l flag. E.g., set ROS_PARALLEL_JOBS='-j4'.

The result is that it sends a flag -pn to make, as follows:

==> Processing catkin package: 'catkin'
...
==> make -j4 in '$PATH/ros_catkin_ws/build_isolated/catkin'
...
<== Failed to process package 'catkin':
Command '['make', '-pn']' returned non-zero exit status 2

Interestingly, it appears that it's invoking make -j4, but then says that the actual flag sent to make is -pn.

I have also attempted to set the flags when invoking catkin_make_isolated, but get the same result (with -pn). For example:
./src/catkin/bin/catkin_make_isolated --install -DCMAKE_BUILD_TYPE=Release -j4

[dirk-thomas]

The make flags -j and -j are only added conditionally, see

catkin/python/catkin/builder.py

Lines 323 to 325 in 4d44675

	jobs = multiprocessing.cpu_count()
	make_args.append('-j{0}'.format(jobs))
	make_args.append('-l{0}'.format(jobs))

. If you pass job flags in any other way the logic won't be triggered (and doesn't add the -l flag which isn't supported on your platform).

Possible options:

• you pass --make-args -j4
  • (the argument must contain a valid job argument otherwise the default logic might get triggered)
• set MAKEFLAGS=-j4 (the command won't show these since they are set as an environment variable)
  • (the argument must contain a valid job argument otherwise the default logic might get triggered)
• set ROS_PARALLEL_JOBS=-j4
  • these argument are passed as is to the command and therefore will be shown and it doesn't matter what they contain - it always overrides the default logic

The -pn argument is likely coming from this invocation:

catkin/python/catkin/builder.py

Line 458 in 4d44675

output = run_command(['make', '-pn'], path, quiet=True)

which is trying to determine the available targets. Maybe that option is not supported by make on your platform either? If that is the case can you please post the version of make you are using. Are there any alternative options in your make version to enumerate the available targets?

[dodsonmg]

That's very helpful. I was conflating two problems.

When I rely on the default job flags (-j4 -l4) the system chokes; however, I am able to pass the -j flag successfully with all of the options you mention.

Once I get through the first choke point, I fall down when I get to -pn.

FreeBSD make does not have a flag equivalent to -p, as far as I can tell.

That said, there is a FreeBSD port of GNU make (gmake), which has all the expected flags (including -l and -p).

Can I force catkin to invoke gmake instead of make?

[dirk-thomas]

Can I force catkin to invoke gmake instead of make?

Likely not without code changes. The make name is hard coded in several places.

[dodsonmg]

My questions appear to be approaching a very basic level:

I can make the changes in builder.py, but this doesn't seem to have any effect when I rebuild catkin.

As a test, I simply replaced the one make invocation on line 458 of builder.py with gmake, so I should get the command gmake -pn when I invoke catkin_make or catkin_make_isolated (right?).

I then went through the normal cmake process to build catkin (assume catkin is the only directory under src):

mkdir build install
cd build
cmake ../src -DCMAKE_INSTALL_PREFIX=../install
make
make install

This proceeds just fine, and I have catkin_make and catkin_make_isolated in install/bin, as expected.

Subsequent attempts to invoke catkin_make or catkin_make_isolated from my install directory, still appear to be using the command make -pn instead of gmake -pn, which implies that the modifications to builder.py were not incorporated into the binary.

Is this because it didn't actually rebuild catkin_make or catkin_make_isolated, but just used the binaries that exist in /src/catkin/bin?

How do I rebuild the binaries with the modified builder.py?

This seems like a trivial question, but I've not previously tried to modify and rebuild a build system...

[dirk-thomas]

Subsequent attempts to invoke catkin_make or catkin_make_isolated from my install directory, still appear to be using the command make -pn instead of gmake -pn, which implies that the modifications to builder.py were not incorporated into the binary.

Have you set any environment variables pointing to the new install space after you built catkin? If not you are probably using the Python code of catkin from a different location. You can output the path of the catkin.builder module being found and used by Python like this:

python -c "import catkin.builder; print(catkin.builder.__file__)"

Also you shouldn't need to build catkin separately. If the modified version is just in the same workspace with all the other packages you are building you should be able to just do a normal build of that workspace using catkin_make_isolated from src/catkin/bin.

[dodsonmg]

Have you set any environment variables pointing to the new install space after you built catkin? If not you are probably using the Python code of catkin from a different location. You can output the path of the catkin.builder module being found and used by Python like this:

python -c "import catkin.builder; print(catkin.builder.__file__)"

I had not, and this was the problem. Fixed.

Also you shouldn't need to build catkin separately. If the modified version is just in the same workspace with all the other packages you are building you should be able to just do a normal build of that workspace using catkin_make_isolated from src/catkin/bin.

Thanks. I understand the structure better now.

[dirk-thomas]

It should be fairly easy to add support for a custom make executable to catkin if you are interested. I would suggest to replace the hard coded make name with a check if the environment variable MAKE_COMMAND is set if if yes, then use its value instead. That should only take a few lines of Python code and would allow and others to build on FreeBSD without the need to touch the source code.

[dodsonmg]

I'd be happy to try. I'm not very competent with git, but there's plenty of guidance and I'll read up on it.

Is the MAKE_COMMAND suggestion preferable to adding a --use-gmake option? The former is more flexible. The latter seems to be intuitively parallel to the --use-nmake and --use-ninja options already provided.

Anyway, I finally have the bare bones ROS installed and running simple examples on FreeBSD 12.1.

[dirk-thomas]

The existing options are in the form of --use-* since they also pass different arguments to CMake to change the generator.

I don't mind either or. Naming the new option --use-gmake sounds good to me too.