Race condition compiling message packages with RTI Connect typesupport enabled (only with rtiddsgen_server)

[tfoote]

This has shown itself on the buildfarm many times and has been resolved by an immediate rebuild with no changes.

Failure: http://build.ros2.org/job/Bbin_uB64__rcl_interfaces__ubuntu_bionic_amd64__binary/13/console

17:39:37 [ 30%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
17:39:37 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
17:39:40 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rcl_interfaces/msg/dds_connext/IntraProcessMessage_.idl doesn't exist. Please make sure that you have specified the correct path.
17:39:40 INFO com.rti.ndds.nddsgen.Main Done (failures)
17:39:40 Traceback (most recent call last):
17:39:40   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
17:39:40     sys.exit(main())
17:39:40   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
17:39:40     service_specs,
17:39:40   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
17:39:40     subprocess.check_call(cmd)
17:39:40   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
17:39:40     raise CalledProcessError(retcode, cmd)
17:39:40 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-d', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/rcl_interfaces/msg/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rcl_interfaces/msg/dds_connext/IntraProcessMessage_.idl']' returned non-zero exit status 255.
17:39:40 make[4]: *** [rosidl_typesupport_connext_cpp/rcl_interfaces/msg/intra_process_message__rosidl_typesupport_connext_cpp.hpp] Error 1
17:39:40 make[4]: *** Deleting file 

Earlier: http://build.ros2.org/job/Bbin_uB64__rcl_interfaces__ubuntu_bionic_amd64__binary/11/console

19:39:02 [ 30%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
19:39:02 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
19:39:04 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rcl_interfaces/msg/dds_connext/IntraProcessMessage_.idl doesn't exist. Please make sure that you have specified the correct path.
19:39:04 INFO com.rti.ndds.nddsgen.Main Done (failures)
19:39:04 Traceback (most recent call last):
19:39:04   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
19:39:04     sys.exit(main())
19:39:04   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
19:39:04     service_specs,
19:39:04   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
19:39:04     subprocess.check_call(cmd)
19:39:04   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
19:39:04     raise CalledProcessError(retcode, cmd)
19:39:04 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-d', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/rcl_interfaces/msg/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-rcl-interfaces-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rcl_interfaces/msg/dds_connext/IntraProcessMessage_.idl']' returned non-zero exit status 255.
19:39:04 CMakeFiles/rcl_interfaces__rosidl_typesupport_connext_cpp.dir/build.make:90: recipe for target 'rosidl_typesupport_connext_cpp/rcl_interfaces/msg/intra_process_message__rosidl_typesupport_connext_cpp.hpp' failed
19:39:04 make[4]: *** [rosidl_typesupport_connext_cpp/rcl_interfaces/msg/intra_process_message__rosidl_typesupport_connext_cpp.hpp] Error 1
19:39:04 make[4]: *** Deleting file 

http://build.ros2.org/view/Bbin_uB64/job/Bbin_uB64__rosgraph_msgs__ubuntu_bionic_amd64__binary/18/console

17:39:26 [  7%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
17:39:26 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
17:39:26 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rosgraph_msgs/msg/dds_connext/Clock_.idl doesn't exist. Please make sure that you have specified the correct path.
17:39:26 INFO com.rti.ndds.nddsgen.Main Done (failures)
17:39:26 Traceback (most recent call last):
17:39:26   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
17:39:26     sys.exit(main())
17:39:26   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
17:39:26     service_specs,
17:39:26   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
17:39:26     subprocess.check_call(cmd)
17:39:26   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
17:39:26     raise CalledProcessError(retcode, cmd)
17:39:26 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-I', '/opt/ros/bouncy/share', '-d', '/tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/rosgraph_msgs/msg/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-rosgraph-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/rosgraph_msgs/msg/dds_connext/Clock_.idl']' returned non-zero exit status 255.
17:39:26 CMakeFiles/rosgraph_msgs__rosidl_typesupport_connext_cpp.dir/build.make:72: recipe for target 'rosidl_typesupport_connext_cpp/rosgraph_msgs/msg/clock__rosidl_typesupport_connext_cpp.hpp' failed

http://build.ros2.org/view/Bbin_uB64/job/Bbin_uB64__std_srvs__ubuntu_bionic_amd64__binary/13/console

17:36:09 [ 11%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
17:36:09 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
17:36:10 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/std_srvs/srv/dds_connext/Empty_Request_.idl doesn't exist. Please make sure that you have specified the correct path.
17:36:10 INFO com.rti.ndds.nddsgen.Main Done (failures)
17:36:10 Traceback (most recent call last):
17:36:10   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
17:36:10     sys.exit(main())
17:36:10   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
17:36:10     service_specs,
17:36:10   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
17:36:10     subprocess.check_call(cmd)
17:36:10   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
17:36:10     raise CalledProcessError(retcode, cmd)
17:36:10 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-d', '/tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/std_srvs/srv/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-std-srvs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/std_srvs/srv/dds_connext/Empty_Request_.idl']' returned non-zero exit status 255.
17:36:10 CMakeFiles/std_srvs__rosidl_typesupport_connext_cpp.dir/build.make:75: recipe for target 'rosidl_typesupport_connext_cpp/std_srvs/srv/empty__rosidl_typesupport_connext_cpp.hpp' failed

http://build.ros2.org/view/Bbin_uB64/job/Bbin_uB64__cartographer_ros_msgs__ubuntu_bionic_amd64__binary/8/console

20:56:10 [ 39%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
20:56:10 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
20:56:13 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/cartographer_ros_msgs/msg/dds_connext/SensorTopics_.idl doesn't exist. Please make sure that you have specified the correct path.
20:56:13 INFO com.rti.ndds.nddsgen.Main Done (failures)
20:56:13 Traceback (most recent call last):
20:56:13   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
20:56:13     sys.exit(main())
20:56:13   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
20:56:13     service_specs,
20:56:13   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
20:56:13     subprocess.check_call(cmd)
20:56:13   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
20:56:13     raise CalledProcessError(retcode, cmd)
20:56:13 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-I', '/opt/ros/bouncy/share', '-d', '/tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/cartographer_ros_msgs/msg/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-cartographer-ros-msgs-2.1.1/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/cartographer_ros_msgs/msg/dds_connext/SensorTopics_.idl']' returned non-zero exit status 255.
20:56:13 make[4]: *** [rosidl_typesupport_connext_cpp/cartographer_ros_msgs/msg/sensor_topics__rosidl_typesupport_connext_cpp.hpp] Error 1
20:56:13 make[4]: *** Deleting file 'rosidl_typesupport_connext_cpp/cartographer_ros_msgs/msg/sensor_topics__rosidl_typesupport_connext_cpp.hpp'

http://build.ros2.org/view/Bbin_uB64/job/Bbin_uB64__visualization_msgs__ubuntu_bionic_amd64__binary/15/console

20:55:52 [ 35%] Generating C++ type support for RTI Connext (using '/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server')
20:55:52 /usr/bin/python3 /opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp --generator-arguments-file /tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp__arguments.json --dds-interface-base-path /tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl --idl-pp /opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server
20:55:55 ERROR com.rti.ndds.nddsgen.Main Fail:  com.rti.ndds.nddsgen.Main$ArgumentException: File /tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/visualization_msgs/msg/dds_connext/ImageMarker_.idl doesn't exist. Please make sure that you have specified the correct path.
20:55:55 INFO com.rti.ndds.nddsgen.Main Done (failures)
20:55:55 Traceback (most recent call last):
20:55:55   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 61, in <module>
20:55:55     sys.exit(main())
20:55:55   File "/opt/ros/bouncy/lib/rosidl_typesupport_connext_cpp/rosidl_typesupport_connext_cpp", line 56, in main
20:55:55     service_specs,
20:55:55   File "/opt/ros/bouncy/lib/python3.6/site-packages/rosidl_typesupport_connext_cpp/__init__.py", line 89, in generate_dds_connext_cpp
20:55:55     subprocess.check_call(cmd)
20:55:55   File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
20:55:55     raise CalledProcessError(retcode, cmd)
20:55:55 subprocess.CalledProcessError: Command '['/opt/rti.com/rti_connext_dds-5.3.1/bin/rtiddsgen_server', '-I', '/tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl', '-I', '/opt/ros/bouncy/share', '-d', '/tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_typesupport_connext_cpp/visualization_msgs/msg/dds_connext', '-language', 'C++', '-namespace', '-update', 'typefiles', '-unboundedSupport', '/tmp/binarydeb/ros-bouncy-visualization-msgs-0.5.0/obj-x86_64-linux-gnu/rosidl_generator_dds_idl/visualization_msgs/msg/dds_connext/ImageMarker_.idl']' returned non-zero exit status 255.
20:55:55 CMakeFiles/visualization_msgs__rosidl_typesupport_connext_cpp.dir/build.make:140: recipe for target 'rosidl_typesupport_connext_cpp/visualization_msgs/msg/image_marker__rosidl_typesupport_connext_cpp.hpp' failed
20:55:55 make[4]: *** [rosidl_typesupport_connext_cpp/visualization_msgs/msg/image_marker__rosidl_typesupport_connext_cpp.hpp] Error 1
20:55:55 make[4]: *** Deleting file 'rosidl_typesupport_connext_cpp/visualization_msgs/msg/image_marker__rosidl_typesupport_connext_cpp.hpp'

Also: http://build.ros2.org/job/Bbin_uB64__tf2_msgs__ubuntu_bionic_amd64__binary/8/console

[dirk-thomas]

For now we have a workaround in place which falls back to the non-server code generator in case the server-based code generator fails: see #39.

@neil-rti FYI this is a severe bug in rtiddsgen_server.

[neil-rti]

Thanks Dirk -- will look into it.

[neil-rti]

This is very likely to be caused by an issue that is present in 5.3 codeGen, and has been fixed in 6.0.
The 6.0 codeGen can't be used in 5.3 builds -- so while this isn't an immediate fix, this issue will probably be resolved with the rmw_connext that supports 6.0 (we're taking a closer look at this now).

[clalancette]

Closing, since with Foxy now being End-of-Life, this repository is no longer used.