Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nbin_uF64__jsk_pcl_ros__ubuntu_focal_amd64__binary takes longer than 120 minutes to build #2592

Open
sloretz opened this issue Apr 26, 2021 · 2 comments

Comments

@sloretz
Copy link

sloretz commented Apr 26, 2021

This job on the buildfarm is currently failing to build because it times out. Got any idea why?
https://build.ros.org/view/Nbin_uF64/job/Nbin_uF64__jsk_pcl_ros__ubuntu_focal_amd64__binary

Here is one that succeeds and one that fails

It looks like the job that fails has no console output for 30 minutes
01:30:41.885 [ 36%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/voxel_grid_large_scale_nodelet.cpp.o
01:30:41.886 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/voxel_grid_large_scale_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/voxel_grid_large_scale_nodelet.cpp
01:31:13.608 [ 36%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/ppf_registration_nodelet.cpp.o
01:31:13.609 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/ppf_registration_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/ppf_registration_nodelet.cpp
01:32:23.018 [ 37%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/target_adaptive_tracking_nodelet.cpp.o
01:32:23.029 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/target_adaptive_tracking_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/target_adaptive_tracking_nodelet.cpp
02:00:00.175 Build timed out (after 120 minutes). Marking the build as failed.
The job that succeeds gets to that same spot 30 minute earlier, and compiles it much faster
00:57:17.884 [ 36%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/voxel_grid_large_scale_nodelet.cpp.o
00:57:17.885 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/voxel_grid_large_scale_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/voxel_grid_large_scale_nodelet.cpp
00:57:48.258 [ 36%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/ppf_registration_nodelet.cpp.o
00:57:48.258 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/ppf_registration_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/ppf_registration_nodelet.cpp
00:58:47.274 [ 37%] Building CXX object CMakeFiles/jsk_pcl_ros.dir/src/target_adaptive_tracking_nodelet.cpp.o
00:58:47.278 ccache /usr/lib/ccache/c++  -DDISABLE_LIBUSB_1_0 -DDISABLE_OPENNI -DDISABLE_OPENNI2 -DDISABLE_PCAP -DDISABLE_PNG -DROSCONSOLE_BACKEND_LOG4CXX -DROS_BUILD_SHARED_LIBS=1 -DROS_PACKAGE_NAME=\"jsk_pcl_ros\" -Djsk_pcl_ros_EXPORTS -Dqh_QHpointer -DvtkRenderingContext2D_AUTOINIT="1(vtkRenderingContextOpenGL2)" -DvtkRenderingCore_AUTOINIT="3(vtkInteractionStyle,vtkRenderingFreeType,vtkRenderingOpenGL2)" -I/tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/include -I/opt/ros/noetic/include -I/opt/ros/noetic/share/xmlrpcpp/cmake/../../../include/xmlrpcpp -I/usr/include/bullet -isystem /usr/include/vtk-7.1 -isystem /usr/include/freetype2 -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/BEFORE -isystem /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/obj-x86_64-linux-gnu/devel/include -isystem /usr/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/pcl-1.10  -std=c++14  -fopenmp -fPIC   -O2 -g -o CMakeFiles/jsk_pcl_ros.dir/src/target_adaptive_tracking_nodelet.cpp.o -c /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/target_adaptive_tracking_nodelet.cpp
00:59:59.983 /tmp/binarydeb/ros-noetic-jsk-pcl-ros-1.2.15/src/target_adaptive_tracking_nodelet.cpp: In member function ‘void jsk_pcl_ros::TargetAdaptiveTracking::supervoxelSegmentation(pcl::PointCloud<pcl::PointXYZRGB>::Ptr, std::map<unsigned int, boost::shared_ptr<pcl::Supervoxel<pcl::PointXYZRGB> > >&, std::multimap<unsigned int, unsigned int>&, float)’:

Old similar issues:
#1421
ros-infrastructure/buildfarm#220

@k-okada
Copy link
Member

k-okada commented Apr 27, 2021 via email

@sloretz
Copy link
Author

sloretz commented May 4, 2021

that's interesting, are there any changes on buildfarm? for example changed aws instance type?

It looks like there have been no changes to the buildfarm infrastructure. My coworker @nuclearsandwich says the job probably needs more RAM/CPU than we allocate for buildfarm jobs (1/4th of 4vCPU/8G RAM), but if by chance there are no other jobs on the same agent then it is able to use extra resources and complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants