Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Buildfarmer log 2024/01/08 - 2024/01/15 #21

Closed
github-actions bot opened this issue Jan 8, 2024 · 9 comments
Closed

Buildfarmer log 2024/01/08 - 2024/01/15 #21

github-actions bot opened this issue Jan 8, 2024 · 9 comments
Labels
buildfarmer-log Log of issues in the buildfarm

Comments

@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2024

Previous log #18
@github-actions github-actions bot added the buildfarmer-log Log of issues in the buildfarm label Jan 8, 2024
@github-actions GitHub Actions
Copy link
Contributor Author

github-actions bot commented Jan 8, 2024

Jan 08

Build regressions 
nightly_linux_humble_coverage|668|2024-01-08 05:34:23.147000|Jenkins channel closing down (Agent disconnected)|build
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_win_rep
- Failing 84.62% in 11 out of 13 builds
	rclcpp.TestGuardCondition.construction_and_destruction
	projectroot.test.rclcpp.test_guard_condition

Issue #2 in gz_rendering-main-win
- Failing 100.0% in 4 out of 4 builds
	GizmoVisualTest.LookAt

Issue #3 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 100.0% in 4 out of 4 builds
	.gz-sim.python.test.testFixture_TEST

Issue #4 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	test_rclcpp.TestNNodesAfterShutdown.test_10_nodes
	projectroot.test_n_nodes__rmw_fastrtps_cpp

Issue #5 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 50.0% in 2 out of 4 builds
	INTEGRATION_gpu_lidar.test_ran

Issue #6 in gz_sim-ci-ign-gazebo3-homebrew-amd64
- Failing 50.0% in 2 out of 4 builds
	ServerRepeat/AckermannSteeringTest.TfPublishes/0 (known issue in: https://github.com/gazebosim/gz-sim/issues/1993 - CLOSED | https://github.com/gazebosim/gz-sim/issues/1993 - OPEN)
	  (also in: "gz_sim-ci-ign-gazebo3-focal-amd64" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/1993 - CLOSED | https://github.com/gazebosim/gz-sim/issues/1993 - OPEN)

Issue #7 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #8 in gz_common-ign-common4-win
- Failing 25.0% in 1 out of 4 builds
	PluginSpecialization.AccessTime

Issue #9 in gz_transport-ci-gz-transport13-jammy-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_Recorder_TEST.test_ran

Issue #10 in gz_transport-ign-transport11-win
- Failing 25.0% in 1 out of 4 builds
	playback.ReplayStep (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-ign-transport8-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-13-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-12-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)

Issue #11 in gz_transport-main-win
- Failing 25.0% in 1 out of 4 builds
	twoProcSrvCallWithoutOutput.ServiceList

Issue #12 in nightly_linux_debug
- Failing 6.25% in 1 out of 16 builds
	rosbag2_transport.xmllint.xunit.missing_result
	projectroot.xmllint

Issue #13 in nightly_linux_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.src.core.ddsc.tests.CUnit_ddsc_discstress_create_writer

@github-actions GitHub Actions
Copy link
Contributor Author

github-actions bot commented Jan 9, 2024

Jan 09

Build regressions
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_win_rep
- Failing 83.33% in 10 out of 12 builds
	rclcpp.TestGuardCondition.construction_and_destruction
	projectroot.test.rclcpp.test_guard_condition

Issue #2 in gz_rendering-main-win
- Failing 100.0% in 4 out of 4 builds
	GizmoVisualTest.LookAt

Issue #3 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 100.0% in 4 out of 4 builds
	.gz-sim.python.test.testFixture_TEST

Issue #4 in nightly_win_rep
- Failing 25.0% in 3 out of 12 builds
	test_rclcpp.TestNNodesAfterShutdown.test_10_nodes
	projectroot.test_n_nodes__rmw_fastrtps_cpp

Issue #5 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 50.0% in 2 out of 4 builds
	INTEGRATION_gpu_lidar.test_ran

Issue #6 in gz_sim-ci-ign-gazebo3-homebrew-amd64
- Failing 50.0% in 2 out of 4 builds
	ServerRepeat/AckermannSteeringTest.TfPublishes/0 (known issue in: https://github.com/gazebosim/gz-sim/issues/1993 - CLOSED | https://github.com/gazebosim/gz-sim/issues/1993 - OPEN)
	  (also in: "gz_sim-ci-ign-gazebo3-focal-amd64" job) (known issue in: https://github.com/gazebosim/gz-sim/issues/1993 - CLOSED | https://github.com/gazebosim/gz-sim/issues/1993 - OPEN)

Issue #7 in nightly_win_rep
- Failing 16.67% in 2 out of 12 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #8 in gz_common-ign-common4-win
- Failing 25.0% in 1 out of 4 builds
	PluginSpecialization.AccessTime

Issue #9 in gz_transport-ci-gz-transport13-jammy-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_Recorder_TEST.test_ran

Issue #10 in gz_transport-ign-transport11-win
- Failing 25.0% in 1 out of 4 builds
	playback.ReplayStep (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-ign-transport8-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-13-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-12-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)

Issue #11 in gz_transport-main-win
- Failing 25.0% in 1 out of 4 builds
	twoProcSrvCallWithoutOutput.ServiceList

Issue #12 in nightly_linux_debug
- Failing 6.67% in 1 out of 15 builds
	rosbag2_transport.xmllint.xunit.missing_result
	projectroot.xmllint

Issue #13 in nightly_linux_repeated
- Failing 6.67% in 1 out of 15 builds
	projectroot.src.core.ddsc.tests.CUnit_ddsc_discstress_create_writer

@Crola1702
Copy link
Contributor

Crola1702 commented Jan 9, 2024 •
edited
Loading

Windows repetated | Test regressions | Investigating 🕵️

Reference build: https://ci.ros2.org/view/nightly/job/nightly_win_rep/3228/#showFailuresLink

Test regressions:

Log output:

unknown file
SEH exception with code 0xc0000005 thrown in the test body.

First time it happened: 2023-05-31 build 3006 (since then it's been happening almost consistently)

This problem might have been fixed by:

Reproducing the issue: Build Status

@Crola1702 Crola1702 mentioned this issue Jan 9, 2024
Merged
@github-actions GitHub Actions
Copy link
Contributor Author

Jan 10

Build regressions 
nightly_linux_repeated|3325|2024-01-10 07:22:04.922000|Jenkins channel closing down (Agent disconnected)|test
nightly_linux-aarch64_debug|2636|2024-01-10 04:00:01.302000|Jenkins channel closing down (Agent disconnected)|build
nightly_win_deb|2968|2024-01-10 04:00:01.337000|Number of failed test exceeds the threshold|test
Rci__nightly-fastrtps-dynamic_ubuntu_jammy_amd64|680|2024-01-09 13:39:45.673000|Unknown Reason|import repositories
gz-msgs11-debbuilder|87|2024-01-10 10:34:14.767000|Error in configuration dependencies|create deb packages
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in gz_rendering-main-win
- Failing 100.0% in 3 out of 3 builds
	GizmoVisualTest.LookAt

Issue #2 in gz_sim-ci-ign-gazebo6-focal-amd64
- Failing 75.0% in 3 out of 4 builds
	UNIT_Server_TEST.test_ran

Issue #3 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 100.0% in 3 out of 3 builds
	.gz-sim.python.test.testFixture_TEST

Issue #4 in gz_sim-ci-ign-gazebo3-focal-amd64
- Failing 66.67% in 2 out of 3 builds
	INTEGRATION_gpu_lidar.test_ran

Issue #5 in gz_common-ign-common4-win
- Failing 25.0% in 1 out of 4 builds
	PluginSpecialization.AccessTime

Issue #6 in gz_gui-ci-ign-gui6-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_ImageDisplay_TEST.test_ran

Issue #7 in gz_transport-ci-gz-transport13-jammy-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_Recorder_TEST.test_ran

Issue #8 in gz_transport-ign-transport11-win
- Failing 25.0% in 1 out of 4 builds
	playback.ReplayStep (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-ign-transport8-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-13-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)
	  (also in: "gz_transport-12-win" job) (known issue in: https://github.com/gazebosim/gz-transport/issues/195 - CLOSED)

Issue #9 in gz_transport-main-win
- Failing 25.0% in 1 out of 4 builds
	twoProcSrvCallWithoutOutput.ServiceList

Issue #10 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	projectroot.test_tutorial_add_two_ints_server_add_two_ints_client_async__rmw_fastrtps_cpp

Issue #11 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	demo_nodes_cpp.TestExecutablesTutorialAfterShutdown.test_last_process_exit_code
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_linux_repeated" job)

@Crola1702
Copy link
Contributor

UNIT_Server_TEST.test_ran reporetd in gazebosim/gz-sim#2282

@github-actions GitHub Actions
Copy link
Contributor Author

Jan 11

Build regressions 
packaging_linux|3292|2024-01-11 08:47:50.568000|Jenkins channel closing down (Agent disconnected)|build underlay packages
packaging_linux-rhel|1794|2024-01-11 09:00:35.751000|Jenkins channel closing down (Agent disconnected)|build underlay packages
nightly_win_deb|2969|2024-01-11 04:00:01.409000|Packages lttngpy  Failed|build
packaging_windows_debug|2070|2024-01-11 04:33:37.642000|Packages lttngpy  Failed|build underlay packages
gz-msgs11-debbuilder|88|2024-01-11 10:17:36.071000|Error in configuration dependencies|create deb packages
gz_rendering-main-win|11|2024-01-10 22:06:11.409000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for gz-rendering9
gz_physics-6-win|13|2024-01-10 20:06:11.761000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for gz-physics6
gz_physics-ign-physics5-win|11|2024-01-10 21:06:04.410000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for ignition-physics5
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 100.0% in 4 out of 4 builds
	.gz-sim.python.test.testFixture_TEST (known issue in: https://github.com/gazebosim/gz-sim/issues/1887 - OPEN | https://github.com/gazebosim/gz-sim/issues/2249 - OPEN)

Issue #2 in nightly_win_rep
- Failing 23.08% in 3 out of 13 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #3 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	projectroot.test_tutorial_add_two_ints_server_add_two_ints_client_async__rmw_fastrtps_cpp

Issue #4 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	demo_nodes_cpp.TestExecutablesTutorialAfterShutdown.test_last_process_exit_code
	demo_nodes_cpp.TestExecutablesTutorial.test_processes_output
	  (also in: "nightly_linux_repeated" job)

Issue #5 in gz_common-ign-common4-win
- Failing 25.0% in 1 out of 4 builds
	PluginSpecialization.AccessTime

Issue #6 in gz_fuel_tools-ci-gz-fuel-tools9-jammy-amd64
- Failing 20.0% in 1 out of 5 builds
	UNIT_gz_TEST.test_ran

Issue #7 in gz_gui-ci-ign-gui6-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_ImageDisplay_TEST.test_ran

Issue #8 in gz_transport-ci-gz-transport13-jammy-amd64
- Failing 33.33% in 1 out of 3 builds
	UNIT_Recorder_TEST.test_ran

Issue #9 in gz_transport-main-win
- Failing 33.33% in 1 out of 3 builds
	twoProcSrvCallWithoutOutput.ServiceList

Issue #10 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	projectroot.test_tutorial_talker_listener__rmw_fastrtps_cpp
	projectroot.test_tutorial_content_filtering_publisher_content_filtering_subscriber__rmw_fastrtps_cpp
	projectroot.test_tutorial_add_two_ints_server_add_two_ints_client__rmw_fastrtps_cpp

@github-actions GitHub Actions
Copy link
Contributor Author

Jan 12

Build regressions 
nightly_win_deb|2970|2024-01-12 04:00:01.274000|Packages lttngpy  Failed|build
packaging_windows_debug|2072|2024-01-12 06:26:21.494000|Packages lttngpy  Failed|build underlay packages
gz-msgs11-debbuilder|89|2024-01-12 09:46:10.401000|Error in configuration dependencies|create deb packages
gz_physics-main-win|15|2024-01-11 23:39:11.098000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for gz-physics8
gz_cmake-3-win|13|2024-01-12 00:06:09.010000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for gz-cmake3
gz_transport-ign-transport11-win|12|2024-01-11 14:06:12.352000|Jenkins channel closing down (Agent disconnected)|colcon compilation with tests for ignition-transport11
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in gz_sim-ci-ign-gazebo6-homebrew-amd64
- Failing 100.0% in 4 out of 4 builds
	.gz-sim.python.test.testFixture_TEST (known issue in: https://github.com/gazebosim/gz-sim/issues/1887 - OPEN | https://github.com/gazebosim/gz-sim/issues/2249 - OPEN)

Issue #2 in gz_fuel_tools-ci-gz-fuel-tools9-jammy-amd64
- Failing 20.0% in 1 out of 5 builds
	UNIT_gz_TEST.test_ran

Issue #3 in gz_gui-ci-ign-gui6-focal-amd64
- Failing 25.0% in 1 out of 4 builds
	UNIT_ImageDisplay_TEST.test_ran

Issue #4 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	rclcpp.TestClockSleep.sleep_until_basic_system

Issue #5 in nightly_linux-aarch64_repeated
- Failing 6.25% in 1 out of 16 builds
	projectroot.test.rclcpp.test_time
	  (also in: "nightly_win_rep" job)
	  (also in: "Ici__nightly-connext_ubuntu_jammy_amd64" job)

@github-actions GitHub Actions
Copy link
Contributor Author

Jan 13

Build regressions 
gz-msgs11-debbuilder|90|2024-01-13 09:48:07.777000|Error in configuration dependencies|create deb packages
gz_gui-ci-main-jammy-amd64|30|2024-01-13 03:58:44.930000|Jenkins channel closing down (Agent disconnected)|compiling
gz_sim-ci-gz-sim7-focal-amd64|28|2024-01-13 03:06:04.487000|Jenkins channel closing down (Agent disconnected)|compiling
gz_gui-ci-ign-gui6-focal-amd64|27|2024-01-13 05:11:11.686000|Unknown Reason|NVIDIA MISMATCH RECOVERY
gz_gui-ci-ign-gui6-focal-amd64|26|2024-01-13 05:06:05.424000|Unknown Reason|NVIDIA MISMATCH RECOVERY
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_win_rep
- Failing 61.54% in 8 out of 13 builds
	rclcpp.TestGuardCondition.construction_and_destruction
	projectroot.test.rclcpp.test_guard_condition

Issue #2 in gz_sim-main-win
- Failing 40.0% in 2 out of 5 builds
	INTEGRATION_save_world.test_ran (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)

Issue #3 in gz_fuel_tools-ci-gz-fuel-tools9-jammy-amd64
- Failing 20.0% in 1 out of 5 builds
	UNIT_gz_TEST.test_ran

Issue #4 in gz_sim-ign-gazebo6-win
- Failing 20.0% in 1 out of 5 builds
	INTEGRATION_multiple_servers.test_ran

Issue #5 in nightly_win_rep
- Failing 7.69% in 1 out of 13 builds
	test_communication.TestPublisherSubscriberAfterShutdown.test_processes_finished_gracefully[Arrays]
	test_communication.TestPublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time[Arrays]
	robot_state_publisher.MovingJointTestAfterShutdown.test_exit_code
	projectroot.test_two_links_moving_joint_launch.py
	projectroot.test_publisher_subscriber__rclpy__rmw_fastrtps_cpp (known issue in: https://github.com/ros2/system_tests/issues/496 - CLOSED)

@github-actions GitHub Actions
Copy link
Contributor Author

Jan 14

Build regressions 
nightly_win_deb|2972|2024-01-14 04:00:01.743000|Number of failed test exceeds the threshold|test
gz-msgs11-debbuilder|91|2024-01-14 09:50:30.668000|Error in configuration dependencies|create deb packages
Test regressions 
Excluding 'performance, connext, fastrtps-dynamic' jobs
Issue #1 in nightly_win_rep
- Failing 30.77% in 4 out of 13 builds
	rosidl_typesupport_introspection_tests.MultiNestedMessageIntrospectionTest/0.CanWriteTypeErasedMessage
	projectroot.test_multi_nested_message_introspection
	  (also in: "nightly_linux-aarch64_repeated" job)

Issue #2 in gz_sim-main-win
- Failing 40.0% in 2 out of 5 builds
	INTEGRATION_save_world.test_ran (known issue in: https://github.com/gazebosim/gz-sim/issues/2177 - CLOSED)

Issue #3 in nightly_win_rep
- Failing 15.38% in 2 out of 13 builds
	test_rclcpp.TestNNodesAfterShutdown.test_10_nodes
	projectroot.test_n_nodes__rmw_fastrtps_cpp

Issue #4 in gz_fuel_tools-ci-gz-fuel-tools9-jammy-amd64
- Failing 20.0% in 1 out of 5 builds
	UNIT_gz_TEST.test_ran

Issue #5 in gz_sim-7-win
- Failing 20.0% in 1 out of 5 builds
	INTEGRATION_multiple_servers.test_ran
	  (also in: "gz_sim-ign-gazebo6-win" job)
	  (also in: "gz_sim-7-win" job)

Issue #6 in nightly_linux_repeated
- Failing 6.67% in 1 out of 15 builds
	ros2topic.ros2topic.test.test_cli.test_cli (known issue in: https://github.com/ros2/ros2cli/issues/861 - CLOSED)
	  (also in: "nightly_linux-rhel_repeated" job) (known issue in: https://github.com/ros2/ros2cli/issues/861 - CLOSED)
	  (also in: "nightly_linux-aarch64_repeated" job) (known issue in: https://github.com/ros2/ros2cli/issues/861 - CLOSED)

Issue #7 in nightly_linux_repeated
- Failing 6.67% in 1 out of 15 builds
	rcl_logging_spdlog.LoggingTest.init_old_flushing_behavior
	projectroot.test_logging_interface

@github-actions github-actions bot mentioned this issue Jan 15, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
buildfarmer-log Log of issues in the buildfarm
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Crola1702