👨‍🌾 Flaky issues with `launch_testing_ros`: `terminate called without an active exception` #324

Crola1702 · 2022-09-12T15:20:25Z

Bug report

Required Info:

Operating System:
- Ubuntu Jammy
Installation type:
- Source, buildfarm
Version or commit hash:
- Rolling, master
DDS implementation:
- Fast-RTPS, CycloneDDS

Steps to reproduce issue

Run Linux nightlies multiple times
Wait for failures

Expected behavior

Build successfully

Actual behavior

Failing in launch_testing_ros with this message:

pure virtual method called
terminate called without an active exception

Log output:

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 946, in run
      self._target(*self._args, **self._kwargs)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 279, in spin
      self.spin_once()
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 605, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
pure virtual method called
terminate called without an active exception
---
Failed   <<< launch_testing_ros [21.8s, exited with code -6]

Additional information

Reference builds on nightly_linux-aarch64: 2045, 2042

Reference build on Nightly Linux release: 2396

This issue might be related to #304

The text was updated successfully, but these errors were encountered:

Crola1702 · 2022-09-16T15:10:12Z

It appeared again today in Linux repeated job: 2778

Crola1702 · 2022-09-21T13:31:36Z

It appeared again today on Linux repeated job: 2783

This one only has terminate called without an active exception

Crola1702 · 2022-09-29T17:37:01Z

It appeared today on Linux repeated job: 2791, and Linux RHEL debug 1275

Both of them only have terminate called without an active exception

Crola1702 · 2022-10-24T14:21:37Z

It appeared today on Linux aarch64 repeated job: 2125

Log output:

pure virtual method called
terminate called without an active exception

Crola1702 · 2022-10-27T12:38:18Z

Now It appeared on Linux aarch64 debug: 2144

Log output:

terminate called without an active exception

Crola1702 · 2022-11-11T12:32:11Z

Happened today in Linux aarch64 debug: 2460

Log output:

terminate called without an active exception\
...
Failed   <<< launch_testing_ros [22.3s, exited with code -6]

Crola1702 · 2022-11-18T15:36:19Z

Today it happened in Rolling Release: 264

Log output:

test/examples/wait_for_topic_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)

  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
pure virtual method called
terminate called without an active exception
...
Failed   <<< launch_testing_ros [20.5s, exited with code -6]

Crola1702 · 2022-11-21T13:56:32Z

Yesterday it happened on Rolling Release: 267

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)

  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---- generated xml file: /tmp/ws/test_results/launch_testing_ros/pytest.xml ----
======================= 8 passed, 13 warnings in 18.95s ========================
terminate called without an active exception
--- stderr: launch_testing_ros

And today it happened again: 268

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)

  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---- generated xml file: /tmp/ws/test_results/launch_testing_ros/pytest.xml ----
======================= 8 passed, 13 warnings in 19.37s ========================
pure virtual method called
terminate called without an active exception

Crola1702 · 2022-12-14T13:30:25Z

Happened today in Linux Repeated: 2867

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-14 (spin)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/home/jenkins-agent/workspace/nightly_linux_repeated/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
terminate called without an active exception
---
Failed   <<< launch_testing_ros [23.2s, exited with code -6]

Crola1702 · 2023-01-06T14:15:21Z

Happened today in aarch Release: 2184

test/examples/wait_for_topic_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/home/jenkins-agent/workspace/nightly_linux-aarch64_release/ws/install/rclpy/lib/python3.10/site-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
pure virtual method called
terminate called without an active exception

Crola1702 · 2023-01-13T13:25:07Z

Happened today in aarch64 debug: 2229

Crola1702 · 2023-01-14T13:19:42Z

Today it happened for the first time in Rolling Nightly Release: 321

Log output:

--- stderr: launch_testing_ros

=============================== warnings summary ===============================
test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

test/examples/check_msgs_launch_test.py: 2 warnings
test/examples/check_node_launch_test.py: 2 warnings
test/examples/set_param_launch_test.py: 2 warnings
test/examples/talker_listener_launch_test.py: 2 warnings
test/examples/wait_for_topic_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)

  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
terminate called without an active exception
---
Failed   <<< launch_testing_ros [20.2s, exited with code -6]

Crola1702 · 2023-03-21T15:33:01Z

It's happening consistently in CycloneDDS since the last week.

launch_testing_ros test phase fails because terminate called without an active exception.

Reference build: https://build.ros2.org/view/Rci/job/Rci__nightly-cyclonedds_ubuntu_jammy_amd64/386/

Log output:

Starting >>> launch_testing_ros
============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.2.2, pluggy-1.0.0
cachedir: /tmp/ws/build_isolated/launch_testing_ros/.pytest_cache
rootdir: /tmp/ws/src/ros2/launch_ros/launch_testing_ros, configfile: pytest.ini, testpaths: test
plugins: launch-testing-ros-0.23.0, launch-testing-1.4.1, ament-copyright-0.14.0, ament-pep257-0.14.0, ament-flake8-0.14.0, ament-lint-0.14.0, rerunfailures-11.1.2, timeout-2.1.0, mock-3.6.1, colcon-core-0.12.1, cov-3.0.0
collected 8 items

test/test_copyright.py .                                                 [ 12%]
test/test_flake8.py .                                                    [ 25%]
test/test_pep257.py .                                                    [ 37%]
test/examples/check_msgs_launch_test.py .                                [ 50%]
test/examples/check_node_launch_test.py .                                [ 62%]
test/examples/set_param_launch_test.py .                                 [ 75%]
test/examples/talker_listener_launch_test.py .                           [ 87%]
test/examples/wait_for_topic_launch_test.py .                            [100%]

=============================== warnings summary ===============================
test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

test/examples/check_msgs_launch_test.py: 2 warnings
test/examples/check_node_launch_test.py: 2 warnings
test/examples/set_param_launch_test.py: 2 warnings
test/examples/talker_listener_launch_test.py: 2 warnings
test/examples/wait_for_topic_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
---- generated xml file: /tmp/ws/test_results/launch_testing_ros/pytest.xml ----
======================= 8 passed, 13 warnings in 21.50s ========================
terminate called without an active exception
--- stderr: launch_testing_ros

=============================== warnings summary ===============================
test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

test/examples/check_msgs_launch_test.py: 2 warnings
test/examples/check_node_launch_test.py: 2 warnings
test/examples/set_param_launch_test.py: 2 warnings
test/examples/talker_listener_launch_test.py: 2 warnings
test/examples/wait_for_topic_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/examples/wait_for_topic_launch_test.py::wait_for_topic_launch_test
  Warning: Exception in thread Thread-8 (spin)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 280, in spin
      self.spin_once()
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 717, in spin_once
      handler, entity, node = self.wait_for_ready_callbacks(timeout_sec=timeout_sec)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 703, in wait_for_ready_callbacks
      return next(self._cb_iter)
    File "/tmp/ws/install_isolated/rclpy/local/lib/python3.10/dist-packages/rclpy/executors.py", line 604, in _wait_for_ready_callbacks
      raise ExternalShutdownException()
  rclpy.executors.ExternalShutdownException

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
terminate called without an active exception
---
Failed   <<< launch_testing_ros [22.3s, exited with code -6]

Blast545 · 2024-09-26T17:25:39Z

@Crola1702 Is this still happening?

fujitatomoya · 2024-10-04T04:55:20Z

@Blast545 @Crola1702 can we close this? this is pretty old and looks like we have not seen this issue for a long time?

Crola1702 · 2024-10-07T16:19:51Z

can we close this? this is pretty old and looks like we have not seen this issue for a long time?

Yes, I haven't seen it for a while. Closing now

Crola1702 closed this as completed Oct 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

👨‍🌾 Flaky issues with `launch_testing_ros`: `terminate called without an active exception` #324

👨‍🌾 Flaky issues with `launch_testing_ros`: `terminate called without an active exception` #324

Crola1702 commented Sep 12, 2022

Crola1702 commented Sep 16, 2022

Crola1702 commented Sep 21, 2022

Crola1702 commented Sep 29, 2022 •

edited

Loading

Crola1702 commented Oct 24, 2022 •

edited

Loading

Crola1702 commented Oct 27, 2022

Crola1702 commented Nov 11, 2022

Crola1702 commented Nov 18, 2022

Crola1702 commented Nov 21, 2022 •

edited

Loading

Crola1702 commented Dec 14, 2022

Crola1702 commented Jan 6, 2023

Crola1702 commented Jan 13, 2023

Crola1702 commented Jan 14, 2023

Crola1702 commented Mar 21, 2023

Blast545 commented Sep 26, 2024

fujitatomoya commented Oct 4, 2024

Crola1702 commented Oct 7, 2024

👨‍🌾 Flaky issues with launch_testing_ros: terminate called without an active exception #324

👨‍🌾 Flaky issues with launch_testing_ros: terminate called without an active exception #324

Comments

Crola1702 commented Sep 12, 2022

Bug report

Steps to reproduce issue

Expected behavior

Actual behavior

Additional information

Crola1702 commented Sep 16, 2022

Crola1702 commented Sep 21, 2022

Crola1702 commented Sep 29, 2022 • edited Loading

Crola1702 commented Oct 24, 2022 • edited Loading

Crola1702 commented Oct 27, 2022

Crola1702 commented Nov 11, 2022

Crola1702 commented Nov 18, 2022

Crola1702 commented Nov 21, 2022 • edited Loading

Crola1702 commented Dec 14, 2022

Crola1702 commented Jan 6, 2023

Crola1702 commented Jan 13, 2023

Crola1702 commented Jan 14, 2023

Crola1702 commented Mar 21, 2023

Blast545 commented Sep 26, 2024

fujitatomoya commented Oct 4, 2024

Crola1702 commented Oct 7, 2024

👨‍🌾 Flaky issues with `launch_testing_ros`: `terminate called without an active exception` #324

👨‍🌾 Flaky issues with `launch_testing_ros`: `terminate called without an active exception` #324

Crola1702 commented Sep 29, 2022 •

edited

Loading

Crola1702 commented Oct 24, 2022 •

edited

Loading

Crola1702 commented Nov 21, 2022 •

edited

Loading