Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Failure (Segmentation fault on shard 0) in RandomNodeOperationsTest.test_node_operations #16533

Closed
vbotbuildovich opened this issue Feb 8, 2024 · 1 comment
Assignees
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure performance

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Feb 8, 2024

https://buildkite.com/redpanda/vtools/builds/11747

Module: rptest.tests.random_node_operations_test
Class: RandomNodeOperationsTest
Method: test_node_operations
Arguments: {
    "num_to_upgrade": 3,
    "enable_failures": true,
    "with_tiered_storage": true
}
test_id:    RandomNodeOperationsTest.test_node_operations
status:     FAIL
run time:   1303.773 seconds

<NodeCrash ip-172-31-8-50: Segmentation fault on shard 0.
>
Traceback (most recent call last):
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 99, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/tests/random_node_operations_test.py", line 430, in test_node_operations
    self.admin_fuzz.wait(20, 180)
  File "/home/ubuntu/redpanda/tests/rptest/services/admin_ops_fuzzer.py", line 854, in wait
    wait_until(check, timeout_sec=timeout, backoff_sec=2)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 53, in wait_until
    raise e
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 44, in wait_until
    if condition():
  File "/home/ubuntu/redpanda/tests/rptest/services/admin_ops_fuzzer.py", line 842, in check
    raise self.error
  File "/home/ubuntu/redpanda/tests/rptest/services/admin_ops_fuzzer.py", line 682, in thread_loop
    self.execute_one()
  File "/home/ubuntu/redpanda/tests/rptest/services/admin_ops_fuzzer.py", line 743, in execute_one
    raise e
  File "/home/ubuntu/redpanda/tests/rptest/services/admin_ops_fuzzer.py", line 725, in execute_one
    wait_until(
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Timeout waiting for {'type': 'delete_topic', 'properties': {'name': 'fuzzy-operator-3394-bdtgbh'}} operation validation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/usr/local/lib/python3.10/dist-packages/ducktape/mark/_mark.py", line 481, in wrapper
    return functools.partial(f, *args, **kwargs)(*w_args, **w_kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 120, in wrapped
    redpanda.raise_on_crash(log_allow_list=log_allow_list)
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 2805, in raise_on_crash
    raise NodeCrash(crashes)
rptest.services.utils.NodeCrash: <NodeCrash ip-172-31-8-50: Segmentation fault on shard 0.
>

JIRA Link: CORE-1761

@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Feb 8, 2024
@travisdowns travisdowns self-assigned this Feb 9, 2024
@travisdowns travisdowns changed the title CI Failure (key symptom) in RandomNodeOperationsTest.test_node_operations CI Failure (Segmentation fault on shard 0) in RandomNodeOperationsTest.test_node_operations Feb 9, 2024
@ztlpn
Copy link
Contributor

ztlpn commented May 14, 2024

Likely fixed by #18305

@ztlpn ztlpn closed this as completed May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure performance
Projects
None yet
Development

No branches or pull requests

5 participants