Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: retry on readyReplicas test #16669

Merged
merged 1 commit into from
Feb 23, 2024
Merged

Conversation

dotnwat
Copy link
Member

@dotnwat dotnwat commented Feb 21, 2024

Presumably if the cluster isn't yet fully initialized then jsonpath query for readyReplicas will return an empty string and converting that to an integer will fail. The entire process is in a wait_until, so we'll just ignore these errors.

test_id:    HighThroughputTest.test_add_and_decommission
status:     FAIL
run time:   2629.113 seconds

ValueError("invalid literal for int() with base 10: ''")
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
    return self.test_context.function(self.test)
  File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 104, in wrapped
    r = f(self, *args, **kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1221, in test_add_and_decommission
    self._stage_add_and_decommission()
  File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1251, in _stage_add_and_decommission
    self._patch_cluster_replicas(cluster_name, orig_replicas)
  File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1165, in _patch_cluster_replicas
    wait_until(
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 53, in wait_until
    raise e
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 44, in wait_until
    if condition():
  File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1166, in <lambda>
    lambda: cluster_ready_replicas(cluster_name) == replicas,
  File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1144, in cluster_ready_replicas
    return int(
ValueError: invalid literal for int() with base 10: ''

Fixes: #16631

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x
  • v23.1.x

Release Notes

  • none

Presumably if the cluster isn't yet fully initialized then jsonpath
query for readyReplicas will return an empty string and converting that
to an integer will fail. The entire process is in a wait_until, so we'll
just ignore these errors.

    test_id:    HighThroughputTest.test_add_and_decommission
    status:     FAIL
    run time:   2629.113 seconds

    ValueError("invalid literal for int() with base 10: ''")
    Traceback (most recent call last):
      File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 184, in _do_run
        data = self.run_test()
      File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 269, in run_test
        return self.test_context.function(self.test)
      File "/home/ubuntu/redpanda/tests/rptest/services/cluster.py", line 104, in wrapped
        r = f(self, *args, **kwargs)
      File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1221, in test_add_and_decommission
        self._stage_add_and_decommission()
      File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1251, in _stage_add_and_decommission
        self._patch_cluster_replicas(cluster_name, orig_replicas)
      File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1165, in _patch_cluster_replicas
        wait_until(
      File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 53, in wait_until
        raise e
      File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 44, in wait_until
        if condition():
      File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1166, in <lambda>
        lambda: cluster_ready_replicas(cluster_name) == replicas,
      File "/home/ubuntu/redpanda/tests/rptest/redpanda_cloud_tests/high_throughput_test.py", line 1144, in cluster_ready_replicas
        return int(
    ValueError: invalid literal for int() with base 10: ''

Fixes: redpanda-data#16631

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
@vbotbuildovich
Copy link
Collaborator

Copy link
Member

@andrewhsu andrewhsu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@travisdowns travisdowns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@piyushredpanda piyushredpanda merged commit e70f9ee into redpanda-data:dev Feb 23, 2024
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI Failure (invalid literal for int()) in HighThroughputTest.test_add_and_decommission
5 participants