tests: ducktape redpanda service for cloud

[andrewhsu]

Fixes https://github.com/redpanda-data/cloudv2/issues/5893, Fixes #10346, Fixes https://github.com/redpanda-data/devprod/issues/670

This PR adds a new ducktape class RedpandaServiceCloud which enables tests to run against redpanda cloud. The new class uses inner class RedpandaServiceCloud.CloudCluster to connect with the redpanda cloud swagger API to provision and delete clusters. There are some hard-coded values passed to the swagger API that will need to be cleaned up in a future PR because the swagger API is v0.1.0.

Client ID and secrets from redpanda cloud UI are needed to connect to the api. Set these settings in ducktape --globals for the new classes to work:

{
    "cloud_oauth_url": "https://auth.example.com/oauth/token",
    "cloud_oauth_client_id": "your-client-id",
    "cloud_oauth_client_secret": "your-client-secret",
    "cloud_oauth_audience": "your-audience",
    "cloud_api_url": "https://your-api.cloud.redpanda.com"
}

A simple test class CloudSelfTest SimpleSelfTest is created in tests/rptest/tests/services_self_test.py to run a few operations that connect to the k8s pods of redpanda cloud. The creation and deletion of a cluster will take over 30 min so you need to bump the test runner timeout:

ducktape \
  --debug \
  --globals=/path/to/ducktape_globals.json \
  --cluster=ducktape.cluster.json.JsonCluster \
  --cluster-file=/path/to/ducktape_cluster.json \
  --test-runner-timeout=3600000 \
  tests/rptest/tests/services_self_test.py::SimpleSelfTest

To skip creation and deletion of a cluster, you can set these additional ducktape --globals values:

{
    "cloud_cluster_id": "chns4udibsvmbjmrtjm0",
    "cloud_delete_cluster": false
}

Output of a CloudSelfTest SimpleSelfTest test run using an existing cluster:

test_id:    rptest.tests.services_self_test.CloudSelfTest.test_cloud
status:     PASS
run time:   25.479 seconds
------------------------------------------------------------------------------
==============================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.8.8
session_id:       2023-05-26--016
run time:         25.493 seconds
tests run:        1
passed:           1
failed:           0
ignored:          0
opassed:          0
ofailed:          0
==============================================================================

Output of a CloudSelfTest SimpleSelfTest test run creating and deleting a cluster:

...
[INFO  - 2023-05-26 12:33:44,599 - redpanda - create - lineno:1365]: waiting for creation of cluster rp-ducktape-cluster-8c90c19c namespaceUuid 178a5705-47da-4bef-a938-c0e371839870, checking every 60.0 seconds
[DEBUG - 2023-05-26 12:33:44,599 - redpanda - _cluster_ready - lineno:1284]: checking readiness of cluster rp-ducktape-cluster-8c90c19c
[DEBUG - 2023-05-26 12:34:45,475 - redpanda - _cluster_ready - lineno:1284]: checking readiness of cluster rp-ducktape-cluster-8c90c19c
[DEBUG - 2023-05-26 12:35:46,243 - redpanda - _cluster_ready - lineno:1284]: checking readiness of cluster rp-ducktape-cluster-8c90c19c
...
[INFO:2023-05-26 13:09:35,611]: RunnerClient: rptest.tests.services_self_test.CloudSelfTest.test_cloud: Tearing down...
[DEBUG - 2023-05-26 13:09:36,464 - redpanda - delete - lineno:1393]: resp: {"id": "choagahmsn17ttei3f0g", "name": "rp-ducktape-cluster-8c90c19c"...
...
test_id:    rptest.tests.services_self_test.CloudSelfTest.test_cloud
status:     PASS
run time:   35 minutes 54.077 seconds
-------------------------------------------------------------------------------
===============================================================================
SESSION REPORT (ALL TESTS)
ducktape version: 0.8.8
session_id:       2023-05-26--018
run time:         35 minutes 54.101 seconds
tests run:        1
passed:           1
failed:           0
ignored:          0
opassed:          0
ofailed:          0
===============================================================================

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.1.x
• v22.3.x
• v22.2.x

Release Notes

• none

[andrewhsu]

a few failures in the build job for 14eab79 look like existing known failures. this PR is ready for review.

[ivotron]

lgtm pending green checkmarks. I assume this has been tested manually against actual RP cloud clusters?

[ivotron]

shouldn't this function be invoked from all existing tests? I wonder if there's a way to inject this factory as part of the invocation to the parent class. Though that would be kind of hacky

[andrewhsu]

i was initially being conservative in this PR by just using the factory function in CloudSelfTest, but i'll change it to replace everywhere. i counted 17 files that get changed which is not crazy and the commit is a simple replace.

i think the factory function fits in well here since its signature is the same as the constructor of RedpandaService.

[ivotron]

i meant to choose request changes in the previous review

[andrewhsu]

I assume this has been tested manually against actual RP cloud clusters?

yes, snippet of local workstation run output copied to description of this PR.

[ivotron]

lgtm, assuming green checks

[andrewhsu]

all tests from ducktape-build-release-clang were green.

unrelated failures from ducktape-build-debug-clang:

• CI Failure (TimeoutError) in KgoRepeaterSelfTest.test_kgo_repeater #10865
• CI Failure (consumer timed out waiting for offsets) in ControllerLogLimitMirrorMakerTests.test_mirror_maker_with_limits #10502
• CI Failure (Timeout - Failed to start) in MultiTopicAutomaticLeadershipBalancingTest.test_topic_aware_rebalance #11044
• Reactor stall in redpanda #1725 (comment)