Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix KafkaCliTools.produce in cloud tests #17896

Merged
merged 3 commits into from
Apr 17, 2024
Merged

Fix KafkaCliTools.produce in cloud tests #17896

merged 3 commits into from
Apr 17, 2024

Conversation

travisdowns
Copy link
Member

KafkaCliTools.produce didn't work if SASL or TLS was enabled because
it failed to pass along the relevant configuration. This command does
not accept a config file via --command-config like many other cli tools
(this is how we pass this information to those tools), but it does
accept a --producer.config argument which serves the same purpose.

Set if additional security config needs to be passed in this way,
which fixes this test in the cloud.

Fixes https://github.com/redpanda-data/core-internal/issues/1256

Backports Required

  • none

Release Notes

  • none

@travisdowns
rptest: remove spurious parens
8222ab2
@travisdowns
rptest: fix KafkaCliTools.produce for SASL and TLS
58d34cc 
KafkaCliTools.produce didn't work if SASL or TLS was enabled because
it failed to pass along the relevant configuration. This command does
not accept a config file via --command-config like many other cli tools
(this is how we pass this information to those tools), but it does
accept a --producer.config argument which serves the same purpose.

Set if additional security config needs to be passed in this way,
which fixes this test in the cloud.

Fixes redpanda-data/core-internal#1256
@travisdowns travisdowns requested a review from savex April 16, 2024 19:10
@travisdowns travisdowns changed the title Td 1256 internal cli tools produce Fix KafkaCliTools.produce in cloud tests Apr 16, 2024
rpdevmp
rpdevmp previously approved these changes Apr 16, 2024
View reviewed changes
Copy link
Contributor

@rpdevmp rpdevmp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Apr 16, 2024 •
edited

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee88b-fc85-4f6c-a679-d0350e7d1f82:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_describe_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee88b-fc88-4136-a9e7-1b1980981fa0:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_empty_list_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee88b-fc83-4af5-b9f4-b069860b48a0:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_list_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee8ab-7a88-4fd8-a7fd-6b12a957d67e:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_list_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee8ab-7a8b-4719-b5f2-903704370958:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_describe_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47887#018ee8ab-7a8d-424b-af54-a92d5dee71bd:

"rptest.tests.transaction_kafka_api_test.TxKafkaApiTest.test_empty_list_transactions"

new failures in https://buildkite.com/redpanda/redpanda/builds/47948#018eed05-c579-4339-9305-030ef7bd69e4:

"rptest.tests.recovery_mode_test.RecoveryModeTest.test_rolling_restart"
"rptest.tests.usage_test.UsageTestCloudStorageMetrics.test_usage_manager_cloud_storage"
"rptest.tests.cloud_storage_chunk_read_path_test.CloudStorageChunkReadTest.test_prefetch_chunks.prefetch=5"
"rptest.tests.archival_test.ArchivalTest.test_reconnect.cloud_storage_type=CloudStorageType.S3"

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Apr 16, 2024 •
edited

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47887#018ee88b-fc88-4136-a9e7-1b1980981fa0

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47887#018ee8ab-7a8b-4719-b5f2-903704370958

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47948#018eecfd-f67c-4502-a921-617226847e9d

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/47948#018eed05-c579-4339-9305-030ef7bd69e4

@travisdowns
rptest: use _run over _execute in KafkaCliTools
885250e 
_run is a less-raw version of _execute which adds the
--bootstrap-broker argument, and importantly also the --command-config
argument which allows SASL/TLS to work. This change converts callers of
_excecute into callers of _run instead if they were compatible with
that (some aren't, e.g., because the underlying script doesn't
support --command config).

This allows the tests to work with SASL and TLS, particularly in
public cloud runs which always have both enabled.
@travisdowns travisdowns force-pushed the td-1256-internal-cli-tools-produce branch from 0e7ea9f to 885250e Compare April 17, 2024 15:26
savex
savex approved these changes Apr 17, 2024
View reviewed changes
Copy link
Contributor

@savex savex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@travisdowns travisdowns merged commit 2990542 into redpanda-data:dev Apr 17, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@savex savex savex approved these changes

@rpdevmp rpdevmp rpdevmp left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@travisdowns @vbotbuildovich @savex @rpdevmp