Return accurate partition count and replication factor in CreateTopics response #16410
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/dt |
oleiman
force-pushed
the
bug/15722/create-topics-response
branch
from
8882459
to
44f8e9a
Compare
oleiman
commented
Jan 31, 2024
oleiman
changed the title
7 tasks
|
new failures in https://buildkite.com/redpanda/redpanda/builds/44558#018d61b4-0b8e-40e9-be5a-8fd15fe69f8d: new failures in https://buildkite.com/redpanda/redpanda/builds/44558#018d6558-10ae-4a38-8f18-e619bf5cf974: |
|
ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/44558#018d61b8-a014-45c1-9fd5-c827b0f20751 |
|
CI Failure:
|
graphcareful
reviewed
Feb 1, 2024
Specifically we noticed that cleanup.policy was populated with a stringified int in CreateTopics results. We implement formatters for several topic config types, so we should use them. Using e.g. "delete" for the cleanup policy value (rather than "1") matches the behavior of Apache Kafka. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
oleiman
force-pushed
the
bug/15722/create-topics-response
branch
from
44f8e9a
to
ba3b786
Compare
oleiman
force-pushed
the
bug/15722/create-topics-response
branch
from
ba3b786
to
8b6dc85
Compare
|
force push contents
|
|
@graphcareful @michael-redpanda - Sorry, needs a bit more review. Pulled in the commits for #7946 because the scope of the changes became much smaller but the DT tests are still dependent on this one landing. |
graphcareful
reviewed
Feb 2, 2024
To amend create_topics responses with the partition count and replication factor of the created (or already existing) topic. Values pulled from metadata cache (similar to append_topic_configs) Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
- If the topic already exists, use that return code (36) - If the topic does not exist, return the partition count and replication factor from the request. - Responses now match Apache Kafka
Adapter for raw_create_topics that takes a list of client-supplied dicts, packs them up in raw requests, and issues a create request. Loads the result into a dict, extracts 'Topics' field, and returns the result (or empty list if something went wrong). Also adds a validate_only bool param to raw_create_topics Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
Test for correct config serialization and treatment of specific and defaulted partition count and replication factor values in responses. Signed-off-by: Oren Leiman <oren.leiman@redpanda.com>
In `CreateTopicsResponseTest`.
oleiman
force-pushed
the
bug/15722/create-topics-response
branch
from
8b6dc85
to
0654662
Compare
|
force push remove |
michael-redpanda
approved these changes
Feb 2, 2024
Comment on lines
+270
to
+271
| result.num_partitions = t.num_partitions; | ||
| result.replication_factor = t.replication_factor; |
There was a problem hiding this comment.
question: just to confirm, if the topic already exists, we do not return that topics partition count or RF?
There was a problem hiding this comment.
Correct. That's what the latest Kafka does.
|
/backport v23.3.x |
|
/backport v23.2.x |
|
/backport v23.1.x |
|
Failed to create a backport PR to v23.1.x branch. I tried: |
This was referenced Feb 2, 2024
Closed
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit makes some changes to the create_topics path to ensure that partition count and replication factor of created (or already existing) topics are correctly populated in responses.
Includes a small change to ensure that topic configs are serialized correctly with respect to existing formatters (e.g. the default cleanup.policy should appear in create topics results as "delete" where previously it was "1").
Fixes #15722
Fixes #7946
Backports Required
Release Notes
Bug Fixes