KAFKA-20410: Add DLQ configuration parameters for Share Groups (KIP-1191)

[sjhajharia]

Summary

This PR adds the configuration foundation for Share Groups Dead-Letter
Queue (DLQ) functionality as specified in KIP-1191: Dead-letter queues
for share
groups.

Changes

New Configurations

Cluster-level configs (GroupConfig.java):

1. errors.deadletterqueue.auto.create.topics.enable (default: false)
   • Enables automatic creation of DLQ topics
2. errors.deadletterqueue.topic.name.prefix (default: "dlq.")
   • Required prefix for DLQ topic names when auto-create is enabled

Group-level configs (GroupConfig.java):

1. errors.deadletterqueue.topic.name (default: "")
   • Specifies the DLQ topic name for a share group
   • Empty string means DLQ disabled for that group
2. errors.deadletterqueue.copy.record.enable (default: false)
   • When true: Copy full original record to DLQ
   • When false: Copy only context metadata (offset, delivery count,
     reason)

Topic-level config (TopicConfig.java):

1. errors.deadletterqueue.group.enable (default: not set)
   • Marks a topic as eligible for use as a DLQ

Validation Logic

• DLQ topic names must not start with __ (reserved for internal
  topics)

Tests

Added 7 comprehensive test cases in ShareGroupConfigTest:

• testDLQConfigDefaults() - Verify default values
• testDLQConfigCustomValues() - Validate custom configurations
• testDLQTopicNameCannotStartWithDoubleUnderscore() - Reject __
  prefix
• testDLQTopicNameMustMatchPrefixWhenAutoCreateEnabled() - Enforce
  prefix with auto-create

Reviewers: David Jacot david.jacot@gmail.com, Andrew Schofield
aschofield@confluent.io

[dajac]

All configs starts with the same prefix. Why don’t we follow the same pattern for those?

[sjhajharia]

We are intentionally not including the group.share. prefix on these configs to keep them generic enough so that in the future, we can re-use them as and when needed and maintain compatibility still.

[AndrewJSchofield]

Thanks for the PR. I've been a bit picky about the doc strings for the configs because they will go into the documentation.

[AndrewJSchofield]

I don't think this validation is quite what I intended. It does the right thing in the wrong way for a subset of the situations the feature needs to cope with.

I don't think the validation of the DLQ topic name config depends upon the auto-create config at all. If the DLQ topic name for the group does not satisfy the broker-level topic name prefix, that is a failure at the time we need to enqueue a record.

[sjhajharia]

Thanks for the comment. I have removed that validation from here.

[AndrewJSchofield]

Given that these configs are intended to be more general than share groups because we might introduce a way to handle poisoned messages for other group types in the future, I would push these configs up a level and not define them in ShareGroupConfig.

[sjhajharia]

Thanks for the feedback. Do you think the GroupCoordinatorConfig class may be a better seat for these configs? I was thinking of the GroupConfig class too, but those config seem very group specific, while we are talking about both cluster level and Group level configs.
The other alternative I could think was to include errors.deadletterqueue.auto.create.topics.enable and errors.deadletterqueue.topic.name.prefix in the GroupCoordinatorConfig class whole the other two can sit in GroupConfig class.
Please let me know which is the direction which seems better here, or if there is another alternative. TIA!

[AndrewJSchofield]

@sjhajharia Sorry for the delay. I think that GroupConfig is the best.

[sjhajharia]

Thanks for the suggestion. I have updated the PR accordingly.