Skip to content

KafkaIO should support dynamic addition of Kafka partitions to assigned topics.  #17929

@damccorm

Description

@damccorm

Kafka topics may add partitions dynamically (doesn't require Kafka to restart, or halt a topic), and the KafkaIO should probably support this.

Note:
Consistently assigning partitions should be taken into account, specifically for the case of reading from multiple topics, where one (or more) of the topics added partitions while the pipeline is running.

Imported from Jira BEAM-727. Original Jira may contain additional context.
Reported by: amitsela.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions