Skip to content

Add possibility of configuration for Coordinator and Worker prefix #12342

@TiagoSilva67

Description

@TiagoSilva67

Proposed Change

Feature Request / Improvement

Summary:
Apache Iceberg Kafka Connect should be able to allow configuration of the naming of structures that need to be compliant with ACL policies in the environment where it is being installed.

Motivation
Currently, Apache Iceberg does not allow to have custom names for Coordinator and Worker in the kafka-connect since the names are hard coded, which causes problem in the using of the connector because it is blocked by ACL policies that are expecting a specific prefix in the structures names.

Image

Image

This is a problem specially for cases where the ACL policies can not be changed to accommodate the kafka-connect names.

Proposed Implementation:

  • Introduce a new sink property that define the prefix
  • Add method to IcebergSinkConfig to retrieve the value of the property from the sink configuration
  • Use the prefix as the name when constructing instances of classes that extend Channel (e.g., Coordinator or Worker)

Query engine

  • None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time

Proposal document

No response

Specifications

  • Table
  • View
  • REST
  • Puffin
  • Encryption
  • Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    improvementPR that improves existing functionalitystale

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions