[Enhancement]: Ability to fully configured CONTROLLER and REPLICATION listeners, then being able to support PLAINTEXT and SASL_PLAINTEXT.

[nbuesing]

Related problem

in secured internal networks, there is a desire to allow inter-broker replication to be done over plaintext to allow for fully leveraging zero-page copies. In addition, to avoid 'User:ANONYMOUS' being a super-user, for systems that are authenticating, then the only way to do this with PLAINTEXT is with SASL (sending client SSL certs over plaintext is not something that can be done, so is not an option).

Thus, I have seen and leveraged Kafka clusters where external communication is SASL_SSL and internal communication is SASL_PLAINTEXT. In these scenarios, it would be nice if the controller and inter-broker communication could also be SASL_PLAINTEXT.

Suggested solution

Allow the internal listeners to be configured the same way as additional listeners. Add in two additional types, "controller" and "replication". This would allow supporting additional valid Kafka configurations.

If SASL is being used, then the user/password must also be provided. This could be done by providing the username, and then the password is pulled via the secret through the existing Strimzi conventions. This user would be automatically added to the super-user list, just like it does for mTLS today. For deployments that don't do any client-based mTLS, this allows for a single setup of authentication and makes it easier for existing Kafka cluster setups to migrate to Strimzi.

Now, this user does need to be created prior to starting the cluster, if not created. This does require the creation of this user before the cluster is started. If sasl.mechanism=plain was supported, then the user could be part of the configuration itself, but that would generate the password and make it part of the configuration.

zookeeper

https://kafka.apache.org/documentation/#security_sasl_scram_credentials

bin/kafka-configs.sh --zookeeper localhost:2182 --zk-tls-config-file zk_tls_config.properties --alter --add-config 'SCRAM-SHA-512=[password=alice-secret]' --entity-type users --entity-name alice

kraft

kafka-storage.sh format --config CONFIG --cluster-id CLUSTER_ID --add-scram SCRAM_CREDENTIAL ...

    listeners:
      - name: interbroker
        port: 9091
        type: replication. # controller would also be a valid new type.
        tls: false
        authentication:
          type: scram-sha-512
          username: broker-user

Alternatives

This existing enhancement request (#2266) is 5 years old, and doesn't discuss supporting SASL_PLAINTEXT, just PLAINTEXT. Ideally, if this is to be supported, having it support more existing cluster configurations that could then be migrated to Strimzi, the better.

Additional context

No response

[scholzj]

Sorry, but this is just a duplicate of #2266. Any possible solution - and yes, some kind of authentication has to be part of it - should be discussed as part of a proposal (https://github.com/strimzi/proposals).

[scholzj]

PS: In case you decide to write the proposal, I think what you describe here has several things that will need to be clarified:

• What should be the authentication? You suggest SCRAM-SHA-512. But that is exposed to the users, so it might not be a good choice. Something based on service accounts might be better (e.g. based on Additional authentication type 'k8s-oidc' that extends 'oauth' #9657)?
• Should the authentication be configurable? Or should the same authentication be used all the time and only TLS encryption be configurable?
• What are the requirements for rotating the credentials?
• How should the API for configuring it look like? What are the requirements on the API from the possible stretch cluster support in the future?
• How will the upgrades / transitions between the configurations work?

[nbuesing]

Thank you, this is very helpful.

rotating credentials for typical clients that are using sasl (scram) is to issue a new username and password, because you cannot change the password on one instance w/out impacting the other -- I will add this to the proposal and think through alternatives.

I like what you are thinking/saying around using scram internally may not be ideal when that is exposed to users externally. My thinking was trying to emulate existing configurations and also trying to avoid adding any custom code to handle additional types -- I will look in the shared issue; thank you.

I will continue to think and work on proposal with this also in mind.

Thanks again.

[ppatierno]

Triaged on 30/5/2024: @nbuesing going to close this one as duplicate of #2266. If you are still interested, you could take care of bringing a proposal for it.