Kafka Authenticaion Mechanism for AWS differs from IBM Cloud

[rtclauss]

When using IBM EventStreams/Kafka the authentication mechanism is via username/password using the PLAIN SASL Mechanism. This can be partially seen in EventStreamsProducer.java:

...
        properties.put(SaslConfigs.SASL_MECHANISM, "PLAIN");
        properties.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.plain.PlainLoginModule required username=\"" + USERNAME + "\" password=\"" + API_KEY + "\";");
 ...       

However, Amazon MSK uses a different set of SASL mechanisms as outlined below:

...
        properties.put(SaslConfigs.SASL_MECHANISM, "SCRAM-SHA-512");
        properties.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"" + USERNAME + "\" password=\"" + API_KEY + "\";");
...

Azure Event Hubs can make use of the same, PLAIN, configuration or it can use OAUTHBEARER tokens with callback.

We need a generic mechanism to specify the SASL configuration for Kafka.

[rtclauss]

Related issue in trade-history

[rtclauss]

Maybe make the SASL_MECHANISM and JAAS class environment variables/secrets?

[jwalcorn]

Fixed this several months ago