Clarification regarding SASL Authentication with Event Streams

[manesioz]

Hello,
I just wanted to clarify something regarding what fields we will out and where we get that information from.

In my event streams instance, I have credentials as follows:

{
  "api_key": "<API_KEY>",
  "apikey": "<API_KEY>",
  "iam_apikey_description": "Auto-generated for key <ID>",
  "iam_apikey_name": "event-streams-credentials",
  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",
  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/<ID>::serviceid:ServiceId-<SERVICE_ID>",
  "instance_id": "<ID>",
  "kafka_admin_url": "<ADMIN_URL>",
  "kafka_brokers_sasl": [
    "broker-3-url",
    "broker-2-url",
    "broker-4-url",
    "broker-1-url",
    "broker-0-url",
    "broker-5-url"
  ],
  "kafka_http_url": "<URL>",
  "password": "<API_KEY>",
  "user": "token"
}

My Kafka Producer is authenticated as follows:

const Kafka = require('node-rdkafka');

const hosts = [
    "broker-3-url",
    "broker-2-url",
    "broker-4-url",
    "broker-1-url",
    "broker-0-url",
    "broker-5-url"
  ]; 

var producer = new Kafka.Producer({
    'debug' : 'security',
    'metadata.broker.list': hosts.join(','),
    'dr_cb': true, //delivery report callback
    'security.protocol': 'sasl_plaintext',
    // 'ssl.ca.location': '/etc/ssl/certs/',
    'sasl.mechanisms': 'PLAIN',
    'sasl.username': 'token',
    'sasl.password': '<API_KEY>',
    'retries': 10,
    'retry.backoff.ms': 10000
});

But when I try to connect I get the following:

{ Error: Local: Broker transport failure
    at Function.createLibrdkafkaError [as create] (/Users/user/node_modules/node-rdkafka/lib/error.js:334:10)
    at /Users/user/node_modules/node-rdkafka/lib/client.js:339:28
  origin: 'local',
  message: 'broker transport failure',
  code: -195,
  errno: -195,
  stack:
   'Error: Local: Broker transport failure\n    at Function.createLibrdkafkaError [as create] (/Users/user/node_modules/node-rdkafka/lib/error.js:334:10)\n    at /Users/user/node_modules/node-rdkafka/lib/client.js:339:28' }

I was wondering if I am not properly including the authentication params or if it is an issue on node-rdkafka's side of things. I couldn't find clear documentation regarding this configuration so any help would be appreciated, thanks!

[manesioz]

cc @mimaison :)

[mimaison]

See the required configurations in https://github.com/ibm-messaging/event-streams-samples/blob/master/kafka-nodejs-console-sample/app.js#L145-L156

It looks like you're setting security.protocol incorrectly. It should be set to sasl_ssl.

[manesioz]

Thanks for the quick reply.
When I used that I got the error

Error: Invalid value "sasl_ssl" for configuration property "security.protocol"

My node-rdkafka version is 1.2.2, with features

[ 'gzip',
  'snappy',
  'sasl',
  'regex',
  'lz4',
  'sasl_gssapi',
  'sasl_plain',
  'plugins' ]

Should I downgrade versions? Odd considering the docs show that security.protocol can have values plaintext, ssl, sasl_plaintext, sasl_ssl.

Edit: I also can't seem to find any information for the param ssl.ca.location, is that found on the credentials page of the event stream?

[mimaison]

It looks like you're missing SSL support. See the required dependencies https://github.com/ibm-messaging/event-streams-samples/blob/master/docs/librdkafka.md

I recommend looking at the sample application we provide to get familiar with the required configurations and the different options. For example, ssl.ca.location is mentioned in https://github.com/ibm-messaging/event-streams-samples/blob/master/kafka-nodejs-console-sample/app.js#L70-L73

[mimaison]

@manesioz Did you get it working? Can we close this issue?

[manesioz]

Apologies, I have been recently side-tracked by other work and have not implemented your solution yet. I will try this soon and get back to you, thanks!