Could Confluent contribute to the Node.js-librdkafka binding?

[solsson]

I'm not a contributor to this project, only a user, but a thought crossed my mind ... @gwenshap (because you were active in envoyproxy/envoy#2852) or someone else at https://github.com/confluentinc/ - wouldn't Confluent be interested in contributing to this client library?

It very much in line with your Kafka clients strategy (afaik clients for C and Java acceptance tested with every Kafka release, bindings to other languages). @webmakersteve is doing a great job following librdkafka. For example the Admin API is supported already.

However based on #531 (comment) and other issues we've followed there's a lot of nitty gritty details with the binding. Maybe just adding node-rdkafka to Confluent's testing and reporting compliance back would help a lot. There should really be a supported Kafka client for such a well-used language as Node.js.

[gwenshap]

I'm in the wrong team :) But IIRC, our client tests were written so everyone can use them. CC @edenhill for details. Also CC @stanislavkozlovski because he likes Ruby and may be interested.

[solsson]

I'm the maintainer of https://github.com/Yolean/kubernetes-kafka which I suppose is significantly easier than maintaining a client library. Naturally the client has a lot more users than the provisioning. With Kafka a lot of the complexity lies in the client. Naturally the open source repo becomes some kind of support forum. Support takes time and effort, probably more than the "maintainer" model allows.

At my organization we use both Kafka and Node.js a lot and prior to this library we tried for example https://github.com/oleksiyk/kafka. While it works well we came to the conclusion that going forward we can only stabilize our code if we use official clients. There's lots of intricacies where we'd otherwise not know if it was our code or the client behaving badly.

This client is reliable and up to date, but as Kafka adoption grows it will require increasing resources to support. For example I suspect based on my experience that significant time goes into sorting out if an issue is actually an issue or more a question on the behavior. Maybe you can run tests and document them, as I suggested before, or have people follow this repository and offload the support side of maintenance?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.