investigate capability of extending kubeless topic command

[ngtuna]

Is this a BUG REPORT or FEATURE REQUEST?: FEATURE REQUEST

What happened:

kubeless topic command is now specified to kafka-zookeeper setup and bitnami/kafka image. We would like to extend this command to use with other kafka setup, following up with the work has done for this issue #196 at PR #591 .

cc @solsson, how is kafka deployed on your environment ?

What you expected to happen:

kubeless topic will work for some common kafka setups on k8s.

[solsson]

We've mostly used a Job that runs the same command you do, for example: https://github.com/Yolean/kubernetes-kafka/blob/v3.0.1/kafka/test/kafkacat.yml#L66

I'd recommend against executing this in a broker pod, see Yolean/kubernetes-kafka#144 (comment).

Now that I look at the kubeless impl, based on #196 (comment), I recommend that you don't specify replication-factor. That way you honor the cluster's default. 1 is unsuitable for any production environment as it blocks all clients if the wrong broker goes down.

Topic creation can be automatic, but delete and list can't. I don't know of any options that can use pure broker access, without a zookeeper dependency (I learned about the existence of CreateTopics DeleteTopics etc today but know nothing about them). Maybe a custom kafka cluster should be required to specify a zookeeper service as well? A kubectl run --attach=true --rm with a kafka image of choice, and the commands you already have, would be the easiest way forward.

Wouldn't it be great to implement this using kubeless functions? Java might be too heavy a runtime, but there could be Node or Python impls that can do topic operations.