[STORM-2472] kafkaspout should work normally in kerberos mode with kafka 0.9.x API

[liu-zhaokun]

https://issues.apache.org/jira/browse/STORM-2472
storm 1.0.x-branch didn't support kafkaspout to consume from kafka in kerberos mode with we can't set kafka's parameter ,'java.security.auth.login.config', in storm's process .So I solve it via preparing a kafkaspout.

[harshach]

@liu-zhaokun you can set it via topology.worker.childopts or worker.childopts. Lets is not put work-around for this when there is viable option to set it. Also 0.9 consumer API is for Alpha and its not been used by anyone.

[liu-zhaokun]

@harshach
It doesn't work even I have set system property in the topology.But we can do it by setting the path to "java.security.auth.login.config" in the 'open' method of kafkaspout.

[liu-zhaokun]

Hello, @harshach
Looking forward to your reply.

[liu-zhaokun]

@HeartSaVioR
Could you help me to review this PR?

[liu-zhaokun]

@vesense
Hello,could you help me to review this PR?

[harshach]

@liu-zhaokun I am still not sure why are we adding this code. One shouldn't be using 0.9 kafka clients and this storm-kafka-client will not work with 0.9 as the interface changed in 0.10 and we made changes to storm-kafka-client to start working only from 0.10 kafka clients only. As I said before 0.9 kafka clients shouldn't be used as they are alpha quality.
If you look the kafka client docs https://kafka.apache.org/documentation/ , you'll see "sasl.jaas.config" this is consumer config one can pass and the storm-kafka-client will allow you to pass this config. So there is necessity to add this code.

[harshach]

@liu-zhaokun following upon my previous comment , do not use stom-kafka-client from 1.0.x-branch as there are lot of bug-fixes went into storm-kafka-client in Storm 1.1 release

[hmcl]

@liu-zhaokun Are you planning on following up on this PR?

[aniketalhat]

@liu-zhaokun (CC: @harshach) How did you manage to solve this? Currently new storm-kafka-client isn't very stable and doesn't support the transactional spout. I've to connect to Kafka (0.11.0.2) via. Kerberos on Storm (1.0.6)

Any ideas?, it will be a great help!

[aniketalhat]

Hello Liu, Thanks for your quick reply, my one concern is that your PR is for storm-kafka-client(new consumer) where as I'm looking for storm-kafka(old consumer) solution, what do you think ?

…

On Tue, May 15, 2018 at 4:40 PM, zhaokun liu ***@***.***> wrote: @aniketalhat <https://github.com/aniketalhat> You can look through my PR，and my intention was to export java.security.auth.login.config=the path to your keytab,and it works.You can try it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2066 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADcPbLdznyCju9ESv2_z3xh4xhWoGpDRks5tyoZ3gaJpZM4M8Nf7> .

-- *Aniket Alhat*