ERROR Closing socket and web-console-consumer-XXXXX

[mungeol]

Hi,

Kafka gives an error whenever I refresh 'consumer group' page.
The full error log shows below.

ERROR Closing socket for /10.0.2.83 because of error (kafka.network.Processor)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at kafka.utils.Utils$.read(Utils.scala:375)
at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
at kafka.network.Processor.read(SocketServer.scala:347)
at kafka.network.Processor.run(SocketServer.scala:245)
at java.lang.Thread.run(Thread.java:745)

Another problem I found is it creates many consumers at zookeeper.
under /consumers

[web-console-consumer-81258, web-console-consumer-40995, web-console-consumer-15923, web-console-consumer-2305, web-console-consumer-83410, web-console-consumer-45966, web-console-consumer-82765, web-console-consumer-49384, web-console-consumer-68405, web-console-consumer-93714, web-console-consumer-9685, web-console-consumer-17315, web-console-consumer-80814, web-console-consumer-18646, web-console-consumer-60258, web-console-consumer-53349, web-console-consumer-47546, web-console-consumer-13959, logstash, web-console-consumer-63632]

Is this normal?

Thanks.

---

environmental info

kafka_2.9.2-0.8.1.1
latest kafka-web-console-master

[ibanner56]

I ran into the second half of this yesterday (see #36). I'll let you know when I figure out what's doing it.

[ibanner56]

Are you just using this repo without any edits?

[ibanner56]

It's being created in the 'feed' function in Topic.scala - I just need to figure out what the correct course of action is.

[ibanner56]

Hm, it looks like the feed function is supposed to be deleting it:

    val in = Iteratee.foreach[String](println).map { _ =>
      consumer.commitOffsets()
      consumer.shutdown()
      deleteZNode(zkClient, "/consumers/" + consumerGroup)
    }

But something isn't working.

[ibanner56]

I added a debug line to check which nodes it's trying to delete and it looks like it's successfully deleting them most of the time, but sometimes empty nodes end up sticking around. Not sure why, entirely.

[mungeol]

Hi,

I used this repo without any change.
And, thank you for your reply.

Thanks.

[ibanner56]

I'm going to take a look tomorrow to see if I can get something from the
Zookeeper logs. I'll post here with updates then.

Sorry for the brevity - sent from my mobile robot.
On Oct 15, 2014 7:34 PM, "mungeol" notifications@github.com wrote:

Hi,

I used this repo without any change.
And, thank you for your reply.

Thanks.

—
Reply to this email directly or view it on GitHub
#37 (comment)
.

[ibanner56]

http://pastebin.com/LceMuMyg

The above is all the output I gathered on a zk entry that failed to delete. I'll be digging through it for a little while.

[ibanner56]

Hey, sorry for the long wait - I updated the delete workflow and, in theory, it no longer leaves empty web-console-consumer nodes. I submitted a pull request, but until it gets merged in you can check out my code here:
https://github.com/ibanner56/kafka-web-console

[mungeol]

Great. I will try it.
Thank you and have a nice day.

On Tue, Oct 28, 2014 at 5:03 AM, Isaac Banner notifications@github.com
wrote:

Hey, sorry for the long wait - I updated the delete workflow and, in
theory, it no longer leaves empty web-console-consumer nodes. I submitted a
pull request, but until it gets merged in you can check out my code here:
https://github.com/ibanner56/kafka-web-console

—
Reply to this email directly or view it on GitHub
#37 (comment)
.

[ibanner56]

I added a little bit more to my code, you'll want to update what your using - I forgot to add a failure case, so if it fails for some reason it wouldn't properly delete. The new version on my fork fixes this.

[mungeol]

OK, I will try the new version as well and give you some feedback if there
is any problem.
Thanks.

On Wed, Oct 29, 2014 at 12:34 AM, Isaac Banner notifications@github.com
wrote:

I added a little bit more to my code, you'll want to update what your
using - I forgot to add a failure case, so if it fails for some reason it
wouldn't properly delete. The new version on my fork fixes this.

—
Reply to this email directly or view it on GitHub
#37 (comment)
.