requirement failed message in 0.11-M4

[xelax]

While running 0.11-M4 I've got this error:

2016-08-10 10:10:50,056 ERROR a.a.OneForOneStrategy - requirement failed
java.lang.IllegalArgumentException: requirement failed
at scala.Predef$.require(Predef.scala:212)
at akka.kafka.KafkaConsumerActor.poll(KafkaConsumerActor.scala:182)
at akka.kafka.KafkaConsumerActor$$anonfun$receive$1.applyOrElse(KafkaConsumerActor.scala:101)
at akka.actor.Actor$class.aroundReceive(Actor.scala:484)
at akka.kafka.KafkaConsumerActor.aroundReceive(KafkaConsumerActor.scala:56)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

[kciesielski]

Can you reproduce this error somehow? It is caused by internal .poll() call returning messages for different partitions than expected. It would be good to have a failing test for this.

[xelax]

I got it when I had 3 different instances of a listener on the same queue and then I kill one of them, and I see the error in the console of the other one. I will try to see whether I can trigger this reliably.

[kciesielski]

Thanks, @xelax

@13h3r I suspect that consumer assignment changed before poll() (https://github.com/akka/reactive-kafka/blob/master/core/src/main/scala/akka/kafka/KafkaConsumerActor.scala#L167) hence poll returned more than expected, WDYT?

[xelax]

Yes, it sounds likely to me, besides my listeners are getting updates in large batches with few thousands of largish messages in them, so it is quite possible that while getting the messages the consumer assignment get changed.

[13h3r]

@xelax did you use auto or manual partition assignment assignment?

@kciesielski How assignment may be changed before poll? And how it may broke the code?

[13h3r]

Please try build from master. Looks like #179 fixes this

[kciesielski]

@13h3r It can be changed by rebalance, but as you pointed to #179 I can see now that pause will be called, so it should be properly handled.

[13h3r]

Could you check it? //@xelax

[xelax]

sorry, was out on vacation last week. I will build and try it out today or tomorrow.
thanks!

On Aug 15, 2016, at 8:52 AM, Alexey Romanchuk notifications@github.com wrote:

Could you check it? //@xelax https://github.com/xelax
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub #189 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AAav9IBylSl_XuhwNPQwf8hRQtlOo7nbks5qgIstgaJpZM4JhXoz.

[xelax]

I tried it out with the new build and the problem seems to be solved.