Camel K SEDA Queue full Issue

[haifzhan]

I am configuring seda component in my Camel K project and the java source is:

      .to("seda:next")
      .threads(5)
      .process(myProcessor)

I print out a counter when a message has been processed and I can see the multiple threads are working

I have setup the queue size 50000, it was working fine the first minute and then it keep puking Queue full error.

[3] : java.lang.IllegalStateException: Queue full
[3] 	at java.base/java.util.AbstractQueue.add(AbstractQueue.java:98)
[3] 	at org.apache.camel.component.seda.SedaProducer.addToQueue(SedaProducer.java:251)
[3] 	at org.apache.camel.component.seda.SedaProducer.process(SedaProducer.java:149)
[3] 	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172)
[3] 	at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463)
[3] 	at 

org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179)
[3] 	at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64)
[3] 	at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)
[3] 	at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398)
[3] 	at org.apache.camel.component.netty.handlers.ServerChannelHandler.processAsynchronously(ServerChannelHandler.java:150)
[3] 	at org.apache.camel.component.netty.handlers.ServerChannelHandler.channelRead0(ServerChannelHandler.java:112)
[3] 	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
[3] 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[3] 	at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61)
[3] 	at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370)
[3] 	at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66)
[3] 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
[3] 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
^C[3] 	at java.base/java.lang.Thread.run(Thread.java:829)
[3]
[3] 2021-10-29 23:37:27,310 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #1 - NettyConsumerExecutorGroup) Failed delivery for (MessageId: 477CF9098E8C1B0-000000000000C681 on ExchangeId: 477CF9098E8C1B0-000000000000C681). Exhausted after delivery attempt: 1 caught: java.lang.IllegalStateException: Queue full
[3]
[3] Message History (complete message history is disabled)

Is there any way to solve this issue? I have read the documentation, unfortunately didn't find any solutions yet.

[davsclaus]

Read the camel seda documentation. Where you can block on queue full etc.
https://camel.apache.org/components/next/seda-component.html

[davsclaus]

Just mind that if you block then there is a 30 sec timeout, and that will cause a failure of the queue is still full.

So you likely have a situation where you send messages quicker into the seda queue, than you do taking them out.
You need to control the rate - its also bad to have 50000 pending messages in-memory waiting to process.

[oscerd]

This was explained.