Low polling frequency with a task executor may lead to Rejected tasks building up in the error channel [INT-2988]

[spring-operator]

Biju Kunjummen opened INT-2988 and commented

This may not be an issue at all:

The issue that I am seeing is, if the poller attached to an endpoint is configured with a very low polling frequency (say 10 milliseconds) and dispatches the message handling to a task executor and if the endpoint is slow to process, since the polling task is added to the task executor once the task executor reaches the capacity it starts rejecting tasks(default behavior) and these rejection messages now end up with the errorHandler channel. If by any chance the errorHandler is configured as a queue channel the messages keep building up and can be considered a memory leak.

Here is a dummy configuration with which this scenario can be easily replicated:

	<channel id="input">
		<queue capacity="5"/>
	</channel>
	
	<channel id="output">
		<queue capacity="5"/>
	</channel>
	
	<channel id="errorChannel">
		<queue/>
	</channel>
	
	<service-activator expression="@slowService.slowActivity(payload)" input-channel="input" output-channel="output">
		<poller fixed-delay="20"  max-messages-per-poll="10" task-executor="taskExecutor"></poller>
	</service-activator>

	<beans:bean name="slowService" class="taskexec.SlowService" />
	
	<task:executor id="taskExecutor" pool-size="10" queue-capacity="10"/>

Again, not sure if this is really a bug, but I think a better message will be helpful, since the polling task is being rejected not the real message.

---

Affects: 2.2.3

[spring-operator]

Biju Kunjummen commented

Here is a gist with a sample configuration using which the behavior can be easily replicated: https://gist.github.com/bijukunjummen/5349969

[spring-operator]

Gary Russell commented

Please see this post on Stack Overflow, which explains what's happening.

http://stackoverflow.com/questions/15811831/spring-integration-queue-poller-seems-to-exhaust-threadpool-without-any-action/15813143#15813143

We should add a note to the reference, so I have changed this to a 'Documentation' 'Improvement' issue.

[artembilan]

I believe this can be fixed via CallerRunsPolicy (or CallerBlocksPolicy) on the ThreadPoolTaskExecutor, where the default one is indeed just an AbortPolicy.

Out of Spring Integration scope, though.