Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DefaultMessageListenerContainer shutdown hangs when using AMQ client with prefetch 0 [SPR-15579] #20138

Open
spring-issuemaster opened this issue May 23, 2017 · 3 comments

Comments

@spring-issuemaster
Copy link
Collaborator

@spring-issuemaster spring-issuemaster commented May 23, 2017

Doug Harmon opened SPR-15579 and commented

JDK 1.8.0_112
spring-jms-4.3.8.RELEASE.jar
activemq-client-5.14.5.jar

When using spring's 4.3.8 DefaultMessageListenerContainer (DMLC) to consume messages from AMQ 5.14.5 server, and using prefetch of 0, spring container hangs when shutting down DMLC (e.g. on tomcat server shutdown).

Applicable 2 threads from spring client app thread dump (logs on separate network so I cannot copy/paste):

"dmlc-1" ...
at org.apache.activemq.FifoMessageDispatchChannel.dequeue(FifoMessageDispatchChannel.java:72)

  • locked <> (a java.lang.Object)
    at org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:486)
    at org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:648)

"http-bio-exec-11"
at org.springframework.jms.listener.DefaultMessageListenerContainer.doShutdown(DefaultMessageListenerContainer.java:571)
...

Applicable spring configuration file:

<bean id="jmsFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0</value>
</property>
</bean>
<bean id="queue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="myQueue" />
</bean>
<bean id="dmlc" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactory" />
<property name="destination" ref="queue" />
<property name="messageListener" ref="myQueueConsumer" />
<property name="sessionTransacted" value="true" />
</bean>


Affects: 4.3.8

Reference URL: https://issues.apache.org/jira/browse/AMQ-6682

Attachments:

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented May 29, 2017

Stéphane Nicoll commented

Can you please attach a sample that reproduces the issues (with the steps to run that locally)

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Aug 21, 2017

Stéphane Nicoll commented

Closing due to lack of feedback.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator Author

@spring-issuemaster spring-issuemaster commented Sep 15, 2017

Itzik Bar David commented

I have experienced the same issue when using DefaultMessageListenerContainer and setting Prefetch=0 in activemq connection.
Please see attached [^spring-jms-shutdown-hangs.zip] for simple reproduction


The problem is in AbstractJmsListeningContainer.java on the shutdown method.
First, it stops the connection and then waits for AsyncMessageListenerInvokers to deactivate themselves. The wait operation hangs.

+Insights+:

  • Since the connection was only stopped (and not closed), it still listens for messages from the broker (in order to have messages in memory for when the connection will be restarted).
  • When using prefetch=0 it doesn't use any timeout when waiting for a message from the broker. so it blocks until a message is available.

+Thoughts+:

  • I can see from the implementation of the shutdown method of AbstractJmsListeningContainer.java, that the shared connection is stopped, and only after waiting for all invokes it is shutdown.
  • What Is the reason for executing it in that order?
    Is it to be able to commit/rollback already running consumers against the broker?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.