New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to Camel route information causes a large number of requests which clobber the UI #1903

Closed
rmoquin opened this Issue Feb 12, 2015 · 14 comments

Comments

Projects
None yet
4 participants
@rmoquin

rmoquin commented Feb 12, 2015

I'm opening a new one for this, even though I see things that could be potentially related. I've been seeing a problem with multiple versions of hawt.io where anytime a count changes on a Camel route, the dynatree containing the route list immediately becomes empty and so does the rest of the Camel information. I decided to finally see if I could get any information on why the dynatree becomes blank after something like a count changes. I noticed two things. If I tell Chrome to show paint rectangles, when I exercise a camel route (causing the data in hawt.io to change in the display), I'll see the Camel Contexts dynatree gets hammered by DOM updates (it must be) caused by a large amount of ajax requests from hawt.io in response to the Camel metric change. If I look in the network timeline, you can see the updates going every 5 seconds and when the Camel data changes, you see a lot of requests going out at the same time.

Attached is a screenshot, notice it shows the page was last refresh about 20 seconds previously but shows that in that 20 second period, 72 total ajax requests were made. Notice that there are a large number of jolokia requests occurring at the same time. I think all the updates from the requests clobber the Camel Contexts dynatree. It make hawtio virtually unusable.
hawtio_lots_of_requests

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Feb 13, 2015

Member

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and thus causes hawtio to redraw the tree. There is new releases on the way that will fix this.

Member

davsclaus commented Feb 13, 2015

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and thus causes hawtio to redraw the tree. There is new releases on the way that will fix this.

@davsclaus davsclaus added the question label Feb 13, 2015

@gashcrumb

This comment has been minimized.

Show comment
Hide comment
@gashcrumb

gashcrumb Feb 13, 2015

Member

Also can you expand one of those requests and show either the request headers or request body? Would helps us track down where in the code those requests are coming from. And yeah, hawtio version would be good too, there's a bit of caching for most views that would cut down on ajax requests but not everywhere...

Member

gashcrumb commented Feb 13, 2015

Also can you expand one of those requests and show either the request headers or request body? Would helps us track down where in the code those requests are coming from. And yeah, hawtio version would be good too, there's a bit of caching for most views that would cut down on ajax requests but not everywhere...

@rmoquin

This comment has been minimized.

Show comment
Hide comment
@rmoquin

rmoquin Feb 16, 2015

After I sent this I realized I didn't give you much detail to go on. I'm
currently using ActiveMQ 5.11.0 and I've tried hawtio 1.4.47, as well as 46
and 45... the problem varies a bit between releases but I didn't remember
having this problem until recently.

Could definitely be AMQ. I can send more detail if needed. It happens in
the latest karaf and only really seems to affect the camel component.

Ryan
On Feb 13, 2015 6:58 AM, "Claus Ibsen" notifications@github.com wrote:

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and
thus causes hawtio to redraw the tree. There is new releases on the way
that will fix this.


Reply to this email directly or view it on GitHub
#1903 (comment).

rmoquin commented Feb 16, 2015

After I sent this I realized I didn't give you much detail to go on. I'm
currently using ActiveMQ 5.11.0 and I've tried hawtio 1.4.47, as well as 46
and 45... the problem varies a bit between releases but I didn't remember
having this problem until recently.

Could definitely be AMQ. I can send more detail if needed. It happens in
the latest karaf and only really seems to affect the camel component.

Ryan
On Feb 13, 2015 6:58 AM, "Claus Ibsen" notifications@github.com wrote:

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and
thus causes hawtio to redraw the tree. There is new releases on the way
that will fix this.


Reply to this email directly or view it on GitHub
#1903 (comment).

@rmoquin

This comment has been minimized.

Show comment
Hide comment
@rmoquin

rmoquin Feb 16, 2015

Oh and camel 2.14.1....
On Feb 13, 2015 6:58 AM, "Claus Ibsen" notifications@github.com wrote:

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and
thus causes hawtio to redraw the tree. There is new releases on the way
that will fix this.


Reply to this email directly or view it on GitHub
#1903 (comment).

rmoquin commented Feb 16, 2015

Oh and camel 2.14.1....
On Feb 13, 2015 6:58 AM, "Claus Ibsen" notifications@github.com wrote:

What version of ActiveMQ, Camel and hawtio do you use?

There is a bug in latest AMQ release that causes it to update mbeans, and
thus causes hawtio to redraw the tree. There is new releases on the way
that will fix this.


Reply to this email directly or view it on GitHub
#1903 (comment).

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Feb 17, 2015

Member

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it before its public GA
http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html

Member

davsclaus commented Feb 17, 2015

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it before its public GA
http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html

@rmoquin

This comment has been minimized.

Show comment
Hide comment
@rmoquin

rmoquin Feb 17, 2015

Awesome, thanks Claus!

Ryan
On Feb 17, 2015 1:51 AM, "Claus Ibsen" notifications@github.com wrote:

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it
before its public GA

http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html


Reply to this email directly or view it on GitHub
#1903 (comment).

rmoquin commented Feb 17, 2015

Awesome, thanks Claus!

Ryan
On Feb 17, 2015 1:51 AM, "Claus Ibsen" notifications@github.com wrote:

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it
before its public GA

http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html


Reply to this email directly or view it on GitHub
#1903 (comment).

@rmoquin

This comment has been minimized.

Show comment
Hide comment
@rmoquin

rmoquin Feb 17, 2015

Hi Claus,

Unfortunately I still hit the same problem with ActiveMQ 5.12-SNAPSHOT on
Karaf 3.0.3, Camel 2.15-SNAPSHOT and Hawtio 1.4.47. 1.4.46 seems exhibits
the same problem, but it's much less severe. I'll get you some logs
showing where the calls are coming from.

Ryan

On Tue, Feb 17, 2015 at 11:05 AM, Ryan Moquin fragility2.0@gmail.com
wrote:

Awesome, thanks Claus!

Ryan
On Feb 17, 2015 1:51 AM, "Claus Ibsen" notifications@github.com wrote:

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it
before its public GA

http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html


Reply to this email directly or view it on GitHub
#1903 (comment).

rmoquin commented Feb 17, 2015

Hi Claus,

Unfortunately I still hit the same problem with ActiveMQ 5.12-SNAPSHOT on
Karaf 3.0.3, Camel 2.15-SNAPSHOT and Hawtio 1.4.47. 1.4.46 seems exhibits
the same problem, but it's much less severe. I'll get you some logs
showing where the calls are coming from.

Ryan

On Tue, Feb 17, 2015 at 11:05 AM, Ryan Moquin fragility2.0@gmail.com
wrote:

Awesome, thanks Claus!

Ryan
On Feb 17, 2015 1:51 AM, "Claus Ibsen" notifications@github.com wrote:

Yes its a bug in ActiveMQ 5.11.0. A 5.11.1 is on the way, you can try it
before its public GA

http://activemq.2283324.n4.nabble.com/VOTE-Release-Apache-ActiveMQ-5-11-1-td4691535.html


Reply to this email directly or view it on GitHub
#1903 (comment).

@rmoquin

This comment has been minimized.

Show comment
Hide comment
@rmoquin

rmoquin Feb 26, 2015

It seems like ActiveMQ 5.11.1 is better than 5.11 sometimes, such as when updates occur, sometimes the midde panel of camel details won't blank out and only the Camel tree view blanks out. Here is a screenshot after a camel count updated. I will see the count increment and then immediately the screenshot is usually the result. Also, an inflight count change doesn't cause this problem, only when something completes a route, or if a route is reloaded. If the route is reloaded, it blanks out never comes back unless you swap tabs or refresh the page.

screen shot 2015-02-26 at 7 41 50 am

This won't be near as helpful as if I was using the uncompressed script, which I think I can use if I install the dev feature in Karaf? These stacks (even though they paste messy) look like they could still be useful. These are a couple of the pieces of code that all triggered as soon as a message exchange completed going through a route. There are 3 distinct traces and 22 total calls when the camel route completes. It appears the DynatreeNode.activate is the culprit.

This is the first of the 22 and is called once and looks to be the normal refresh every 5 seconds:


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:16


This is triggered second and only once.


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
h @ app.js?f9f85ff34e487c35:17
c.maybeReloadTree @ app.js?f9f85ff34e487c35:54
(anonymous function) @ app.js?f9f85ff34e487c35:13
lcb @ app.js?f9f85ff34e487c35:16
r.success @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:16
g.success @ app.js?f9f85ff34e487c35:16
l @ app.js?f9f85ff34e487c35:1
m.fireWith @ app.js?f9f85ff34e487c35:1
d @ app.js?f9f85ff34e487c35:3
d @ app.js?f9f85ff34e487c35:4


This is triggered 3rd but the call is repeated 20 times.


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:77
i @ app.js?f9f85ff34e487c35:14
(anonymous function) @ app.js?f9f85ff34e487c35:14
f.$eval @ app.js?f9f85ff34e487c35:14
f.$digest @ app.js?f9f85ff34e487c35:14
f.$apply @ app.js?f9f85ff34e487c35:14
C @ app.js?f9f85ff34e487c35:54
e.length.h @ app.js?f9f85ff34e487c35:54
DynaTreeNode._activate @ app.js?f9f85ff34e487c35:25
DynaTreeNode.activate @ app.js?f9f85ff34e487c35:25
h @ app.js?f9f85ff34e487c35:54
h @ app.js?f9f85ff34e487c35:62
(anonymous function) @ app.js?f9f85ff34e487c35:62

rmoquin commented Feb 26, 2015

It seems like ActiveMQ 5.11.1 is better than 5.11 sometimes, such as when updates occur, sometimes the midde panel of camel details won't blank out and only the Camel tree view blanks out. Here is a screenshot after a camel count updated. I will see the count increment and then immediately the screenshot is usually the result. Also, an inflight count change doesn't cause this problem, only when something completes a route, or if a route is reloaded. If the route is reloaded, it blanks out never comes back unless you swap tabs or refresh the page.

screen shot 2015-02-26 at 7 41 50 am

This won't be near as helpful as if I was using the uncompressed script, which I think I can use if I install the dev feature in Karaf? These stacks (even though they paste messy) look like they could still be useful. These are a couple of the pieces of code that all triggered as soon as a message exchange completed going through a route. There are 3 distinct traces and 22 total calls when the camel route completes. It appears the DynatreeNode.activate is the culprit.

This is the first of the 22 and is called once and looks to be the normal refresh every 5 seconds:


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:16


This is triggered second and only once.


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
h @ app.js?f9f85ff34e487c35:17
c.maybeReloadTree @ app.js?f9f85ff34e487c35:54
(anonymous function) @ app.js?f9f85ff34e487c35:13
lcb @ app.js?f9f85ff34e487c35:16
r.success @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:16
g.success @ app.js?f9f85ff34e487c35:16
l @ app.js?f9f85ff34e487c35:1
m.fireWith @ app.js?f9f85ff34e487c35:1
d @ app.js?f9f85ff34e487c35:3
d @ app.js?f9f85ff34e487c35:4


This is triggered 3rd but the call is repeated 20 times.


send @ app.js?f9f85ff34e487c35:4
$.extend.ajax @ app.js?f9f85ff34e487c35:3
request @ app.js?f9f85ff34e487c35:16
(anonymous function) @ app.js?f9f85ff34e487c35:77
i @ app.js?f9f85ff34e487c35:14
(anonymous function) @ app.js?f9f85ff34e487c35:14
f.$eval @ app.js?f9f85ff34e487c35:14
f.$digest @ app.js?f9f85ff34e487c35:14
f.$apply @ app.js?f9f85ff34e487c35:14
C @ app.js?f9f85ff34e487c35:54
e.length.h @ app.js?f9f85ff34e487c35:54
DynaTreeNode._activate @ app.js?f9f85ff34e487c35:25
DynaTreeNode.activate @ app.js?f9f85ff34e487c35:25
h @ app.js?f9f85ff34e487c35:54
h @ app.js?f9f85ff34e487c35:62
(anonymous function) @ app.js?f9f85ff34e487c35:62

@jcable

This comment has been minimized.

Show comment
Hide comment
@jcable

jcable Mar 30, 2015

I'm having this problem too. Is there any update? I'll check out tomorrow with some AMQ-free routes and confirm it's AMQ causing the problem for me too.

jcable commented Mar 30, 2015

I'm having this problem too. Is there any update? I'll check out tomorrow with some AMQ-free routes and confirm it's AMQ causing the problem for me too.

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Nov 21, 2015

Member

Yep got this today with 5.12.1 on Karaf 4.0.3 with Camel 2.16 and 2.17-SNAPSHOT.

Member

davsclaus commented Nov 21, 2015

Yep got this today with 5.12.1 on Karaf 4.0.3 with Camel 2.16 and 2.17-SNAPSHOT.

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Nov 21, 2015

Member

We had this issue before in AMQ where it will add/remove mbeans all the time.

Member

davsclaus commented Nov 21, 2015

We had this issue before in AMQ where it will add/remove mbeans all the time.

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Nov 21, 2015

Member

Ah you likely need to use a pooled connection factory with AMQ so the connection is re-used which causes the mbeans to not be created all the time
http://camel.apache.org/activemq

Member

davsclaus commented Nov 21, 2015

Ah you likely need to use a pooled connection factory with AMQ so the connection is re-used which causes the mbeans to not be created all the time
http://camel.apache.org/activemq

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Nov 21, 2015

Member

Here is how I configured a connection pool that works

  <!-- the ConnectionFactory to connect to the JMS broker -->
  <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
    <property name="brokerURL" value="tcp://localhost:61616"/>
    <property name="userName" value="karaf"/>
    <property name="password" value="karaf"/>
  </bean>

  <!-- pool the connection factory so it runs faster -->
  <bean id="pooledConnectionFactory"
        class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
    <property name="maxConnections" value="8"/>
    <property name="connectionFactory" ref="jmsConnectionFactory"/>
  </bean>

Member

davsclaus commented Nov 21, 2015

Here is how I configured a connection pool that works

  <!-- the ConnectionFactory to connect to the JMS broker -->
  <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
    <property name="brokerURL" value="tcp://localhost:61616"/>
    <property name="userName" value="karaf"/>
    <property name="password" value="karaf"/>
  </bean>

  <!-- pool the connection factory so it runs faster -->
  <bean id="pooledConnectionFactory"
        class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
    <property name="maxConnections" value="8"/>
    <property name="connectionFactory" ref="jmsConnectionFactory"/>
  </bean>

@davsclaus

This comment has been minimized.

Show comment
Hide comment
@davsclaus

davsclaus Nov 22, 2015

Member

Yay we now filter out those events that would cause this update. The next release should be good.

Member

davsclaus commented Nov 22, 2015

Yay we now filter out those events that would cause this update. The next release should be good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment