New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spin in serf #1154

Closed
jeffkaufman opened this Issue Oct 15, 2015 · 5 comments

Comments

Projects
None yet
2 participants
@jeffkaufman
Copy link
Contributor

jeffkaufman commented Oct 15, 2015

We have a report of the serf thread getting into 100% cpu usage. Symbolized backtrace:

Thread 10 (Thread 0x7f9e3bac3700 (LWP 17903)):
#0  0x00007f9e505a4b10 in serf_bucket_aggregate_prepend_iovec () from /usr/lib/apache2/modules/mod_pagespeed.so
#1  0x00007f9e505a17e0 in ssl_encrypt () from /usr/lib/apache2/modules/mod_pagespeed.so
#2  0x00007f9e5059f048 in common_databuf_prep () from /usr/lib/apache2/modules/mod_pagespeed.so
#3  0x00007f9e5059f0ac in serf_databuf_read () from /usr/lib/apache2/modules/mod_pagespeed.so
#4  0x00007f9e5059ef42 in serf_default_read_iovec () from /usr/lib/apache2/modules/mod_pagespeed.so
#5  0x00007f9e505a46c8 in read_aggregate () from /usr/lib/apache2/modules/mod_pagespeed.so
#6  0x00007f9e505a3f90 in serf__process_connection () from /usr/lib/apache2/modules/mod_pagespeed.so
#7  0x00007f9e5059ebda in serf_event_trigger () from /usr/lib/apache2/modules/mod_pagespeed.so
#8  0x00007f9e5059ed4f in serf_context_run () from /usr/lib/apache2/modules/mod_pagespeed.so
#9  0x00007f9e5059a515 in net_instaweb::SerfUrlAsyncFetcher::Poll(long) () from /usr/lib/apache2/modules/mod_pagespeed.so
#10 0x00007f9e5059ae25 in net_instaweb::SerfThreadedFetcher::SerfThreadFn(apr_thread_t*, void*) () from /usr/lib/apache2/modules/mod_pagespeed.so
#11 0x00007f9e54d26b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#12 0x00007f9e54a7070d in clone () from /lib/x86_64-linux-gnu/libc.so.6
#13 0x0000000000000000 in ?? ()
@jeffkaufman

This comment has been minimized.

Copy link
Contributor

jeffkaufman commented Oct 15, 2015

This is in a forked copy of serf's ssl_buckets.c, which is very out of date. The file has changed a lot, and this section has a lot of extra checks. I suspect this bug is fixed upstream.

We've been meaning to update our copy of serf for a while. I'll do it now.

@jeffkaufman

This comment has been minimized.

Copy link
Contributor

jeffkaufman commented Oct 16, 2015

Targeting this for the initial 1.10 beta: our copy of serf is heavily modified, enough that this is too complex a change for a 1.9 point release.

@jeffkaufman

This comment has been minimized.

Copy link
Contributor

jeffkaufman commented Dec 8, 2015

This is very likely fixed with the upgrade to the latest stable serf in 468997a

@jeffkaufman jeffkaufman closed this Dec 8, 2015

@crowell

This comment has been minimized.

Copy link
Contributor

crowell commented Dec 8, 2015

note that this updated serf is only in branch 33, and is not present in current stable or beta 1.9.32.x releases..

@jeffkaufman

This comment has been minimized.

Copy link
Contributor

jeffkaufman commented Dec 8, 2015

Yes, this fix will go out in 1.10.

We integrate very tightly with serf, so its upgrade is intrusive enough that including it in a 1.9 bugfix release would be iffy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment