Gearman-manager processes take 100% CPU even with no jobs running

[bartzy]

Hi,

Running Gearman-manager (installing with install.sh, Debian squeeze 32-bit), with 1 job file with dedicated_count=4, the CPU utilization of the Gearman-manager processes is 100%.

No jobs are being sent through gearmand.

How can I debug this?

Thanks,
Bar.

[brianlmoon]

You would need to strace the process to see what it is doing.

[Bellardia]

I'm having the same issue.. Processes spawn up and consume 100% of cpu (with no work to do).

strace:

rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
time(NULL) = 1320026681
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, ~[RTMIN RT_1], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
time(NULL) = 1320026681

[dbaltas]

I was also seeing php processes taking too much CPU when loading the pecl-manager of GearmanManager.
Tested on gearman 0.34 and gearman PECL client 0.83 and 0.81

When I removed the "dedicated_only = true" from the job config, the php processes CPU usage went down to almost nothing.

[GearmanManager]
count = 3
host = localhost:4730
worker_dir=./pecl-workers

[hello_job]
dedicated_count = 5
dedicated_only = true  #removing this resolved the CPU that worker php processes were requiring

The above config produces 5 workers for hello_job with high CPU usage.
Removing the dedicated_only produces 8 workers with low CPU usage

[anthonymanzo]

WTF? How to fix this and use dedicated_only? I am having this problem.

[rubao]

100% CPU

strace:

select(4, [3], [], NULL, {1, 0}) = 1 (in [3], left {0, 999998})
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "S", 1, MSG_PEEK, NULL, NULL) = 1
select(4, [3], [], NULL, {1, 0}) = 1 (in [3], left {0, 999998})
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "S", 1, MSG_PEEK, NULL, NULL) = 1
select(4, [3], [], NULL, {1, 0}) = 1 (in [3], left {0, 999998})
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "S", 1, MSG_PEEK, NULL, NULL) = 1
select(4, [3], [], NULL, {1, 0}) = 1 (in [3], left {0, 999998})
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "S", 1, MSG_PEEK, NULL, NULL) = 1
select(4, [3], [], NULL, {1, 0}) = 1 (in [3], left {0, 999998})
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 1 ([{fd=3, revents=POLLIN}])
recvfrom(3, "S", 1, MSG_PEEK, NULL, NULL) = 1

[pleasereachmeoutonmoeinxyz]

same problem!

[brianlmoon]

Hmm, that strace is all stuff that happens in the PECL lib, GearmanManager does not handle the connections to the server. Are you using the most recent versions of the PECL lib and libgearman?

[conradjones]

Also what version of PHP are you running.

[dbaltas]

@brianlmoon which version of libgearman and pecl gearman are supported?
One idea might be to download libgearman 1.0.6 and pecl 1.1.1.
For ubuntu 12.04 the default libgearman is 0.x which has known issues.

[conradjones]

It works fine with unbuntu 14.04 and the latest version available in that.

On Fri, Jul 17, 2015 at 1:03 PM, Dimitris Baltas notifications@github.com
wrote:

@brianlmoon https://github.com/brianlmoon which version of libgearman
and pecl gearman are supported?
One idea might be to download libgearman 1.0.6 and pecl 1.1.1.
For ubuntu 12.04 the default libgearman is 0.x which has known issues.

—
Reply to this email directly or view it on GitHub
#13 (comment)
.

[brianlmoon]

@dbaltas There is nothing in GearmanManager that requires any certain version of the PECL library. I personally use https://github.com/brianlmoon/net_gearman. So I don't dog food PECL unfortunately. I can test it, make sure it functions. But, for long running, high performance issues, I am relying on the community to fill in those gaps.