Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

FATAL:ngx_thread_system.cc(43) Check failed: may_start_threads_. #237

Closed
abh opened this Issue Apr 6, 2013 · 20 comments

Comments

Projects
None yet
7 participants

abh commented Apr 6, 2013

Since upgrading from acec3ae (~March 20th) to c127e43 (Apr 5th) I get this error regularly when running .../sbin/nginx -s reload:

[0406/215608:FATAL:ngx_thread_system.cc(43)] Check failed: may_start_threads_. 

The master dies, but the worker and cache manager processes continue to run.

abh commented Apr 6, 2013

I spent a while trying to bisect it, but gave up because most of the commits seems to not even compile so I never found a newer commit that worked. I'm pretty sure I must have done something wrong. :-/

Contributor

jeffkaufman commented Apr 7, 2013

Yikes; it shouldn't do that. I'll look at it on monday.

@abh what OS/nginx version?

abh commented Apr 8, 2013

1.2.7 on x86_64 RHEL 6(.4).

I built nginx with

./configure --add-module=../ngx_pagespeed/ --with-http_ssl_module --with-http_realip_module --with-file-aio --with-ipv6 && make && make install

Contributor

jeffkaufman commented Apr 8, 2013

I'm trying to reproduce this and not succeeding:

$ cd nginx-1.2.7
$ make clean
$ ./configure --prefix=/home/jefftk/nginx \
              --with-http_ssl_module \
              --with-http_realip_module \
              --with-file-aio \
              --with-ipv6 \
              --with-debug \
              --add-module=/home/jefftk/ngx_pagespeed
$ make && make install
$ cd ../ngx_pagespeed
$ test/nginx_system_test.sh 8050 8051 \
      /home/jefftk/modpagespeed-svn-checkout \
      /home/jefftk/nginx/sbin/nginx
...
Failing Tests:
  In-place resource optimization
  In-place resource optimization
  In-place resource optimization
  convert_meta_tags
  insert_dns_prefetch
  insert_dns_prefetch
FAIL.
$ /home/jefftk/nginx/sbin/nginx -s reload -c \
     /home/jefftk/ngx_pagespeed/test/tmp/pagespeed_test.conf
$ grep FATAL ~/ngx_pagespeed/test/tmp/error.log
$ uname -i
x86_64

This is on ubuntu though.

Is there anything else different about your system?

Contributor

jeffkaufman commented Apr 10, 2013

Is this still not working?

abh commented Apr 10, 2013

I haven't had a chance to update and try again yet, sorry!

Contributor

jeffkaufman commented Apr 10, 2013

This particular check has to do with making sure we handle threads and forking properly. We don't want to start any threads before we fork.

Could you send me your config?

Contributor

jeffkaufman commented Apr 15, 2013

@abh is this still not working?

abh commented Apr 17, 2013

Yup, just updated to HEAD and it still happens.

2013/04/17 04:26:13 [alert] 12689#0: [ngx_pagespeed 1.4.0.0-2748] [0417/042613:FATAL:ngx_thread_system.cc(43)] Check failed: may_start_threads_. 
Backtrace:
    nginx: master process /usr/local/nginx/sbin/nginx() [0x488eda]
    nginx: master process /usr/local/nginx/sbin/nginx() [0x47e196]
    nginx: master process /usr/local/nginx/sbin/nginx() [0x486ab4]
    nginx: master process /usr/local/nginx/sbin/nginx() [0x47c498]
    nginx: master process /usr/local/nginx/sbin/nginx() [0x5eff1e]
    /lib64/libpthread.so.0() [0x33da407851]
    /lib64/libc.so.6(clone+0x6d) [0x33da0e890d]

What should I do to get a more useful backtrace?

abh commented Apr 23, 2013

#277 fixes this for me (RHEL 6.4, x86_64, nginx 1.2.8).

Contributor

jlporter commented Apr 23, 2013

Just a quick clarifying point, this error should only happen if you are building PSOL from source with BUILDTYPE = Debug. You aren't seeing this with either the binary PSOL or a release build, correct?

abh commented Apr 23, 2013

Uh, I'm not sure actually.

I checkout the git repository in ../ngx_pagespeed and compile with

#!/bin/sh
 ./configure --add-module=../ngx_pagespeed/ --with-http_ssl_module --with-http_realip_module --with-file-aio      --with-ipv6  && make -j10
Contributor

jeffkaufman commented Apr 23, 2013

@abh are you using --with-debug ? When is your checkout from?

abh commented Apr 23, 2013

I don't use --with-debug. Should I?

The checkout that worked now was from your branch from #277.

The previous commit that worked was listed in the first comment in this ticket, I think. I tried bisect'ing to find another that worked but they all failed in different ways (usually failed building).

Contributor

jeffkaufman commented Apr 23, 2013

@abh you don't need to use --with-debug but if you're not using it then it's kind of strange that you got this problem.

We know the branch from #277 works and c127e43 does not. If we want to narrow this down further the next thing to check would be whether current master (which doesn't include the change in #277) works for you. Or, if you'd like to just move on with things, given that it's working for you now, that's ok too.

Hi devs,
I've just encountered with this after trying to get pagespeed included into nginx.

System:
Debian Wheezy/Testing up to date, kernel 3.2.0-4-amd64

my nginx upgrade script ( should reproduce the issue )
http://petermolnar.eu/download/upgrade.nginx-version -> this uses the HEAD of master from git, meaning the version I checked is 6cfdc06.

pagespeed config in nginx.conf http section:
pagespeed on;
pagespeed FileCachePath /var/cache/nginx/;
pagespeed MemcachedThreads 2;
pagespeed MemcachedServers "127.0.0.1:11211";
pagespeed RewriteLevel PassThrough;
pagespeed EnableFilters collapse_whitespace,remove_comments,insert_image_dimensions,extend_cache,combine_css,combine_heads,move_css_to_head,trim_urls,convert_jpeg_to_progressive,recompress_jpeg,recompress_png,convert_gif_to_png,convert_jpeg_to_webp,convert_to_webp_lossless;

How can I help solving this?

Confirming this.
I switched to nginx, all is fine, but ngx_pagespeed is TOTALLY UNUSABLE for now.
I tried it with latest versions, 1.4.2 (stable), 1.5.3 (mainline) and also tried to compile all from source (including psol).
Always same error. Nginx stops responding after second+ ab or siege test.
Let's try:
ab -n 1000 -c 10 http://artactivator.com/
for first time, all goes fine. 50 req/s for drupal index.php page
but second time, there are timeouts, ab stops working. Ok, trying siege.
Req/s slowing slowing down each test, until nginx will give 500 error

In nginx's error log I receive such errors randomly:
Check failed: may_start_threads_
NgxBaseFetch::RequestCollection: Broken pipe

Disabling ngx_pagespeed shows about 50 req/s every stresstest on every page and nginx working fine.
I want to use it, but don't want to switch back to apache....

#237 (comment)

try this if you have apache + memcached + ngixn

location / {
set $memcached_key "$uri?$args";
memcached_pass localhost: 11211;

error_page 403 404 405 500 502 504 =200 @memcached;
}
location @memcached {
proxy_pass http://ip server: port;
proxy_redirect http:// $host: port/ /;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}

Contributor

jeffkaufman commented Jun 20, 2014

This was fixed a while ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment