Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PHP7] Crash of php7.0-fpm with opcache and opcache.fast_shutdown #322

Closed
mfn opened this issue Apr 15, 2016 · 5 comments
Closed

[PHP7] Crash of php7.0-fpm with opcache and opcache.fast_shutdown #322

mfn opened this issue Apr 15, 2016 · 5 comments

Comments

@mfn
Copy link

mfn commented Apr 15, 2016

We're running 5.6/7.0 alongside and when enabling opcache with opcache.fast_shutdown=1 in PHP7 we experienced crashes almost instantly.

The log entry from /var/log/upstart/php7.0-fpm.log:

child 26210 exited on signal 11 (SIGSEGV) after 56.330280 seconds from start

Removing opcache.fast_shutdown=1 makes this problem go away. The same setting, on the same server, is activated for PHP 5.6 without problems.

xdebug is only installed but not enabled. I've only enabled the following php modules to trigger that crash (the minimum I needed to run the app and trigger the SIGSEGV):

  • opcache
  • pdo
  • json
  • pgsql
  • pdo_pgsql

The crash doesn't happen for simple scripts. We experience this crash with CakePHP 2.8.3 (the latest in the 2.x line).

According to #300 (comment) I installed the *dbgsym packages and was able to produce this backtrace of a crashing fpm child:

Program received signal SIGSEGV, Segmentation fault.
0x00007f358e7434a0 in resetPQExpBuffer () from /usr/lib/x86_64-linux-gnu/libpq.so.5
(gdb) bt
#0  0x00007f358e7434a0 in resetPQExpBuffer () from /usr/lib/x86_64-linux-gnu/libpq.so.5
#1  0x00007f358e736cf7 in ?? () from /usr/lib/x86_64-linux-gnu/libpq.so.5
#2  0x00007f358e7386cb in PQsendQuery () from /usr/lib/x86_64-linux-gnu/libpq.so.5
#3  0x00007f358e739d21 in PQexec () from /usr/lib/x86_64-linux-gnu/libpq.so.5
#4  0x00007f358e95d733 in pgsql_stmt_dtor (stmt=0x7f358585f540) at /build/php7.0-1U3tsT/php7.0-7.0.5/ext/pdo_pgsql/pgsql_statement.c:64
#5  0x00007f358ed7cd6a in php_pdo_free_statement (stmt=0x7f358585f540) at /build/php7.0-1U3tsT/php7.0-7.0.5/ext/pdo/pdo_stmt.c:2316
#6  0x00007f3592162941 in zend_objects_store_del (object=0x7f358585f690) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_objects_API.c:182
#7  0x00007f3592128926 in _zval_dtor_func_for_ptr (p=<optimized out>) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_variables.c:109
#8  0x00007f3592128959 in i_zval_ptr_dtor (zval_ptr=0x7f358f6786f8) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_variables.h:58
#9  _zval_dtor_func_for_ptr (p=0x7f358f6786f0) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_variables.c:122
#10 0x00007f359215e014 in i_zval_ptr_dtor (zval_ptr=0x7f358f684228) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_variables.h:58
#11 zend_object_std_dtor (object=0x7f358f684100) at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_objects.c:69
#12 0x00007f35921625a0 in zend_objects_store_free_object_storage (objects=0x7f3585830008, objects@entry=0x7f359250b530 <executor_globals+816>)
    at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_objects_API.c:103
#13 0x00007f359211b913 in shutdown_executor () at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend_execute_API.c:357
#14 0x00007f359212a1c5 in zend_deactivate () at /build/php7.0-1U3tsT/php7.0-7.0.5/Zend/zend.c:967
#15 0x00007f35920ccee1 in php_request_shutdown (dummy=<optimized out>) at /build/php7.0-1U3tsT/php7.0-7.0.5/main/main.c:1826
#16 0x00007f3591fbf9c6 in main (argc=<optimized out>, argv=<optimized out>) at /build/php7.0-1U3tsT/php7.0-7.0.5/sapi/fpm/fpm/fpm_main.c:1996

Google lead led me to https://bugs.php.net/bug.php?id=71573 which is very recent, but explicitly says opcache is disabled already.

Very likely this is not related to your packages, but I'm not sure how to take it from here. I'm unable to come up with a small reproducible script :-/

So in spirit of the never-forgetting internet I created it here, even if it gets closed immediately maybe it helps someone else.

Packages from 7.0.5-2+deb.sury.org~trusty+1

ii  php-common                          1:36+deb.sury.org~trusty+1
ii  php-igbinary                        1.2.1-10-ge0e66b9+1.2.1-2+deb.sury.org~trusty+6
ii  php-redis                           2.2.7-389-g2887ad1+2.2.7-1+deb.sury.org~trusty+1
ii  php-xdebug                          2.4.0-1+deb.sury.org~trusty+1
ii  php5.6-cli                          5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-common                       5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-curl                         5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-fpm                          5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-gd                           5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-intl                         5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-json                         1.3.9-2+deb.sury.org~trusty+1
ii  php5.6-mbstring                     5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-mcrypt                       5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-mysql                        5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-opcache                      5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-pgsql                        5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-readline                     5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php5.6-xml                          5.6.20+dfsg-2+deb.sury.org~trusty+1
ii  php7.0-cli                          7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-common                       7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-curl                         7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-fpm                          7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-gd                           7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-intl                         7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-json                         7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-mbstring                     7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-mcrypt                       7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-mysql                        7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-opcache                      7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-pgsql                        7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-readline                     7.0.5-2+deb.sury.org~trusty+1
ii  php7.0-xml                          7.0.5-2+deb.sury.org~trusty+1

ps: thanks for your wonderful work!
@oerdnj
Copy link
Owner

oerdnj commented Apr 15, 2016

@mfn Also try installing postgresql-X.Y-dbg (where X.Y is your postgresql version) to produce a full backtrace and I suggest you add the full backtrace to the upstream bug. That might help them debug the issue.

Also if you can reliably reproduce the bug, I suggest to add it to the bug as well even though it is not a simple script. A reproducible bug is always easier to fix.

@mfn
Copy link
Author

mfn commented Apr 16, 2016

Thanks for the encouragement. It's totally reproducible for me; here's an updated stacktrace with pgsql db:

Program received signal SIGSEGV, Segmentation fault.
resetPQExpBuffer (str=str@entry=0x7fb575b5fe10) at /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/pqexpbuffer.c:152
152     /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/pqexpbuffer.c: No such file or directory.
(gdb) bt
#0  resetPQExpBuffer (str=str@entry=0x7fb575b5fe10) at /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/pqexpbuffer.c:152
#1  0x00007fb568e99cf7 in PQsendQueryStart (conn=conn@entry=0x7fb575b5fab8) at /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/fe-exec.c:1352
#2  0x00007fb568e9b6cb in PQsendQuery (conn=conn@entry=0x7fb575b5fab8, query=query@entry=0x7fb575ba2550 "DEALLOCATE pdo_stmt_00000007")
    at /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/fe-exec.c:1115
#3  0x00007fb568e9cd21 in PQexec (conn=0x7fb575b5fab8, query=0x7fb575ba2550 "DEALLOCATE pdo_stmt_00000007")
    at /build/postgresql-9.5-DVsMQj/postgresql-9.5-9.5.2/build/../src/interfaces/libpq/fe-exec.c:1829
#4  0x00007fb5690c0733 in pgsql_stmt_dtor (stmt=0x7fb55dc72380) at /build/php7.0-kNWWO9/php7.0-7.0.5/ext/pdo_pgsql/pgsql_statement.c:64
#5  0x00007fb574f39d6a in php_pdo_free_statement (stmt=0x7fb55dc72380) at /build/php7.0-kNWWO9/php7.0-7.0.5/ext/pdo/pdo_stmt.c:2316
#6  0x00007fb57845b941 in zend_objects_store_del (object=0x7fb55dc724d0) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_objects_API.c:182
#7  0x00007fb578421926 in _zval_dtor_func_for_ptr (p=<optimized out>) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_variables.c:109
#8  0x00007fb578421959 in i_zval_ptr_dtor (zval_ptr=0x7fb575a98920) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_variables.h:58
#9  _zval_dtor_func_for_ptr (p=0x7fb575a98918) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_variables.c:122
#10 0x00007fb578457014 in i_zval_ptr_dtor (zval_ptr=0x7fb575a60928) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_variables.h:58
#11 zend_object_std_dtor (object=0x7fb575a60800) at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_objects.c:69
#12 0x00007fb57845b5a0 in zend_objects_store_free_object_storage (objects=0x7fb575b5fe10, objects@entry=0x7fb578804530 <executor_globals+816>)
    at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_objects_API.c:103
#13 0x00007fb578414913 in shutdown_executor () at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend_execute_API.c:357
#14 0x00007fb5784231c5 in zend_deactivate () at /build/php7.0-kNWWO9/php7.0-7.0.5/Zend/zend.c:967
#15 0x00007fb5783c5ee1 in php_request_shutdown (dummy=<optimized out>) at /build/php7.0-kNWWO9/php7.0-7.0.5/main/main.c:1826
#16 0x00007fb5782b89c6 in main (argc=<optimized out>, argv=<optimized out>) at /build/php7.0-kNWWO9/php7.0-7.0.5/sapi/fpm/fpm/fpm_main.c:1996

I'll add this information to the PHP report too.

@oerdnj
Copy link
Owner

oerdnj commented May 13, 2016

I've pulled fix for PHP#71573 into ppa:ondrej/php-qa version php7.0_7.0.7~rc1-1+donate.sury.org~*+3, please try when it finishes building.

@mfn
Copy link
Author

mfn commented May 13, 2016

Thanks, I did => no change, immediately crashes as soon as opcache.fast_shutdown is enabled.

@oerdnj oerdnj mentioned this issue Jul 11, 2016
Closed
@mfn
Copy link
Author

mfn commented Nov 19, 2019

Haven't seen this crash anymore, also not using this PHP version anymore 🤷‍♀️ => closing

@mfn mfn closed this as completed Nov 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mfn @oerdnj