PHP 8.0.20 (ZTS) zend_signal_handler_defer crashes on apache #8789

gavgavych · 2022-06-15T14:32:16Z

Description

PHP (zts) and Apache build:

httpd -V
Server version: Apache/2.4.54 (Unix)
Server built:   Jun 14 2022 17:15:53
Server's Module Magic Number: 20120211:124
Server loaded:  APR 1.7.0, APR-UTIL 1.6.1, PCRE 8.39 2016-06-14
Compiled using: APR 1.7.0, APR-UTIL 1.6.1, PCRE 8.39 2016-06-14
Architecture:   64-bit
Server MPM:     event
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)

php --version
PHP 8.0.20 (cli) (built: Jun 14 2022 17:24:49) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v4.0.20, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.20, Copyright (c), by Zend Technologies
    with SourceGuardian v13.0.2, Copyright (c) 2000-2022, by SourceGuardian Ltd.

The following code:

info.php:

<?php

phpinfo();

$: wrk -t 3 -c 3 -d 20 "http://localhost/info.php"

Resulted in this output:

gdb:

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f1f4a448da4 in zend_signal_handler_defer (signo=1, siginfo=0x7f1f0f7cd830, context=0x7f1f0f7cd700) at /php/Zend/zend_signal.c:96
96    /php/Zend/zend_signal.c: No such file or directory.
[Current thread is 1 (Thread 0x7f1f0f7ce700 (LWP 7751))]
(gdb) bt
#0  0x00007f1f4a448da4 in zend_signal_handler_defer (signo=1, siginfo=0x7f1f0f7cd830, context=0x7f1f0f7cd700) at /php/Zend/zend_signal.c:96
#1  <signal handler called>
#2  __pthread_kill (threadid=<optimized out>, signo=1) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
#3  0x00007f1f4acab4e2 in wakeup_listener () from /opt/server/lib/webmanager/mod_mpm_event.so
#4  0x00007f1f4acab51c in signal_threads () from /opt/server/lib/webmanager/mod_mpm_event.so
#5  0x00007f1f4acad4d0 in check_infinite_requests () from /opt/server/lib/webmanager/mod_mpm_event.so
#6  0x00007f1f4acae541 in listener_thread () from /opt/server/lib/webmanager/mod_mpm_event.so
#7  0x000055644d671f09 in thread_start ()
#8  0x00007f1f4af44ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#9  0x00007f1f4ae74def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Slice: system.slice
Boot ID: 10b358fb6fbd4abfac2ee9b9a6f14cca
Machine ID: ff5db65b2db34c9cbf8de05ba4d322e6
Hostname: debian
Storage: /var/lib/systemd/coredump/core.httpd.998.10b358fb6fbd4abfac2ee9b9a6f14cca.7649.1655302730000000.zst
 Message: Process 7649 (httpd) of user 998 dumped core.                
                Stack trace of thread 7751:
                #0  0x00007f1f4a448da4 zend_signal_handler_defer (libphp.so + 0x406da4)
                #1  0x00007f1f4af50140 __restore_rt (libpthread.so.0 + 0x14140)
                #2  0x00007f1f4af4cf44 __pthread_kill (libpthread.so.0 + 0x10f44)
                #3  0x00007f1f4acab4e2 wakeup_listener (mod_mpm_event.so + 0x54e2)
                #4  0x00007f1f4acab51c signal_threads (mod_mpm_event.so + 0x551c)
                #5  0x00007f1f4acad4d0 check_infinite_requests (mod_mpm_event.so + 0x74d0)
                #6  0x00007f1f4acae541 listener_thread (mod_mpm_event.so + 0x8541)
                #7  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #8  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #9  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7733:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x000055644d6b2b05 ap_queue_pop_something (httpd + 0x84b05)
                #2  0x00007f1f4acb020d worker_thread (mod_mpm_event.so + 0xa20d)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7649:
                #0  0x00007f1f4af4f08c __libc_read (libpthread.so.0 + 0x1308c)
                #1  0x000055644d6b12cc ap_mpm_podx_check (httpd + 0x832cc)
                #2  0x00007f1f4acb1abf child_main (mod_mpm_event.so + 0xbabf)
                #3  0x00007f1f4acb1ff7 make_child (mod_mpm_event.so + 0xbff7)
                #4  0x00007f1f4acb208e startup_children (mod_mpm_event.so + 0xc08e)
                #5  0x00007f1f4acb3192 server_main_loop (mod_mpm_event.so + 0xd192)
                #6  0x00007f1f4acb357f event_run (mod_mpm_event.so + 0xd57f)
                #7  0x000055644d67344d ap_run_mpm (httpd + 0x4544d)
                #8  0x000055644d669555 main (httpd + 0x3b555)
                #9  0x00007f1f4ad9dd0a __libc_start_main (libc.so.6 + 0x26d0a)
                #10 0x000055644d6675ba _start (httpd + 0x395ba)
                
                Stack trace of thread 7650:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7651:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7652:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7653:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7654:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 7655:
                #0  0x00007f1f4af4b7b2 futex_wait_cancelable (libpthread.so.0 + 0xf7b2)
                #1  0x00007f1f4a799c11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007f1f4a799dd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x000055644d671f09 thread_start (httpd + 0x43f09)
                #4  0x00007f1f4af44ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007f1f4ae74def __clone (libc.so.6 + 0xfddef)

PHP Version

PHP 8.0.20

Operating System

Debian 11

The text was updated successfully, but these errors were encountered:

cmb69 · 2022-06-15T15:13:28Z

Does that also happen when SourceGuardian is disabled?

gavgavych · 2022-06-15T15:41:00Z

yes, it's the same segfault

php --version
PHP 8.0.20 (cli) (built: Jun 14 2022 17:24:49) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v4.0.20, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.20, Copyright (c), by Zend Technologies

wrk -t 3 -c 3 -d 20 "http://localhost/info.php"

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f1f4a448da4 in zend_signal_handler_defer (signo=1, siginfo=0x7f1f0f7cd830, context=0x7f1f0f7cd700) at /php/Zend/zend_signal.c:96
96     /php/Zend/zend_signal.c: No such file or directory.
[Current thread is 1 (Thread 0x7f1f0f7ce700 (LWP 8496))]
(gdb) bt
#0  0x00007f1f4a448da4 in zend_signal_handler_defer (signo=1, siginfo=0x7f1f0f7cd830, context=0x7f1f0f7cd700) at /php/Zend/zend_signal.c:96
#1  <signal handler called>
#2  __pthread_kill (threadid=<optimized out>, signo=1) at ../sysdeps/unix/sysv/linux/pthread_kill.c:56
#3  0x00007f1f4acab4e2 in wakeup_listener () from /opt/server/lib/webmanager/mod_mpm_event.so
#4  0x00007f1f4acab51c in signal_threads () from /opt/server/lib/webmanager/mod_mpm_event.so
#5  0x00007f1f4acad4d0 in check_infinite_requests () from /opt/server/lib/webmanager/mod_mpm_event.so
#6  0x00007f1f4acae541 in listener_thread () from /opt/server/lib/webmanager/mod_mpm_event.so
#7  0x000055644d671f09 in thread_start ()
#8  0x00007f1f4af44ea7 in start_thread (arg=<optimized out>) at pthread_create.c:477
#9  0x00007f1f4ae74def in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

gavgavych · 2022-06-15T22:52:42Z

Tested on ArchLinux

Apache from ArchLinux "Extra Repo"

httpd -V
Server version: Apache/2.4.53 (Unix)
Server built:   Mar 15 2022 01:43:41
Server's Module Magic Number: 20120211:124
Server loaded:  APR 1.7.0, APR-UTIL 1.6.1, PCRE 10.40 2022-04-14
Compiled using: APR 1.7.0, APR-UTIL 1.6.1, PCRE 10.39 2021-10-29
Architecture:   64-bit
Server MPM:     event
  threaded:     yes (fixed thread count)
    forked:     yes (variable process count)

PHP 8.1.7 (ZTS) from AUR (https://aur.archlinux.org/packages/php-zts)

php --version
PHP 8.1.7 (cli) (built: Jun 15 2022 18:19:58) (ZTS)
Copyright (c) The PHP Group
Zend Engine v4.1.7, Copyright (c) Zend Technologies

With default httpd.conf and php.ini

wrk -t 3 -c 3 -d 20 "http://localhost/info.php"

Hostname: archlinux
       Storage: /var/lib/systemd/coredump/core.httpd.33.6cb5833603d6457894e9a03247f59720.399997.1655332630000000.zst (present)
     Disk Size: 2.2M
       Message: Process 399997 (httpd) of user 33 dumped core.
 Stack trace of thread 400065:
                #0  0x00007f21eee46e49 zend_signal_handler_defer (libphp.so + 0x446e49)
                #1  0x00007f21f163e8e0 n/a (libc.so.6 + 0x3e8e0)
                #2  0x00007f21f1711c9e epoll_wait (libc.so.6 + 0x111c9e)
                #3  0x00007f21f18bc98e n/a (libapr-1.so.0 + 0x2898e)
                #4  0x00007f21f15f8a5a n/a (mod_mpm_event.so + 0xaa5a)
                #5  0x00007f21f168c54d n/a (libc.so.6 + 0x8c54d)
                #6  0x00007f21f1711874 __clone (libc.so.6 + 0x111874)

                Stack trace of thread 400008:
                #0  0x00007f21f1689119 n/a (libc.so.6 + 0x89119)
                #1  0x00007f21f168b920 pthread_cond_wait (libc.so.6 + 0x8b920)
                #2  0x000055e01870413e ap_queue_pop_something (httpd + 0x7313e)
                #3  0x00007f21f15f4add n/a (mod_mpm_event.so + 0x6add)
                #4  0x00007f21f168c54d n/a (libc.so.6 + 0x8c54d)
                #5  0x00007f21f1711874 __clone (libc.so.6 + 0x111874)

                Stack trace of thread 400061:
                #0  0x00007f21f1689119 n/a (libc.so.6 + 0x89119)
                #1  0x00007f21f168b920 pthread_cond_wait (libc.so.6 + 0x8b920)
                #2  0x000055e01870413e ap_queue_pop_something (httpd + 0x7313e)
                #3  0x00007f21f15f4add n/a (mod_mpm_event.so + 0x6add)
                #4  0x00007f21f168c54d n/a (libc.so.6 + 0x8c54d)
                #5  0x00007f21f1711874 __clone (libc.so.6 + 0x111874)

                Stack trace of thread 400039:
                #0  0x00007f21f1689119 n/a (libc.so.6 + 0x89119)
                #1  0x00007f21f168b920 pthread_cond_wait (libc.so.6 + 0x8b920)
                #2  0x000055e01870413e ap_queue_pop_something (httpd + 0x7313e)
                #3  0x00007f21f15f4add n/a (mod_mpm_event.so + 0x6add)
                #4  0x00007f21f168c54d n/a (libc.so.6 + 0x8c54d)
                #5  0x00007f21f1711874 __clone (libc.so.6 + 0x111874)

                Stack trace of thread 400035:
                #0  0x00007f21f1689119 n/a (libc.so.6 + 0x89119)
                #1  0x00007f21f168b920 pthread_cond_wait (libc.so.6 + 0x8b920)
                #2  0x000055e01870413e ap_queue_pop_something (httpd + 0x7313e)
                #3  0x00007f21f15f4add n/a (mod_mpm_event.so + 0x6add)
                #4  0x00007f21f168c54d n/a (libc.so.6 + 0x8c54d)
                #5  0x00007f21f1711874 __clone (libc.so.6 + 0x111874)

[New LWP 400059]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `/usr/bin/httpd -k start -DFOREGROUND'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f21eee46e49 in zend_signal_handler_defer (signo=1, siginfo=0x7f2191ffa7f0, context=0x7f2191ffa6c0) at /usr/src/debug/php-8.1.7/Zend/zend_signal.c:96
96              if (EXPECTED(SIGG(active))) {
[Current thread is 1 (Thread 0x7f2191ffb640 (LWP 400065))]
(gdb) bt
#0  0x00007f21eee46e49 in zend_signal_handler_defer (signo=1, siginfo=0x7f2191ffa7f0, context=0x7f2191ffa6c0) at /usr/src/debug/php-8.1.7/Zend/zend_signal.c:96
#1  <signal handler called>
#2  0x00007f21f1711c9e in epoll_wait () from /usr/lib/libc.so.6
#3  0x00007f21f18bc98e in ?? () from /usr/lib/libapr-1.so.0
#4  0x00007f21f15f8a5a in ?? () from /etc/httpd/modules/mod_mpm_event.so
#5  0x00007f21f168c54d in ?? () from /usr/lib/libc.so.6
#6  0x00007f21f1711874 in clone () from /usr/lib/libc.so.6

So, it's just happen with php 8.1 too

gavgavych · 2022-06-16T14:02:21Z

php 7.4 (ZTS)

php --version
PHP 7.4.30 (cli) (built: Jun 16 2022 13:09:46) ( ZTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.30, Copyright (c), by Zend Technologies

Message: Process 20788 (httpd) of user 998 dumped core.
                
                Stack trace of thread 20890:
                #0  0x00007fc77f4369cb zend_signal_handler_defer (libphp7.so + 0x3829cb)
                #1  0x00007fc78070c140 __restore_rt (libpthread.so.0 + 0x14140)
                #2  0x00007fc780708f44 pthread_kill (libpthread.so.0 + 0x10f44)
                #3  0x0000563c5795bb54 wakeup_listener (httpd + 0x9bb54)
                #4  0x0000563c5795bb8e signal_threads (httpd + 0x9bb8e)
                #5  0x0000563c5795d9c2 check_infinite_requests (httpd + 0x9d9c2)
                #6  0x0000563c5795e99d listener_thread (httpd + 0x9e99d)
                #7  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #8  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #9  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20788:
                #0  0x00007fc78070b08c read (libpthread.so.0 + 0x1308c)
                #1  0x0000563c579434ac ap_mpm_podx_check (httpd + 0x834ac)
                #2  0x0000563c57961c00 child_main (httpd + 0xa1c00)
                #3  0x0000563c579620d9 make_child (httpd + 0xa20d9)
                #4  0x0000563c5796216a startup_children (httpd + 0xa216a)
                #5  0x0000563c5796317f server_main_loop (httpd + 0xa317f)
                #6  0x0000563c57963538 event_run (httpd + 0xa3538)
                #7  0x0000563c5790562d ap_run_mpm (httpd + 0x4562d)
                #8  0x0000563c578fb735 main (httpd + 0x3b735)
                #9  0x00007fc780559d0a __libc_start_main (libc.so.6 + 0x26d0a)
                #10 0x0000563c578f979a _start (httpd + 0x3979a)
                
                Stack trace of thread 20789:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20791:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20790:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20859:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20792:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20793:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20794:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20841:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20842:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20795:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20857:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20797:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20865:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20798:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20796:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20881:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20800:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20799:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20853:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20801:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20802:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20803:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20863:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20804:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20805:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20880:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20806:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20808:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20809:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20864:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20807:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20851:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20811:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20878:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20810:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20812:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20872:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20814:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20813:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20844:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20815:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20869:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20817:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20816:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20886:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20820:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20819:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20888:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20822:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20883:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20818:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20884:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20876:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20823:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20855:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20821:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20824:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20856:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20846:
                #0  0x00007fc780791269 n/a (libpcre.so.3 + 0xc269)
                #1  0x00007fc780791833 pcre_compile2 (libpcre.so.3 + 0xc833)
                #2  0x0000563c57908ca2 ap_regcomp (httpd + 0x48ca2)
                #3  0x0000563c578fe575 ap_pregcomp (httpd + 0x3e575)
                #4  0x00007fc77fea996a cmd_rewriterule (mod_rewrite.so + 0xc96a)
                #5  0x0000563c57931f7c invoke_cmd (httpd + 0x71f7c)
                #6  0x0000563c57933504 ap_walk_config_sub (httpd + 0x73504)
                #7  0x0000563c579335b4 ap_walk_config (httpd + 0x735b4)
                #8  0x0000563c57934ca3 ap_parse_htaccess (httpd + 0x74ca3)
                #9  0x0000563c57924a58 ap_directory_walk (httpd + 0x64a58)
                #10 0x0000563c5791e20d core_map_to_storage (httpd + 0x5e20d)
                #11 0x0000563c579212c4 ap_run_map_to_storage (httpd + 0x612c4)
                #12 0x0000563c579229da ap_process_request_internal (httpd + 0x629da)
                #13 0x0000563c57950bb8 ap_process_async_request (httpd + 0x90bb8)
                #14 0x0000563c5794c382 ap_process_http_async_connection (httpd + 0x8c382)
                #15 0x0000563c5794c57f ap_process_http_connection (httpd + 0x8c57f)
                #16 0x0000563c5793f99e ap_run_process_connection (httpd + 0x7f99e)
                #17 0x0000563c5795d0b4 process_socket (httpd + 0x9d0b4)
                #18 0x0000563c57960601 worker_thread (httpd + 0xa0601)
                #19 0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #20 0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #21 0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20887:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20825:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x00007fc77fefac11 get_next (mod_http2.so + 0x46c11)
                #2  0x00007fc77fefadd2 slot_run (mod_http2.so + 0x46dd2)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)
                
                Stack trace of thread 20885:
                #0  0x00007fc7807077b2 pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0 + 0xf7b2)
                #1  0x0000563c57944ce5 ap_queue_pop_something (httpd + 0x84ce5)
                #2  0x0000563c57960495 worker_thread (httpd + 0xa0495)
                #3  0x0000563c579040e9 thread_start (httpd + 0x440e9)
                #4  0x00007fc780700ea7 start_thread (libpthread.so.0 + 0x8ea7)
                #5  0x00007fc780630def __clone (libc.so.6 + 0xfddef)

Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fc77f4369cb in zend_signal_handler_defer (signo=1, siginfo=0x7fc6c07f7830, context=0x7fc6c07f7700) at /opt/php-7.4.30/Zend/zend_signal.c:96
96      /opt/php-7.4.30/Zend/zend_signal.c: No such file or directory.
[Current thread is 1 (Thread 0x7fc6c07f8700 (LWP 20890))]
(gdb) bt
#0  0x00007fc77f4369cb in zend_signal_handler_defer (signo=1, siginfo=0x7fc6c07f7830, context=0x7fc6c07f7700) at /opt/php-7.4.30/Zend/zend_signal.c:96
#1  <signal handler called>
#2  0x00007fc780708f44 in pthread_kill () from /lib/x86_64-linux-gnu/libpthread.so.0
#3  0x0000563c5795bb54 in wakeup_listener ()
#4  0x0000563c5795bb8e in signal_threads ()
#5  0x0000563c5795d9c2 in check_infinite_requests ()
#6  0x0000563c5795e99d in listener_thread ()
#7  0x0000563c579040e9 in thread_start ()
#8  0x00007fc780700ea7 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#9  0x00007fc780630def in clone () from /lib/x86_64-linux-gnu/libc.so.6

gavgavych · 2022-06-16T15:07:38Z

UPD.
PHP (ZTS) 7.2.31 works fine

php --version
PHP 7.2.31 (cli) (built: Dec 16 2021 13:13:52) ( ZTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.31, Copyright (c) 1999-2018, by Zend Technologies
    with SourceGuardian v12.1.1, Copyright (c) 2000-2021, by SourceGuardian Ltd.

gavgavych · 2022-08-11T13:07:36Z

Hi!
Any updates?

Fixes phpGH-8789. Fixes phpGH-10015. This is one small part of the underlying bug for phpGH-10737, as in my attempts to reproduce the issue I constantly hit this crash easily. (The fix for the other underlying issue for that bug will follow soon.) It's possible that a signal arrives at a thread that never handled a PHP request before. This causes the signal globals to dereference a NULL pointer because the TSRM pointers for the thread aren't set up to point to the thread resources yet. PR phpGH-9766 previously fixed this for master by ignoring the signal if the thread didn't handle a PHP request yet. While this fixes the crash bug, I think the solution is suboptimal for 3 reasons: 1) The signal is ignored and a message is printed saying there is a bug. However, this is not a bug at all. For example in Apache, the signal set up happens on child process creation, and the thread resource creation happens lazily when the first request is handled by the thread. Hence, the fact that the thread resources aren't set up yet is not actually buggy behaviour. 2) I believe since it was believed to be buggy behaviour, that fix was only applied to master, so 8.1 & 8.2 keep on crashing. 3) We can do better than ignoring the signal. By just initialising the resources if they don't exist, in the very same way the request handler in Apache works, we can gracefully handle the signal.

Fixes phpGH-8789. Fixes phpGH-10015. This is one small part of the underlying bug for phpGH-10737, as in my attempts to reproduce the issue I constantly hit this crash easily. (The fix for the other underlying issue for that bug will follow soon.) It's possible that a signal arrives at a thread that never handled a PHP request before. This causes the signal globals to dereference a NULL pointer because the TSRM pointers for the thread aren't set up to point to the thread resources yet. PR phpGH-9766 previously fixed this for master by ignoring the signal if the thread didn't handle a PHP request yet. While this fixes the crash bug, I think the solution is suboptimal for 3 reasons: 1) The signal is ignored and a message is printed saying there is a bug. However, this is not a bug at all. For example in Apache, the signal set up happens on child process creation, and the thread resource creation happens lazily when the first request is handled by the thread. Hence, the fact that the thread resources aren't set up yet is not actually buggy behaviour. 2) I believe since it was believed to be buggy behaviour, that fix was only applied to master, so 8.1 & 8.2 keep on crashing. 3) We can do better than ignoring the signal. By just acting in the same way as if the signals aren't active. This means we need to take the same path as if the TSRM had already shut down.

* PHP-8.1: Fix GH-8789 and GH-10015: Fix ZTS zend signal crashes due to NULL globals

* PHP-8.2: Fix GH-8789 and GH-10015: Fix ZTS zend signal crashes due to NULL globals

gavgavych added Bug Status: Needs Triage labels Jun 15, 2022

cmb69 added SAPI: apache2handler Status: Needs Feedback and removed Status: Needs Triage labels Jun 15, 2022

github-actions bot added Status: Needs Triage and removed Status: Needs Feedback labels Jun 15, 2022

dunglas mentioned this issue Sep 30, 2022

segmentation fault when sending SIGINT to PHP embedded in another program #9649

Closed

cmb69 mentioned this issue Nov 30, 2022

zend_signal_handler_defer crashes on apache shutdown #10015

Closed

nielsdos mentioned this issue Mar 15, 2023

Fix ZTS zend signal crashes due to NULL globals #10861

Closed

nielsdos added a commit that referenced this issue Mar 18, 2023

Merge branch 'PHP-8.1' into PHP-8.2

d479b5a

* PHP-8.1: Fix GH-8789 and GH-10015: Fix ZTS zend signal crashes due to NULL globals

nielsdos closed this as completed in 06ae750 Mar 18, 2023

nielsdos added a commit that referenced this issue Mar 18, 2023

Merge branch 'PHP-8.2'

33dd226

* PHP-8.2: Fix GH-8789 and GH-10015: Fix ZTS zend signal crashes due to NULL globals

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PHP 8.0.20 (ZTS) zend_signal_handler_defer crashes on apache #8789

PHP 8.0.20 (ZTS) zend_signal_handler_defer crashes on apache #8789

gavgavych commented Jun 15, 2022 •

edited

Loading

cmb69 commented Jun 15, 2022

gavgavych commented Jun 15, 2022

gavgavych commented Jun 15, 2022 •

edited

Loading

gavgavych commented Jun 16, 2022

gavgavych commented Jun 16, 2022 •

edited

Loading

gavgavych commented Aug 11, 2022

PHP 8.0.20 (ZTS) zend_signal_handler_defer crashes on apache #8789

PHP 8.0.20 (ZTS) zend_signal_handler_defer crashes on apache #8789

Comments

gavgavych commented Jun 15, 2022 • edited Loading

Description

PHP Version

Operating System

cmb69 commented Jun 15, 2022

gavgavych commented Jun 15, 2022

gavgavych commented Jun 15, 2022 • edited Loading

gavgavych commented Jun 16, 2022

gavgavych commented Jun 16, 2022 • edited Loading

gavgavych commented Aug 11, 2022

gavgavych commented Jun 15, 2022 •

edited

Loading

gavgavych commented Jun 15, 2022 •

edited

Loading

gavgavych commented Jun 16, 2022 •

edited

Loading