Troubles with Snoopy and libapache2-mod-php5 on Debian 8

[jibriss]

Checklist before I submit this issue report

I confirm that:

• I have tested this with the latest version of Snoopy available @ https://github.com/a2o/snoopy#i1-version-information-and-download-links (not the latest Snoopy version packaged by your favorite distribution, it might be out of date)
• I have read Snoopy documentation @ https://github.com/a2o/snoopy/blob/master/README.md & https://github.com/a2o/snoopy/tree/master/doc
• I have checked FAQ entries @ https://github.com/a2o/snoopy/blob/master/doc/FAQ.md
• I have checked existing Snoopy issues for duplicates and found none @ https://github.com/a2o/snoopy/issues?q=is%3Aissue

Issue description - Environment basics

(replace 'FILLIN' with actual values from your environment)

Key	Value	Comments
Architecture:	x86_64	x86 is not supported anymore
Linux distribution:	Debian
Distribution version:	8 - Jessie
Snoopy version:	2.4.6
Snoopy config file was used:	yes	snoopy-install.sh enables this by default
Snoopy threading support enabled:	no	not enabled by default as of v2.4.4

Issue description - How was Snoopy installed

Installed with snoopy-install.sh like explained at https://github.com/a2o/snoopy#installation

Issue description - Describe the problem you are having

Hello, I have problems with Snoopy and my PHP installation
Once Snoopy is installed, PHP Apache mod can't load any PHP extension.
When displaying the PHP info page, I see Additional .ini files parsed : (none). I expected to see a list of my extension .ini files

root@web1new-stage-gs:~> snoopy-enable 
SNOOPY: Adding to /etc/ld.so.preload:     /usr/local/lib/libsnoopy.so
SNOOPY: Hint #1: Reboot your machine to load Snoopy system-wide.
SNOOPY: Hint #2: Check your log files for output.
SNOOPY: Enabled.
root@web1new-stage-gs:~> /etc/init.d/apache2 restart
[ ok ] Restarting apache2 (via systemctl): apache2.service.

### No PHP extension anymore

root@web1new-stage-gs:~> /etc/init.d/apache2 restart
[ ok ] Restarting apache2 (via systemctl): apache2.service.
root@web1new-stage-gs:~> snoopy-disable 
SNOOPY: Removing from /etc/ld.so.preload: /usr/local/lib/libsnoopy.so
SNOOPY: Disabled.
SNOOPY: Hint: Your system needs to be restarted to finish Snoopy cleanup.
root@web1new-stage-gs:~> /etc/init.d/apache2 restart
[ ok ] Restarting apache2 (via systemctl): apache2.service.

### PHP extensions are back

It works with libapache2-mod-fastcgi, but not with libapache2-mod-php5

[arlin2050]

+1

[arlin2050]

I have the same problem. Here is the error :

$ strace -ofile -f -o /tmp/apache.log apache2ctl -X -f apache2.conf
munmap(0x7fca0cd7c000, 4096)      = 0
5876  open("./php-apache2handler.ini", O_RDONLY) = -1 ENOENT (No such file or directory)
5876  open("/etc/php5/apache2/php-apache2handler.ini", O_RDONLY) = -1 ENOENT (No such file or directory)
5876  open("./php.ini", O_RDONLY)       = -1 ENOENT (No such file or directory)
5876  open("/etc/php5/apache2/php.ini", O_RDONLY) = 12
5876  ioctl(12, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffe79039780) = -1 ENOTTY (Inappropriate ioctl for device)
5876  fstat(12, {st_mode=S_IFREG|0644, st_size=72664, ...}) = 0
5876  mmap(NULL, 72696, PROT_READ, MAP_PRIVATE, 12, 0) = 0x7fca0cd6b000
5876  fstat(12, {st_mode=S_IFREG|0644, st_size=72664, ...}) = 0
5876  mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fca0cd6a000
5876  lseek(12, 0, SEEK_CUR)            = 0
5876  open("", O_RDONLY)                = -1 ENOENT (No such file or directory)
5876  munmap(0x7fca0cd6b000, 72696)     = 0
5876  close(12)                         = 0
5876  munmap(0x7fca0cd6a000, 4096)      = 0
5876  openat(AT_FDCWD, "/etc/php5/apache2/conf.d", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 12
5876  brk(0x55f6c693c000)               = 0x55f6c693c000
5876  getdents(12, /* 3 entries */, 32768) = 80
5876  getdents(12, /* 0 entries */, 32768) = 0
5876  brk(0x55f6c6934000)               = 0x55f6c6934000
5876  close(12)                         = 0
5876  stat("/etc/php5/apache2/conf.d/curl.ini", {st_mode=S_IFREG|0777, st_size=68, ...}) = 0
5876  open("/etc/php5/apache2/conf.d/curl.ini", O_RDONLY) = 12
5876  ioctl(12, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7ffe79039780) = -1 ENOTTY (Inappropriate ioctl for device)
5876  fstat(12, {st_mode=S_IFREG|0777, st_size=68, ...}) = 0
5876  mmap(NULL, 100, PROT_READ, MAP_PRIVATE, 12, 0) = 0x7fca0cd7c000
5876  fstat(12, {st_mode=S_IFREG|0777, st_size=68, ...}) = 0
5876  mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fca0cd7b000
5876  lseek(12, 0, SEEK_CUR)            = 0
5876  open("", O_RDONLY)                = -1 ENOENT (No such file or directory)

I use snoopy 2.4.6, php 5.6.30 with apache mod on debian jessie 8.9 x86_64.
Thanks,

[mikaelmasson]

+1
I've got the same problem on 4 servers (Debian 8/Jessie + PHP 5.6 + Snoopy 2.4.6 + Apache 2.4).

Thanks,

Mike

[bostjan]

@jibriss @arlin2050 @mikaelmasson do you have any chroot set up for your appache?

Can you produce a LXC (or Docker) container creation instructions to reproduce this? I am using Apache+PHP all the time and I've had zero problems like the ones you are describing.

[bostjan]

Since Debian 8 (Jessie) is out of the picture at this point, I tried and I was able to reproduce this on Debian 9 (Stretch).

After using strace (with strace -tt -f -v -o apache2-trace.out apache2 -k start -X command, to run a single process only so simplify things) to get a basic idea about what Apache is doing, I've found a slight difference between no-Snoopy and with-Snoopy runs.

No Snoopy:

24912 22:59:26.960410 stat("/etc/php/7.0/apache2/conf.d/10-opcache.ini", {st_dev=makedev(8, 1), st_ino=682473, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=79, st_atime=2020-11-24T22:22:43+0000.973222886, st_mtime=2020-10-06T17:08:28+0000, st_ctime=2020-11-24T22:22:43+0000.841222590}) = 0
24912 22:59:26.960577 open("/etc/php/7.0/apache2/conf.d/10-opcache.ini", O_RDONLY) = 9
24912 22:59:26.960691 ioctl(9, TCGETS, 0x7fff6281c370) = -1 ENOTTY (Inappropriate ioctl for device)
24912 22:59:26.960804 fstat(9, {st_dev=makedev(8, 1), st_ino=682473, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=79, st_atime=2020-11-24T22:22:43+0000.973222886, st_mtime=2020-10-06T17:08:28+0000, st_ctime=2020-11-24T22:22:43+0000.841222590}) = 0
24912 22:59:26.960955 mmap(NULL, 111, PROT_READ, MAP_PRIVATE, 9, 0) = 0x7f3feb194000
24912 22:59:26.961089 lseek(9, 0, SEEK_CUR) = 0
24912 22:59:26.961202 munmap(0x7f3feb194000, 111) = 0
24912 22:59:26.961321 close(9)          = 0
# ...(and the pattern repeats for all other .ini files)
#
# ...(then, right after all .ini files are parsed, PHP proceeds straight to loading extensions)
24912 22:59:26.977413 open("/usr/lib/php/20151012/opcache.so", O_RDONLY|O_CLOEXEC) = 9
24912 22:59:26.977520 read(9, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0Pq\0\0\0\0\0\0"..., 832) = 832
24912 22:59:26.977600 fstat(9, {st_dev=makedev(8, 1), st_ino=1943233, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=392, st_size=199448, st_atime=2020-11-24T22:22:51+0000.121238890, st_mtime=2020-10-06T17:08:28+0000, st_ctime=2020-11-24T22:22:32+0000.785197854}) = 0
24912 22:59:26.977723 mmap(NULL, 2332344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 9, 0) = 0x7f3fe222b000
24912 22:59:26.977805 mprotect(0x7f3fe225a000, 2093056, PROT_NONE) = 0
24912 22:59:26.977881 mmap(0x7f3fe2459000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 9, 0x2e000) = 0x7f3fe2459000
24912 22:59:26.977966 mmap(0x7f3fe245c000, 34488, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3fe245c000
24912 22:59:26.978105 close(9)          = 0
24912 22:59:26.978402 mprotect(0x7f3fe2459000, 8192, PROT_READ) = 0
# ... (and the pattern repeats for all extensions)

With Snoopy:

24918 22:59:52.967402 close(9)          = 0
24918 22:59:52.967455 stat("/etc/php/7.0/apache2/conf.d/10-opcache.ini", {st_dev=makedev(8, 1), st_ino=682473, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=79, st_atime=2020-11-24T22:22:43+0000.973222886, st_mtime=2020-10-06T17:08:28+0000, st_ctime=2020-11-24T22:22:43+0000.841222590}) = 0
24918 22:59:52.967530 open("/etc/php/7.0/apache2/conf.d/10-opcache.ini", O_RDONLY) = 9
24918 22:59:52.967579 ioctl(9, TCGETS, 0x7ffce6dfc160) = -1 ENOTTY (Inappropriate ioctl for device)
24918 22:59:52.967626 fstat(9, {st_dev=makedev(8, 1), st_ino=682473, st_mode=S_IFREG|0644, st_nlink=1, st_uid=0, st_gid=0, st_blksize=4096, st_blocks=8, st_size=79, st_atime=2020-11-24T22:22:43+0000.973222886, st_mtime=2020-10-06T17:08:28+0000, st_ctime=2020-11-24T22:22:43+0000.841222590}) = 0
24918 22:59:52.967690 mmap(NULL, 111, PROT_READ, MAP_PRIVATE, 9, 0) = 0x7ff44c7c7000
24918 22:59:52.967740 lseek(9, 0, SEEK_CUR) = 0
24918 22:59:52.967782 open("\4", O_RDONLY) = -1 ENOENT (No such file or directory)
24918 22:59:52.967833 munmap(0x7ff44c7c7000, 111) = 0
24918 22:59:52.967878 close(9)          = 0
# ...(and the pattern repeats for all other .ini files)
# ...(but no .so files are loaded afterwards)

The difference during the .ini parsing phase is(are) that added bogus open() call(s), one for each extension:

open("\4", O_RDONLY)

At the moment, I only have a vague theory why this might be happening:

• Snoopy's native code doesn't get executed at all as long as the running program doesn't call the execv()/execve() functions.
• All Snoopy's symbols (variable names, function names) are prefixed with a snoopy_ prefix, and PHP does not have the word snoopy nowhere in its sources.
• However, the ini parsing library is imported into this repository as-is, and thus embedded in the resulting Snoopy's libsnoopy.so as-is too, including all symbols. This could somehow be messing with how PHP (when running inside Apache) calls own ini parsing facilities. If that's indeed the case, I am actually surprised we're not seeing hard crashes.

[bostjan]

@jibriss, @arlin2050, @mikaelmasson, the hypothesis above is confirmed.

Temporary workaround:

Building Snoopy with --disable-config-file fixes avoids the issue.

[bostjan]

@jibriss, @arlin2050 or @mikaelmasson, if any of you has a few spare cycles to test out the changes from PR #168 it's already merged in master now, if they indeed resolve the issue on your side, let me know how it goes.

Branch to use the code from:
https://github.com/a2o/snoopy/tree/master (at commit 2b0b97c or later).