Segmentation fault on php_stream_eof

[omerhasan]

Hi,

I seem to be getting the following segmentation fault on version 2.1.3 that I packaged on 2012-02-28 20:32. I've tried using 2.2.1 but received many connection related problems that resulted in dropped connections. I'm not sure if this has been fixed already in issue #183 so would like to know if the following has been already fixed/related to issue #183.

Thank you.

$ sudo gdb /usr/sbin/php-fpm core-php5-fpm.29725
GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
http://bugs.launchpad.net/gdb-linaro/...
Reading symbols from /usr/sbin/php-fpm...Reading symbols from /usr/lib/debug/usr/sbin/php5-fpm...done.
done.

warning: core file may not match specified executable file.
[New LWP 29725]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Core was generated by `php-fpm: pool www '.
Program terminated with signal 11, Segmentation fault.
#0 _php_stream_eof (stream=0x2507d98) at /build/buildd/php5-5.3.6/main/streams/streams.c:642

642 /build/buildd/php5-5.3.6/main/streams/streams.c: No such file or directory.
in /build/buildd/php5-5.3.6/main/streams/streams.c
(gdb) bt
#0 _php_stream_eof (stream=0x2507d98) at /build/buildd/php5-5.3.6/main/streams/streams.c:642
#1 0x00007fd7dd50d77f in redis_check_eof (redis_sock=0x2507d98) at /home/omer/src/phpredis/library.c:34
#2 0x00007fd7dd50edb0 in redis_sock_read_multibulk_reply (ht=0, return_value=0x287c470, return_value_ptr=0x0, this_ptr=0x29f9f88, return_value_used=1,

redis_sock=0x2507d98, z_tab=0x2508d88, ctx=0x0) at /home/omer/src/phpredis/library.c:904

#3 0x00007fd7dd50b56b in redis_sock_read_multibulk_multi_reply_loop (ht=0, return_value=0x287c470, return_value_ptr=0x0, this_ptr=0x29f9f88,

return_value_used=1, redis_sock=0x2507d98, z_tab=0x2508d88, numElems=0) at /home/omer/src/phpredis/redis.c:4862

#4 0x00007fd7dd50b795 in redis_sock_read_multibulk_pipeline_reply (ht=0, return_value=0x287c470, return_value_ptr=0x0, this_ptr=0x29f9f88,

return_value_used=<optimized out>, redis_sock=0x2507d98) at /home/omer/src/phpredis/redis.c:4692

#5 0x00007fd7dd50b934 in zim_Redis_exec (ht=0, return_value=0x287c470, return_value_ptr=0x0, this_ptr=0x29f9f88, return_value_used=1)

at /home/omer/src/phpredis/redis.c:4841

#6 0x000000000070befd in zend_do_fcall_common_helper_SPEC (execute_data=0x2089700) at /build/buildd/php5-5.3.6/Zend/zend_vm_execute.h:316
#7 0x00000000006bcceb in execute (op_array=0x207d0e0) at /build/buildd/php5-5.3.6/Zend/zend_vm_execute.h:107
#8 0x0000000000698340 in zend_execute_scripts (type=0, retval=0x800679a00, file_count=3) at /build/buildd/php5-5.3.6/Zend/zend.c:1266
#9 0x00000000006450e3 in php_execute_script (primary_file=0x0) at /build/buildd/php5-5.3.6/main/main.c:2297
#10 0x000000000042b19a in main (argc=33849712, argv=0xde0160) at /build/buildd/php5-5.3.6/sapi/fpm/fpm/fpm_main.c:1921

[michael-grunder]

Hey,

Do you happen to know the PHP code causes this for you, and how you're connecting to Redis (TCP/unix socket). It looks like it's reading an empty MULTI BULK response, but then again, given the segfault it could be memory corruption somewhere.

Also, are you able to upgrade to the latest version and test. I think the version you're using is a bit old. :)

[omerhasan]

I've upgraded the driver and instead of getting a segfault from php-fpm I'm now receiving the following:

Uncaught exception 'RedisException' with message 'read error on connection'

With the following bit of code:

php
$id = 'some_id';
$r = new Redis();
$r->connect(REDIS_HOST, REDIS_PORT, REDIS_TIMEOUT);

$data = $r->multi(Redis::PIPELINE)
->smembers($id . ":f1")
->lRange($id .":f2", 0, -1)
->lRange($id .":f3", 0, -1)
->hGetAll($id . ":f4")
->exec(); // EXCEPTION OCCURS HERE

Thanks

[michael-grunder]

That message is often encountered with timeouts. What timeout are you using to connect, and what is the value of default_socket_timeout in php.ini?

Locally that code works just fine for me (of course). :)

[omerhasan]

We don't specify any timeout to the driver as the default appears to be 0 or unlimited according to the docs but for the php ini value it is set to:

default_socket_timeout => 60 => 60

We don't seem to get this every time but we seem to get it all at once and then the issue goes away for some amount of varying minutes and comes back. On the redis side we have:

timeout 300

Thanks.

[michael-grunder]

There are a few issues about "read error on connection" here, and it can be difficult to track down. We actually don't get these anymore (or if we do it's weeks in between), but a couple of things to consider.

• Often it occurs under heavy load (so it actually is timing out). If your timeout is 60 seconds, I doubt this is the issue.
• The second place we found issues here is with the systems FD limitations (web server or redis box). If you're running a web request that opens a lot of connections (or hits a certain volume), you run out of descriptors and get this error.

I'm somewhat spitballing here, but those were issues we worked through to make it go away.

[omerhasan]

We've traced this down to bad networking on the Cloud Environment we were using (Rackspace) and seems good now after testing out with Amazon.

Thanks!