New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pcntl_fork causing “errno=32 Broken pipe” #474

Closed
penfold45 opened this Issue May 17, 2014 · 45 comments

Comments

Projects
None yet
@penfold45
Copy link

penfold45 commented May 17, 2014

Hi

I am having issues with PHP notices after upgrading from php 5.4.26 to 5.4.28. I original thought it was a php issue but it may be related to this Redis extension.

The full details can be found here http://stackoverflow.com/questions/23713480/after-php-upgrade-pcntl-fork-causing-errno-32-broken-pipe

But basically when I am forking, whenever a child closes, except for the first child, I get an error saying:

Notice: Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0

It feels like the first child is closing the connection to Redis and then the following children try and close the same connection.

It appears that this has only just started being an issue since either PHP 5.4.27 or 5.4.28

Could this be a phpredis issue?

I have Redis Version => 2.2.5 installed

@michael-grunder

This comment has been minimized.

Copy link
Member

michael-grunder commented May 19, 2014

Hey there,

While I don't know specifically why, I have a strong suspicion that the Zend library might get a little quirky around sockets within the context of a forking. That being said, I'll see if I can replicate your issue based on the stack overflow question you mentioned.

Cheers dude,
Mike

@penfold45

This comment has been minimized.

Copy link

penfold45 commented May 19, 2014

Brilliant thanks. For the time being I have just disabled php notice errors for the scripts that fork. Makes my log files a little quieter ;)

@penfold45

This comment has been minimized.

Copy link

penfold45 commented Jun 5, 2014

Hi. I was woundering if you have had chance to look into this yet?

@penfold45

This comment has been minimized.

Copy link

penfold45 commented Sep 30, 2014

Bump*

@jessem

This comment has been minimized.

Copy link

jessem commented Oct 29, 2014

+1 Also seeing this issue with workers that fork in PHPResque https://github.com/chrisboulton/php-resque

@splitice

This comment has been minimized.

Copy link

splitice commented Nov 16, 2014

+1 also seeing this in php resque workers.

Appears harmless... guessing its "QUIT\r\n"

dominics pushed a commit to vend/php-resque that referenced this issue Nov 18, 2014

Dominic Scheirlinck
@jat001

This comment has been minimized.

Copy link

jat001 commented Jan 12, 2015

I know how to fix this bug. Just close the connect to redis before pcntl_fork and re-connect to redis after pcntl_fork.

@anton000

This comment has been minimized.

Copy link

anton000 commented Jan 21, 2015

no updates?

@roeyb

This comment has been minimized.

Copy link

roeyb commented Feb 3, 2015

+1

3 similar comments
@diandianxiyu

This comment has been minimized.

Copy link

diandianxiyu commented Feb 11, 2015

+1

@rlweb

This comment has been minimized.

Copy link

rlweb commented Feb 20, 2015

+1

@schlaus

This comment has been minimized.

Copy link

schlaus commented Mar 20, 2015

+1

@lexaurin

This comment has been minimized.

Copy link

lexaurin commented Mar 26, 2015

Same here. PHP 5.5.9, phpredis 2.2.7

Simple script to replicate this bug https://gist.github.com/lexaurin/8897f1be48b9e8747b7d/e8a5fcd750623b6fc250f9c529fc2e65cc55893a

Expected result:

bool(true)

Actual result:

bool(true)
PHP Notice:  Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0

Notice: Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0

PHP Notice:  Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0

Notice: Unknown: send of 6 bytes failed with errno=32 Broken pipe in Unknown on line 0

Connections before script (lsof -i :6379):

COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 20319 root    4u  IPv4 10422993      0t0  TCP *:6379 (LISTEN)

Connections after first child forked:

COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 20319 root    4u  IPv4 10422993      0t0  TCP *:6379 (LISTEN)
redis-ser 20319 root    6u  IPv4 10446216      0t0  TCP config0.local:6379->config0.local:52050 (ESTABLISHED)
php       21993  lex    8u  IPv4 10446860      0t0  TCP config0.local:52050->config0.local:6379 (ESTABLISHED)
php       21994  lex    8u  IPv4 10446860      0t0  TCP config0.local:52050->config0.local:6379 (ESTABLISHED)

Connections after second child forked:

COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 20319 root    4u  IPv4 10422993      0t0  TCP *:6379 (LISTEN)
php       21993  lex    8u  IPv4 10446860      0t0  TCP config0.local:52050->config0.local:6379 (CLOSE_WAIT)
php       22020  lex    8u  IPv4 10446860      0t0  TCP config0.local:52050->config0.local:6379 (CLOSE_WAIT)

Connections after third child forked:

COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
redis-ser 20319 root    4u  IPv4 10422993      0t0  TCP *:6379 (LISTEN)

Note: Fact that child closes "both" connections at the end is right (since it shares file descriptor with parent).

@lexaurin

This comment has been minimized.

Copy link

lexaurin commented Mar 26, 2015

Modified my gist a bit to see which process produces notice https://gist.github.com/lexaurin/8897f1be48b9e8747b7d/d46ccd5c2ffabb1c3a09118db80654cb382c6c96
and it's interesting - third child at the end and following children and then parent at the end.

My log:

bool(true)
Parent pid: 22968
Child pid: 22969
Child pid: 22971
Child pid: 22972
Error in PID 22972: Unknown: send of 6 bytes failed with errno=32 Broken pipe
Child pid: 22973
Error in PID 22973: Unknown: send of 6 bytes failed with errno=32 Broken pipe
Child pid: 22974
Error in PID 22974: Unknown: send of 6 bytes failed with errno=32 Broken pipe
Error in PID 22968: Unknown: send of 6 bytes failed with errno=32 Broken pipe 
@sanvibyfish

This comment has been minimized.

Copy link

sanvibyfish commented May 29, 2015

I have same problem, I use pconnect to connect redis.

After fork, I pconnect again, But nothing help.

Sometime always throw exception like this.

Exception Redis::lLen(): send of 38 bytes failed with errno=32 Broken pipe in

This code just manager jobs, and run it.

        $files = glob(__DIR__ . '/jobs/*.php');
        foreach ($files as $_file) {
          $this->workers[$name . $i] = pcntl_fork();
          $redis = new Redis();
          $redis->pconnect('127.0.0.1', 0.3);
          if ($this->workers[$name . $i] === 0) {
            posix_kill(posix_getpid(), SIGTERM);
          }
        }
        while (count($this->workers)) {
            $pid = pcntl_wait($status);
            usleep(50000);
        }

Then the unix crontab will run it. Same job will pconnect Redis, after pcntl_fork() pconnect Redis can't fix

@xuecai

This comment has been minimized.

Copy link

xuecai commented Sep 9, 2015

use pcntl_fork causing "errno=32 Broken pipe"
phpredis 2.2.7

maybe regiest global socket in PHP_RINIT_FUNCTION and close socket in PHP_RSHUTDOWN_FUNCTION can fix this question?

@lixiphp

This comment has been minimized.

Copy link

lixiphp commented Oct 19, 2015

+1

@driq

This comment has been minimized.

Copy link

driq commented Nov 2, 2015

Having the same issue when using https://github.com/kriswallsmith/spork

@lixiphp

This comment has been minimized.

Copy link

lixiphp commented Nov 3, 2015

this bug is from php pcntl, just in the worker colse redis, will solve this problem.

@giraclion

This comment has been minimized.

Copy link

giraclion commented Feb 3, 2016

Same issue happens when using PHP 5.5.9 + phpredis 2.2.7 + redis engine 2.8.19 + Magento CE 1.9.0.1 + Cm_Cache_Backend_Redis environment.
Error:

Redis::exec(): send of 8192 bytes failed with errno=32 Broken pipe
@jaymecd

This comment has been minimized.

Copy link

jaymecd commented Mar 14, 2016

+1

3 similar comments
@ricardosierra

This comment has been minimized.

Copy link

ricardosierra commented Mar 15, 2016

+1

@piotrkochan

This comment has been minimized.

Copy link

piotrkochan commented Apr 5, 2016

+1

@jimyaghi

This comment has been minimized.

Copy link

jimyaghi commented Jun 8, 2016

+1

@sapzxc

This comment has been minimized.

Copy link

sapzxc commented Jul 9, 2016

+1 :)

@vanquang9387

This comment has been minimized.

Copy link

vanquang9387 commented Aug 2, 2016

+1

2 similar comments
@consatan

This comment has been minimized.

Copy link

consatan commented Aug 31, 2016

+1

@damonto

This comment has been minimized.

Copy link

damonto commented Sep 19, 2016

+1

@robbieaverill

This comment has been minimized.

Copy link

robbieaverill commented Oct 6, 2016

You guys know there's now a "+1" equivalent "like button" on GitHub right? Without spamming the participants with emails (unless that's your objective!)

@j-d

This comment has been minimized.

Copy link

j-d commented Nov 2, 2016

+1

@7c

This comment has been minimized.

Copy link

7c commented Nov 18, 2016

php 5.5.9 suddenly started doing same behaviour. I cannot use it anymore in production. This is high priority bug

@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Nov 18, 2016

Could someone provide script which cause this error?

@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Nov 23, 2016

@lexaurin, thanks.
There is no error on my gentoo box

php redis-pipe-bug.php 
bool(true)
Parent pid: 27921
Child pid: 27922
Child pid: 27923
Child pid: 27924
Child pid: 27925
Child pid: 27926

Tested on php5.3-php7.1

@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Nov 23, 2016

@lexaurin depends environment?

@lexaurin

This comment has been minimized.

Copy link

lexaurin commented Nov 23, 2016

I can't reproduce it on my desktop too (PHP 7.0.8-0ubuntu0.16.04.3)

Problem was on Debian machine a while ago and I don't have access there anymore. PHP 5.5.9, phpredis 2.2.7 at that time.

Maybe someone with "+1" can provide more infomration and test it out.

@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Nov 23, 2016

phpredis 2.2.7 is to old. problem may be fixed long time ago :)

@lexaurin

This comment has been minimized.

Copy link

lexaurin commented Nov 23, 2016

I know, it was reported in 2014. If no one can confirm it on newer versions of phpredis I suggest to close this bug with resolution "Upgrade to 3.x"

@ppanphper

This comment has been minimized.

Copy link

ppanphper commented Jan 4, 2017

Env: PHP7.1.0+phpredis-3.1.0
use RedisCluster connect redis cluster server.
when the high amount of concurrency will appear when the following error:
[2017-01-04 03:18:24.999684] [Notice] RedisCluster::set(): send of 3182 bytes failed with errno=32 Broken pipe at /home/development/pandy/Swoole_Pool/RedisPool/RedisClient.php[512]
[2017-01-04 03:18:24.999892] [Notice] RedisCluster::set(): send of 3182 bytes failed with errno=32 Broken pipe at /home/development/pandy/Swoole_Pool/RedisPool/RedisClient.php[512]

@zoglun

This comment has been minimized.

Copy link

zoglun commented Mar 23, 2017

Same error happen to me under debian8, redis 3.2.7, with php7.0 & Mediawiki 1.28.
PHP Notice: Redis::set(): send of 82 bytes failed with errno=32 Broken pipe in /wiki/includes/libs/redis/RedisConnRef.php on line 96

@JPNestNgocTruong

This comment has been minimized.

Copy link

JPNestNgocTruong commented Mar 28, 2017

Same error happens to me under CentOS 6 , Redis 3.2.1 with PHP 7.0 & Magento 2

[2017-03-25 00:00:38] Subscription Logger Publish profile to message queue.CRITICAL: ErrorException: Redis::hGet(): send of 71 bytes failed with errno=32 Broken pipe in /opt/nestle/magento/releases/20170324062419/vendor/colinmollenhour/credis/Client.php:1044
Stack trace:
#0 [internal function]: Composer\Util\ErrorHandler::handle(8, 'Redis::hGet(): ...', '/opt/nestle/mag...', 1044, Array)
#1 /opt/nestle/magento/releases/20170324062419/vendor/colinmollenhour/credis/Client.php(1044): Redis->hGet('zc:k:b22_GLOBAL...', 'd')
#2 /opt/nestle/magento/releases/20170324062419/lib/internal/Cm/Cache/Backend/Redis.php(211): Credis_Client->__call('hget', Array)
#3 /opt/nestle/magento/releases/20170324062419/vendor/magento/zendframework1/library/Zend/Cache/Core.php(306): Cm_Cache_Backend_Redis->load('b22_GLOBAL_PRIM...', false)
@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Mar 28, 2017

@ppanphper, @zoglun, @ngoctruongbc, do you have a script to reproduce this issue?

@virgofx

This comment has been minimized.

Copy link

virgofx commented Mar 28, 2017

In my opinion this should be closed as it's a duplicate of #70 -- the broken pipe is the effective result that leads to all sorts of read/write errors.

@yatsukhnenko Script to reproduce is here: #70 (comment)

@yatsukhnenko

This comment has been minimized.

Copy link
Member

yatsukhnenko commented Mar 28, 2017

Duplicate #70

@weixuan19930

This comment has been minimized.

Copy link

weixuan19930 commented Sep 18, 2018

Same error happens in swoole worker...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment