Socket 'EPIPE' broken pipe handling #243
Comments
|
Can you share some more details such as: operating system + version, PHP version, libevent version, libevent ext version, minimal testing script? |
|
OS is Ubuntu Server 12.04 I have installed libevent ext using: I dont have a minimal testing script at the time, I will look around to see how to emulate a broken pipe error. |
|
I wasnt able to reproduce it, except once while actually yanking out my network connection. I am not sure what causes the problem, my setup or the libevent loop used. But the strace output suggests that broken streams remain in the event loop for some reason. However since I made two minor changes in Connection.php and Buffer.php I haven't ran into the problems on my production server. I know you probably cant do much without reproducing it. But if the issue pops up again here is what I changed: Buffer.php (This solved the recurring epoll_wait on a EPIPE socket) public function handleWrite()
{
if (!is_resource($this->stream) || ('generic_socket' === $this->meta['stream_type'] && feof($this->stream))) {
$this->emit('error', array(new \RuntimeException('Tried to write to closed or invalid stream.')));
return;
}
set_error_handler(array($this, 'errorHandler'));
$sent = fwrite($this->stream, $this->data);
restore_error_handler();
if (false === $sent || 0 === $sent) { // This will break the connection if socket's OS send buffer is full, retry count would be more fancy
$this->emit('error', array(new \ErrorException(
$this->lastError['message'],
0,
$this->lastError['number'],
$this->lastError['file'],
$this->lastError['line']
)));
return;
}
$len = strlen($this->data);
if ($len >= $this->softLimit && $len - $sent < $this->softLimit) {
$this->emit('drain');
}
$this->data = (string) substr($this->data, $sent);
if (0 === strlen($this->data)) {
$this->loop->removeWriteStream($this->stream);
$this->listening = false;
$this->emit('full-drain');
}
}Later I noticed I had the same problem with sockets that I never write too, only read. The strace showed high-pace repeated 0 bytes recv from the same socket raising CPU usage as well. Since there the epoll returns the socket, there should either be bytes read or the socket has been closed. Therefore I changed the Connection.php too: Connection.php public function handleData($stream)
{
if (!is_resource($stream) || feof($stream)) {
$this->end();
return;
}
$data = fread($stream, $this->bufferSize);
if ('' === $data || false === $data) {
$this->end();
} else {
$this->onData($data);
}
}Above can be simplified, I left it as is since it worked for me. I have fread in it, I know there is some discussion whether to use stream_socket_recv or this, I guess that would work just as well. |
|
Seems this commit fixed the issue as well: (the addition of feof helps, the Buffer modification is not necessary) 424e873#diff-ff2311b1d39f83cb5ca5c77f10391d2c Will close this issue myself. Edit: seems that I used a way old version. Sorry for all the spamming. Thanks again for this great component! |
First of all, thanks for bringing cool stuff like this to php!
After a while my event loop takes 100% cpu, however it remains functional. I tried to debug the reason. I am not 100% sure but I think I have a plausible theory of what is causing it.
The Buffer class does not (and cannot) detect broken pipes when writing using fwrite. React keeps it in the loop's 'write' list indefinitely. Causing high cpu usage on my machine.
Here is an strace of the php process with high cpu. The event loop (libevent in my case) is still functioning, but it keeps running into the broken pipe.
I think I ran into the following php 'bug':
http://www.php.net/manual/en/function.fwrite.php#96951
https://bugs.php.net/bug.php?id=39598
The post suggest that you cannot depend on the return value of fwrite to detect errors. The first link suggests testing whether the value is 0. Although this not a perfect solution, since 0 is also returned when the socket's internal send buffer is full.
Perhaps a fix could be that the buffer emits an error out after a few consecutive 0-byte writes?
The text was updated successfully, but these errors were encountered: