stream_select checking of other streams when read buffered stream present

[bukka]

Description

This is a result of analysis of the bug #75584 which points out a case where other supplied streams are not checked for read and write if there is a stream that has got buffered data. It means that only streams with buffered data are reported as changed. It is not necessarily a bug as the all data gets eventually processed but the behavior is certainly not optimal and should be optimized.

The optimization can be done by still checking (using select with 0 timeout) of the streams (fds) for write, read without buffered data and exceptional ones when a stream with buffered data is present. The buffered stream(s) would be removed from the checked set passed to select and automatically added to the returned set. In such way only the subset of descriptors would be checked with immediate return to verify if they are ready.

To give an example, following streams are defined

• $w1 - write stream ready for writing data
• $w2 - write stream not ready for writing data
• $r1 - read stream with buffered data available
• $r2 - read stream without buffered data but data to be read
• $r3 - read stream without buffered data and no data to be read
• $e1 - stream with arrived exceptional data

and stream_select is executed as:

$read = [$r1, $r2, $r3];
$write = [$w1, $w2];
$except = [$e1];
stream_select($read, $write, $except, 1);

It will currently set $read array to [$r1] and $write and $except will be empty even though data is available in them. Only until the full buffer is read other streams would show in the arrays.

The optimization would change the behavior and instead of immediately returning after the setting $read with a buffered stream, it would also check the rest of the streams. It means it would basically do following check:

$read = [$r2, $r3];
$write = [$w1, $w2];
$except = [$e1];
stream_select($read, $write, $except, 0);

The result would be then merged and would be like following in this case:

$read = [$r1, $r2];
$write = [$w1];
$except = [$e1];

It should be noted that this approach might result in more syscalls but that seems like something that users should expect as they expect all the descriptors to be checked when calling stream_select.

[twose]

I don't approve of that, for three reasons:

1. This would increase the complexity of the code.
2. The increase in complexity does not pay off in performance, instead, performance decreases. In the best case, more descriptors may be ready after processing the buffered data. And, in a sense, this is equal to dropping the optimization of "return immediately when streams have buffered data", Originally, it was a very fast syscall-free operation.
3. We just need to ensure eventual consistency, not the result of a single call. I think this is an important design philosophy. The user's expectations are built on the situation that he knows everything, not the black box. And the benefit of following eventual consistency to design an API is that optimizations can be applied internally without scruples in the future.

One more small point, in my experience, code that treats API behavior as a black box tends to be more robust, while code that relies on the internal behavior of the API will always go wrong.

And sorry that I don't have enough energy to research it for the time being, I just noticed this issue, so I may lack in-depth knowledge of the original issue... but I still want to share my points of view, this is just my personal opinion, correct me if I am wrong :)

[bukka]

@twose Thanks for the feedback. There are some good points that we should definitely consider. Especially I see the point for number 2 that has some merits but I'm not sure it outweighs the problems and inconsistencies the current logic has.

I think that the primary problem with the current logic is that if user passes set of fd's, the expectation should be to get consistently answer to most of them which would be inline with how the C select, poll, epoll and others work. I know that they don't give assurance about that but they still answer such question and check all fds consistently. This is mainly to prevent various strange inconsistencies that we currently have.

For example consider this code

<?php
file_put_contents('test.data', 'hello');
$r1 = fopen('test.data', 'r');
$r2 = fopen('test.data', 'r');
$read = [$r1, $r2];
$write = null;
$except = null;

stream_select($read, $write, $rexcept, 0);
var_dump($read);

// fills buffer in $r1
var_dump(fread($r1, 1));

stream_select($read, $write, $rexcept, 0);
var_dump($read);

The result of this is following

array(2) {
  [0]=>
  resource(6) of type (stream)
  [1]=>
  resource(7) of type (stream)
}
string(1) "h"
array(1) {
  [0]=>
  resource(6) of type (stream)
}

The issue is that we haven't read anything from $r2 but suddenly it looks like there are no data to read when $r1 has some data in the buffer. We are talking about level triggered events so this should keep reporting that $r2 still has data to read. What's implemented is just some weird mix of level and edge triggered events. It gets even stranger when reading more data that empty completely buffer which is done in the reported bug linked above.

I don't think this has anything to do with eventual consistency because IMO it is a completely inconsistency... 😄

Do you still think that the currently implementation can be considered as a correct behavior?