Prevent "interval" overflow in ExtUvLoop #196

PabloKowalczyk · 2019-05-20T07:38:59Z

This PR fixes #194.

WyriHaximus · 2019-05-20T20:08:06Z

@PabloKowalczyk looking into why Travis lost track of the required UV dependencies

PabloKowalczyk · 2019-05-20T20:33:03Z

@WyriHaximus command sudo apt-get install libuv1-dev || true silently failed with message E: Unable to locate package libuv1-dev, maybe this is a reason?

WyriHaximus · 2019-05-20T22:10:47Z

@PabloKowalczyk yeah most likely, but ensure why that is an issue all of the sudden

WyriHaximus · 2019-05-23T06:05:50Z

@PabloKowalczyk just filed #197 to fix this issue.

PabloKowalczyk · 2019-05-23T06:16:06Z

@WyriHaximus good job, after your merge #197 i will rebase my branch and push changes again.

WyriHaximus · 2019-05-23T19:49:05Z

@PabloKowalczyk FYI #197 has been merged

PabloKowalczyk · 2019-05-24T06:50:06Z

Rebase done, but there is other issue:

React\Tests\EventLoop\Timer\ExtUvTimerTest::testAddPeriodicTimerCancelsItself
Failed asserting that 0.004443168640136719 is equal to 0.005 or is greater than 0.005.

IMO it is not caused by my change, i can't reproduce it locally and seems "random" to me. What do you think @WyriHaximus?

WyriHaximus · 2019-05-24T07:09:01Z

@PabloKowalczyk sometimes we have to kick the build to fix that error ;)

PabloKowalczyk · 2019-05-24T07:29:36Z

Yep, now better :)

clue

@PabloKowalczyk Thank you for looking into this and keeping this updated!

I've added a couple of remarks below, what do you think about this? 👍

src/ExtUvLoop.php

clue · 2019-05-28T10:40:41Z

src/ExtUvLoop.php

+        }
+
+        // Subtract 2 because 1 doesn't seems to work and may give wrong results
+        $maxValue = ((int) (\PHP_INT_MAX / 1000)) - 2;


This calculation looks really odd, i.e. it doesn't explain why or what "wrong results" could be. As an alternative, what do you think about something like this:

$result = $interval * 1000; if ($result > PHP_INT_MAX) { // nope } return (int)$result;

Your code will work only if $result will be about ~1025 more than PHP_INT_MAX,
check this example: https://3v4l.org/OH3T7

You're right. PHP will automatically convert an int on overflow to a float, but PHP_INT_MAX + 1 > PHP_INT_MAX === false. If you cast this float value back to int, you'll get an integer overflow and a negative value in return. Here's the gist:

$ms = (int)($interval * 1000); if ($ms < 0 || $interval > \PHP_INT_MAX) { // nope } return $ms;

Look at this code: https://3v4l.org/WcQBR - should it output "Ok"?

Are we talking about the boundary value? I don't really mind tbqh. We're discussing integer overflows that happen for timer intervals of round ~25 days on 32 bit platforms and millions of years on 64 bit platforms, so I don't think it's really relevant if it's +/-1 ms. What I do care about is that the outcome is consistent and reliable.

I'm not suggesting this solution is perfect, I was mostly approaching it from a different perspective because the original implementation contained an apparently arbitrary -2 offset.

Either solution is fine for me as long as it works reliably 👍

I see you point of view, we could move -2-responsibility to user and do not bother with it in implementation - https://3v4l.org/AADvB, but we still need to do (PHP_INT_MAX / 1000) - 2 in tests (\React\Tests\EventLoop\AbstractLoopTest::testTimerIntervalCanBeFarInFuture) - maybe with extensive comment about -2-issue.
On the other hand i'm completely fine with leaving current implementation as is, so it is up to you to choose the best for you/project/users/etc.

The maximum working interval is (int) (\PHP_INT_MAX / 1000) - 1. If the value is higher, you can simply reject it before doing any conversion.

ghost · 2019-06-08T16:49:21Z

src/ExtUvLoop.php

+    private function convertFloatSecondsToMilliseconds($interval, $allowZero = false)
+    {
+        if ($interval < 0) {
+            $interval = 0;


Might as well do a return here instead of going through all these calculations?

It could return early only if i duplicate $allowZero-related logic from last line:

return $result === 0 && !$allowZero ? 1 : $result ;

So new code will look like:

if ($interval < 0) { return $allowZero ? 0 : 1 ; }

IMO it is better to leave it as is.

I'm not sure for what it even is? Libuv allows 0, so differentiating between 0 and 1 is imo unnecessary.

I must admit i don't know, but my guess is addPeriodicTimer with 0 will run code only once, also adding 1 was part of old code: https://github.com/reactphp/event-loop/blob/v1.1.0/src/ExtUvLoop.php#L130 and https://github.com/reactphp/event-loop/blob/v1.1.0/src/ExtUvLoop.php#L153-L154

We only need a non-zero repeat (periodic timer). And then it'd be better to do an inline check there, so it only affects the repeat value and not the timeout value.

Good catch, actually it makes conversion easier. Please check new code.

ghost · 2019-06-09T10:50:23Z

src/ExtUvLoop.php

+        $maxValue = (int) (\PHP_INT_MAX / 1000);
+        $intInterval = (int) $interval;
+
+        if ($intInterval >= $maxValue) {


I've just tested this and unfortunately there's an edge case, which makes the comparison fail, even though the interval is significantly larger (as float) than the PHP integer max value.

Based on https://3v4l.org/TIkG3, the suggestion should work around that. You should probably add test cases for these.

Suggested change

if ($intInterval >= $maxValue) {

if ($interval > \PHP_INT_MAX || ($intInterval <= 0 && $interval > 0)) {

You are right, fixed and tested.

ghost · 2019-07-10T05:27:31Z

@WyriHaximus Can you kick the build again please? :) https://travis-ci.org/reactphp/event-loop/jobs/543631451

WyriHaximus · 2019-07-10T06:04:32Z

@CharlotteDunois done, thanks for the heads up 👍

ghost · 2019-07-10T06:09:31Z

@clue Could you please review the PR again? From my side it looks from a quick glance ok.

clue

Thank you for the update, the changes LGTM! Can you squash this to a reasonable number of commits before we can merge this?

PabloKowalczyk · 2019-07-11T06:48:20Z

@clue should i do the squash? You can also do squash and merge by GitHub UI.

WyriHaximus · 2019-07-26T12:52:18Z

@PabloKowalczyk yes please.

PabloKowalczyk · 2019-07-27T06:21:32Z

Done.

clue · 2019-07-27T08:54:17Z

tests/AbstractLoopTest.php

+        $this->loop->addTimer(-1, function () {});
+
+        $this->assertRunFasterThan(0.002);
+    }


We currently have this odd split in our tests suite, can you move this to the similar tests in AbstractTimerTests?

Other than that, this test looks good to me, but we've seen similar tests fail sporadically due to timer inaccuracies, see also

event-loop/tests/Timer/AbstractTimerTest.php

Lines 28 to 45 in babf91e

public function testAddTimerWillBeInvokedOnceAndBlocksLoopWhenRunning()

{

// Make no strict assumptions about actual time interval. Common

// environments usually provide millisecond accuracy (or better), but

// Travis and other CI systems are slow.

// We try to compensate for this by skipping accurate tests when the

// current platform is known to be inaccurate. We test this by sleeping

// 3x1ms and then measure the time for each iteration before running the

// actual test.

for ($i = 0; $i < 3; ++$i) {

$start = microtime(true);

usleep(1000);

$time = microtime(true) - $start;

if ($time < 0.001 || $time > 0.002) {

$this->markTestSkipped('Platform provides insufficient accuracy (' . $time . ' s)');

}

}

Perhaps you can update this to use a similar logic to make sure an inaccurate platform does not cause this test to fail?

…t to AbstractTimerTest

clue

Changes LGTM, thanks for keeping up with this!

WyriHaximus · 2019-08-12T18:26:23Z

Thanks 👍 !

WyriHaximus requested review from WyriHaximus, clue and jsor May 20, 2019 19:56

PabloKowalczyk force-pushed the interval-overflow-fix branch from 754fae7 to 4e32262 Compare May 24, 2019 06:33

WyriHaximus approved these changes May 24, 2019

View reviewed changes

jsor approved these changes May 24, 2019

View reviewed changes

clue reviewed May 28, 2019

View reviewed changes

ghost reviewed Jun 8, 2019

View reviewed changes

ghost reviewed Jun 9, 2019

View reviewed changes

clue reviewed Jul 10, 2019

View reviewed changes

Prevent "interval" overflow in ExtUvLoop

5d2e219

PabloKowalczyk force-pushed the interval-overflow-fix branch from 730109f to 5d2e219 Compare July 27, 2019 06:20

clue requested changes Jul 27, 2019

View reviewed changes

PabloKowalczyk force-pushed the interval-overflow-fix branch from 7ec5707 to a032124 Compare August 3, 2019 08:01

Move "testTimerIntervalBelowZeroRunsImmediately" from AbstractLoopTes…

a3165da

…t to AbstractTimerTest

PabloKowalczyk force-pushed the interval-overflow-fix branch from a032124 to a3165da Compare August 5, 2019 13:16

clue approved these changes Aug 12, 2019

View reviewed changes

WyriHaximus merged commit 85a0b7c into reactphp:master Aug 12, 2019

PabloKowalczyk deleted the interval-overflow-fix branch August 12, 2019 18:27

clue added this to the v1.1.1 milestone Jan 1, 2020

clue added the bug label Jan 1, 2020

	if ($intInterval >= $maxValue) {
	if ($interval > \PHP_INT_MAX \|\| ($intInterval <= 0 && $interval > 0)) {

	public function testAddTimerWillBeInvokedOnceAndBlocksLoopWhenRunning()
	{
	// Make no strict assumptions about actual time interval. Common
	// environments usually provide millisecond accuracy (or better), but
	// Travis and other CI systems are slow.
	// We try to compensate for this by skipping accurate tests when the
	// current platform is known to be inaccurate. We test this by sleeping
	// 3x1ms and then measure the time for each iteration before running the
	// actual test.
	for ($i = 0; $i < 3; ++$i) {
	$start = microtime(true);
	usleep(1000);
	$time = microtime(true) - $start;

	if ($time < 0.001 \|\| $time > 0.002) {
	$this->markTestSkipped('Platform provides insufficient accuracy (' . $time . ' s)');
	}
	}

Prevent "interval" overflow in ExtUvLoop #196

Prevent "interval" overflow in ExtUvLoop #196

Conversation

PabloKowalczyk commented May 20, 2019

WyriHaximus commented May 20, 2019

PabloKowalczyk commented May 20, 2019

WyriHaximus commented May 20, 2019

WyriHaximus commented May 23, 2019

PabloKowalczyk commented May 23, 2019

WyriHaximus commented May 23, 2019

PabloKowalczyk commented May 24, 2019

WyriHaximus commented May 24, 2019

PabloKowalczyk commented May 24, 2019

clue left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Jul 10, 2019

WyriHaximus commented Jul 10, 2019

ghost commented Jul 10, 2019

clue left a comment

Choose a reason for hiding this comment

PabloKowalczyk commented Jul 11, 2019

WyriHaximus commented Jul 26, 2019

PabloKowalczyk commented Jul 27, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clue left a comment

Choose a reason for hiding this comment

WyriHaximus commented Aug 12, 2019