BackpressureIntegrationTest failing

[dturanski]

I'm running this on macOS Sierra 10.12.6 MacBookPro, 2.5GH Intel Core 17 16GB. It passes if I set madMultipleCutoff = 1. Anything higher will fail at least one of the tests.

[rmichela]

Backpressure detection is tricky, because there is no API event to say it happened. Instead, we have to rely on a statistical method that can be unreliable. Basically, the faster and more powerful your computer is, the less likely backpressure will be triggered.

You can safely ignore this test if everything else is passing.

[dturanski]

Everything else appears to be passing. It looks like early days, but may want to @ignore it in master. Just sayin` I can submit a PR for that if you want

[cbornet]

Wouldn't simulating a long lasting operation on the consumer be a way to ensure backpressure ?

[dturanski]

True but how do you reliably detect BP?

[cbornet]

So the consumers are actually slowed down by thread-sleeping.
@rmichela are you sure the test would fail on fast-hardware ? I would say it should fail on the contrary on slow HW when the producer is not fast-enough compared to this waiting time.

[cbornet]

Maybe a good other way to test would be a quite long consumer sleep, eg. 1s. Then assert that after 2.5s you only get one tick difference with the number of ticks that was detected at 1.5s

[rmichela]

You may be right. We can add longer sleeps to the consumer to force backpressure even on slow machines.

[cbornet]

So the assertions on the number of ticks during 1s will not work since there seems to be some kind of buffering (probably at gRPC level)
Another idea : measure the max time between 2 ticks and assert that it's superior than a given value.

[cbornet]

I have a PoC working : I use a non waiting producer (IntStream) and a non waiting consumer except for one of the early values (eg. 3) where I wait for a long time (eg. 2s). During that time the producer will fill the buffer until it is full (about 128 values) and then will get backpressured. The filling of the buffer takes far less than 1s. So in the assertion I test that the max time between 2 producer onNext is more than 1s.
This way no need for a complex detector, the test should be quite reliable for any hardware and is much faster (less than 3s).

WDYT ?

[dturanski]

Sounds right

[rmichela]

Fantastic!

On Sat, Nov 18, 2017 at 2:10 AM Christophe Bornet ***@***.***> wrote: Closed #13 <#13> via #14 <#14>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKPNU75Sjx34eLNd9tm2qdO37AhzVYDks5s3q0VgaJpZM4QcDCO> .

-- Ryan Michela Principal Member of Technical Staff | Salesforce  <http://smart.salesforce.com/sig/rmichela//us_mb/default/link.html>