Add throttleLatest to Observable #1396

Wosin · 2021-03-23T00:18:06Z

This closes #1385.

This PR introduces throttleLatest as defined in here. PR introduces some tests to verify the correctness of the behaviour implemented.

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala

oleg-py · 2021-03-23T09:30:52Z

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala

+            out.onNext(lastEvent).syncTryFlatten.syncOnContinue {
+              isDone = true
+              out.onComplete()
+              task.cancel()
+            }


I wonder why hasValue and shouldEmitNext are @volatile?

This seems to be the only place where something can be written outside of self.synchronized block (if Ack fails to be synchronous) and it might theoretically be possible that the last event gets emitted twice if that block runs in parallel with the one in def run since the isDone = true here might not be propagated to the thread doing def run. Otherwise, JSR-133 should guarantee memory barriers when entering and exiting synchronized blocks.

The simple fix I can think of is to just do

if (!isDone) { isDone = true ... rest of the branching code ... }

to both tell the def run to stop itself and keep the write in synchronized block and not in a Future continuation so everything can see it.

I'm going to tag @alexandru so he can tell me that I'm wrong there.

Ahh! I think it doesn't anymore, I've added that when I was thinking about synchronization. But I will verify this.

I think you're right @oleg-py and I think we could even do:

if (!isDone) { isDone = true task.cancel() ... rest of the branching code ... }

oleg-py · 2021-03-23T09:38:56Z

...x-reactive/shared/src/test/scala/monix/reactive/internal/operators/ThrottleLatestSuite.scala

+      val o = Observable
+        .intervalAtFixedRate(2.second)
+        .take(sourceCount.toLong)
+        .throttleLatest(1.second, true)


If the rate of output is at 2 seconds and you're throttling by 1 second after first element (as per RxJava doc you linked), isn't that a noop?

Yeah, probably true. I will think about something, but as far as I know we can't control the exact elements that are going to be throttled in this test, same as in SampleOnceSuite.

You should be able to do something like

Observable(1).delayOnNext(200.millis) ++ Observable(2).delayOnNext(100.millis) ++ ...

to create a predictable chain.

I'm also concerned about Sample two lines below, as that is literally saying it's not dropping anything from the input (2nd parameter of sample is expected output size). The purpose of throttling is to drop some so we should check this if possible. I think ThrottleFirstSuite has an example where it drops half.

You should be able to do something like

Observable(1).delayOnNext(200.millis) ++ Observable(2).delayOnNext(100.millis) ++ ...

to create a predictable chain.

Not sure what's the difference between that and just doing the intervalAtFixedRate to be honest :)

I think something like that can be useful if we would like to have a specific test case in a test outside BaseOperatorSuite, e.g. ReactiveX example

oleg-py

There are few suspicious spots. I'd also like to see test that shows exactly which elements have been retained when using a producer that's significantly more frequent than the throttle interval.

oleg-py · 2021-03-23T09:40:23Z

...x-reactive/shared/src/test/scala/monix/reactive/internal/operators/ThrottleLatestSuite.scala

+  def createObservable(sourceCount: Int) = Some {
+    if (sourceCount == 1) {
+      val o = Observable.now(100L).delayExecution(500.millis).throttleLatest(1.second, true)
+      Sample(o, 1, 100, 500.millis, 1.second)


Please test with something with more than one element in the output :D That might prove that emitLast handling is correct but does very little to actually prove that throttling is working as expected.

I don't think we are tsting with one element on the output here :) I think createObservable gets called from outside and we pass the sourceCount fromBaseOperatorSuite, so that is randomized.
I've added this sourceCount ==1 check because I've noticed that in several suites, but now that I've started chcecking I think it's not needed :)

Right. Disregard that line comment but please adjust your additional tests to have more than 1-2 elements :)

Sure, sounds reasonable I will add few more tests.

I think we could still test cases from SampleOnceSuite

Sure, gonna take a look at them.

Avasil

Thanks a lot for the PR @Wosin and sorry for the delay with review

Avasil · 2021-04-11T10:10:25Z

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala

+            out.onNext(lastEvent).syncTryFlatten.syncOnContinue {
+              isDone = true
+              out.onComplete()
+              task.cancel()
+            }


I think you're right @oleg-py and I think we could even do:

if (!isDone) { isDone = true task.cancel() ... rest of the branching code ... }

Avasil · 2021-04-11T10:12:52Z

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala

+      private[this] var lastEvent: A = _
+      @volatile private[this] var hasValue = false
+      @volatile private[this] var shouldEmitNext = true
+      private[this] var lastAck: Future[Ack] = _


seems to be unused

Yeah, will remove that.

Avasil · 2021-04-11T10:15:23Z

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala

+      override def onComplete(): Unit = self.synchronized {
+        if (!isDone) {
+          if (emitLast && hasValue) {
+            out.onNext(lastEvent).syncTryFlatten.syncOnContinue {


Are we guaranteed that the last onNext has returned?

It seems like shouldEmitNext handles back-pressure for onNext but I'm not so sure for onComplete

Avasil · 2021-04-11T10:20:27Z

...x-reactive/shared/src/test/scala/monix/reactive/internal/operators/ThrottleLatestSuite.scala

+  def createObservable(sourceCount: Int) = Some {
+    if (sourceCount == 1) {
+      val o = Observable.now(100L).delayExecution(500.millis).throttleLatest(1.second, true)
+      Sample(o, 1, 100, 500.millis, 1.second)


I think we could still test cases from SampleOnceSuite

Avasil · 2021-04-11T10:22:57Z

...x-reactive/shared/src/test/scala/monix/reactive/internal/operators/ThrottleLatestSuite.scala

+      val o = Observable
+        .intervalAtFixedRate(2.second)
+        .take(sourceCount.toLong)
+        .throttleLatest(1.second, true)


I think something like that can be useful if we would like to have a specific test case in a test outside BaseOperatorSuite, e.g. ReactiveX example

paualarco · 2021-04-18T22:34:17Z

monix-reactive/shared/src/main/scala/monix/reactive/Observable.scala

+    */
+  final def throttleLatest(period: FiniteDuration, emitLast: Boolean): Observable[A] =
+    new ThrottleLatestObservable[A](self, period, emitLast)
  /** Emit the most recent items emitted by the source within


@Wosin shouldn't be a line break between the two methods? :P

Which methods do You mean ?:P

I mean line 3375 and 3376, the end of throttleLatest and the scaladoc for sample.

Oh, yeah You are right ofc. I thought scalafmt should get that.

Avasil

Thanks, LGTM 👍

oleg-py

Yeah, looks good 👍

DomWos added 4 commits March 23, 2021 00:58

Initial implementation of ThrottleLatestObservable

17f5a45

Add throttleLatest to Observable

994e883

Fix formatting.

d9a5d3d

Fix doc.

dd7416a

oleg-py reviewed Mar 23, 2021

View reviewed changes

...ctive/shared/src/main/scala/monix/reactive/internal/operators/ThrottleLatestObservable.scala Show resolved Hide resolved

oleg-py reviewed Mar 23, 2021

View reviewed changes

oleg-py requested changes Mar 23, 2021

View reviewed changes

Avasil reviewed Apr 11, 2021

View reviewed changes

DomWos and others added 4 commits April 13, 2021 14:33

Fix tests.

3e947e9

Added additional tests.

ee7c55e

Revert accidental changes.

98793fb

Fix formatting.

9df5be5

Wosin requested review from Avasil and oleg-py April 15, 2021 20:59

paualarco reviewed Apr 18, 2021

View reviewed changes

Avasil approved these changes Apr 24, 2021

View reviewed changes

oleg-py approved these changes Apr 24, 2021

View reviewed changes

Avasil merged commit 02ee9b9 into monix:series/3.x Apr 24, 2021

Uh oh!

Add throttleLatest to Observable #1396

Add throttleLatest to Observable #1396

Uh oh!

Conversation

Wosin commented Mar 23, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wosin Mar 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Wosin Mar 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oleg-py left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Avasil left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Avasil left a comment

Choose a reason for hiding this comment

Uh oh!

oleg-py left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Wosin commented Mar 23, 2021 •

edited

Loading

Wosin Mar 28, 2021 •

edited

Loading

Wosin Mar 28, 2021 •

edited

Loading