Back-pressure

[zeroware]

Hello,

I have a very fast source and a slow consumer.
How can I slow my source to adjust to the speed of subscriber ?

I think this is called back-pressure but not sure if it's doable with Bacon.

[phadej]

Short comment, freely quoting Erik Meijer: don't use observables if you have fast source and slow consumer. Use iterables. (Change from bacon.js' push into something elses pull-based approach).

[zeroware]

@phadej Are you saying that Bacon does not cover my use case by design ?

[phadej]

Can't say for sure, because I don't know your case. But if you want to have something like:

function worker(inputStream) {
  inputStream.onValue(function (x) {
    doSlowStuff(x);
    inputStream.pushNext();
  })
}

then the bacon.js isn't what you want. Compare to iterable approach

function worker(inputGenerator) {
  while (true) {
    val x = inputGenerator();
    doSlowStuff(x);
  }
}

There are stream.skipWhile and similars, but that isn't backpressure, it's about skipping events if you can't keep up. Like in games, you skip frames if you can't keep up - game state evolves on the same rate still.

The back-pressure cannot be generally implemented in bacon.js (or other reactive libraries), consider for example a mouse-click stream. Bacon can't tell user not to click too fast.

[raimohanska]

What @phadej said. On the other hand, depending on your case, you might be fine with .bufferingThrottle that'll set a minimum delay between dispatched events. It'll queue the incoming events in memory though, so it won't scale well in all cases.

[zeroware]

In my case i'm querying a database in a continuous manner to get new informations. Problem is that the source is sending way to much informations for the consumer to keep up. Buffering into memory can be ok but not for long like you said. I think the doc should mention the pull vs push paradigm and what Bacon offers.

[rpominov]

Here an overview of currently avaliable backpressure strategies from RxJS, but It should be also applicable to Bacon https://github.com/Reactive-Extensions/RxJS/blob/master/doc/gettingstarted/backpressure.md

[bleadof]

You can also use highland. They have different strategies for back-pressure.

http://highlandjs.org/