New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Typelevel membership for monifu #4

Closed
milessabin opened this Issue Dec 18, 2015 · 33 comments

Comments

7 participants
@milessabin
Member

milessabin commented Dec 18, 2015

  • Monifu is a high-performance Scala / Scala.js library for composing asynchronous and event-based programs using observable sequences
  • Monifu supports the Typelevel code of conduct.

See https://github.com/alexandru/monifu.

@milessabin

This comment has been minimized.

Member

milessabin commented Dec 18, 2015

👍

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

Big 👍

@tpolecat

This comment has been minimized.

Member

tpolecat commented Dec 18, 2015

I'm concerned that the central abstraction Observable is so fundamentally side-effecting. Reactive streams is not a functional spec. Does anyone else care?

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

Does anyone else care?

No, not with respect to monifu joining typelevel - the fact that it does non-blocking async streaming on JVM/JS is something that I believe we don't currently have.

Sure, if there is a better way that we find later, we all benefit.

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

@tpolecat

This comment has been minimized.

Member

tpolecat commented Dec 18, 2015

I don't see anything about side-effects in that discussion. Maybe that's not what you were referring to.

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

I was referring to the general discussion on Observables

@tpolecat

This comment has been minimized.

Member

tpolecat commented Dec 18, 2015

Just be clear, when I asked "does anyone else care?" it may have sounded snarky but it was intended a sincere question. I'm interested in the general sense of the org with relation to controlled effects.

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 18, 2015

@tpolecat let me answer some of those concerns.

For one it is true that the reactive-streams specification is side-effecting, however it is a very pragmatic one, being needed for interoperability between libraries (a very nice thing to have, think of Play Framework) and also being very efficient when communicating over a network. And Monifu is not using the Reactive Streams protocol internally, it only integrates with it, Monifu's internal protocol being more user friendly.

At it's core, Monifu communication protocol is defined primarily by these interfaces:

// the producer
trait Observable[+T] {
  def subscribe(s: Subscriber[T]): Cancelable
}

// the consumer
trait Observer[-T] {
  def onNext(e: T): Future[Ack]
  def onError(ex: Throwable): Unit
  def onComplete(): Unit
}

// the consumer + an execution context
trait Subscriber[-T] extends Observer[T] {
  implicit def scheduler: Scheduler
}

// an execution context with scheduling capabilities
trait Scheduler extends ExecutionContext {
  def scheduleOnce(initialDelay: FiniteDuration)(action: => Unit): Cancelable
}

// needed because Scala does not have it 
trait Cancelable {
  def cancel(): Boolean 
}

Now if you'll look closely, the Observable interface is pure when applying operators like map, filter, flatMap. For example, when compared with Future, when applying those operators it does not submit tasks into an ExecutionContext like Future does, it doesn't execute anything yet. The actual side-effects happen when you do subscribe. That's when you feed it its Scheduler.

This is very much like what happens in all such equivalent libraries that claim to be FP. For example with Iteratees, you take what is a pure transformation specification (the Iteratee) and feed it into something that can feed it data (and then the side-effects happen). This is much like how the Task in Scalaz functions, when you call run() on it. Or how in scalaz-stream or in Akka streams you give the producer a Sink it can push values into.

THE difference between what Monifu does and more FP approaches is that the protocol of communication is more dirty. Dirty in the sense that it's basically a state machine with some rules, rules which are not enforced by the compiler, but that are very common in a communication protocol (e.g. you send onNext repeatedly, but by calling onNext successively you have to apply back-pressure by waiting for Continue, until you have to finish, when you call onComplete or onError, but after that you're done and no more calls are allowed and please, pretty please don't call those methods concurrently in any context).

Surely the Iteratee is very attracting because it's basically a function whose result is a another function which is supposed to process the next element in the stream and that may make us feel all gooey inside, but (on the JVM at least) that has proved until now to be inefficient and IMHO that doesn't buy you much except complexity of understanding. And compared with the Iteratee, in Monifu the users are not supposed to interact much with this communication protocol, because the library is supposed to give you every high-level operator or producer builder needed to safely do any of the tasks required. Monifu and the Rx model in general is also not elegant in the sense of describing everything with a minimal set of operations. Because even though that's possible, the model leans on pragmatism and efficiency and for example you'll never see Monifu (or Rx in general) trigger stack overflow exceptions.

And I'm biased of course, but in my opinion Monifu is what Scala should be leaning towards: pragmatic (pop) functional programming.

Hope that sheds some light on the mentality with which Monifu was build :-)

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

@tpolecat Thanks for the clarification - I do concede I read the comment directly in relation to Monifu, not in the general sense, so my apologies there.

But do I care...absolutely!!

@tpolecat

This comment has been minimized.

Member

tpolecat commented Dec 18, 2015

@alexandru awesome, thanks for the response. Very helpful.

So, I think there may be an important distinction with regard to Task/IO, which is that calling .unsafeRun/.unsafePerformIO is external to the logical program; at least in principle it happens only once (in main) and is not observable to the program itself. It's kind of an FFI that hooks you up with the impure part of the language.

On the other hand .subscribe seems to me to be integral to the construction of the logic of your program. But I'm not very familiar with this style so it may actually be an end-of-the-world activity like .unsafePerformIO. Maybe you could clarify.

A related concern is that the internal state might means that constructing an Observable is a side-effect; i.e., is this

val a: Observable[Foo] = expr
...
val b = a
...

always the same program as this?

val a: Observable[Foo] = expr
...
val b = expr
...
@alexandru

This comment has been minimized.

Member

alexandru commented Dec 18, 2015

@tpolecat, cool, I like to talk about it :-)

.unsafeRun/.unsafePerformIO ... it's kind of an FFI that hooks you up with the impure part of the language.

That is exactly right, although in terms of the API, the devil is in the details. Rx is often said to be about FRP, although in truth that might be a wrong thing to say. What you end up with in Rx/Monifu is with logic for combining values often taken from multiple streams by means of the usual suspects, like concat (alias for flatten/flatMap in Monifu), merge, zip, or combineLatest, amongst others. And you also end up transforming that data by means of operators such as scan (it's like a foldLeft, but it emits every intermediate step), an operator that allows you to model state machines. Operators like combineLatest and scan capture the essence of FRP. And if this model has a strength (compared with alternatives) is in handling concurrent stuff.

You can treat .subscribe() as being at the edge of the world and it's what I do, because OK, on one edge you've got one or multiple producers of data pushing signals, signals that go through a pipe of transformations and on the other edge you've got to do stuff, like save those values in a database, in OpenTSDB, or send them on a web-socket connection, or through Akka remoting, or log them to disk, and so on. But in between the producers and the consumers you can (and the model encourages you to) model the logic with pure functions and values.

But the interface of Observer resembles the interface of an Actor (e.g. onNext is basically a receive), it's fairly intuitive, so what my colleagues are sometimes doing is to place business logic in the Observer itself. You can even have a wrapper that allows you to work with something akin to the context.become in Akka. I don't like it when they treat Observers as being actors and this is where the obtuse implementation details of something like Akka Streams helps, but you know, full disclosure :-P

A related concern is that the internal state might means that constructing an Observable is a side-effect; i.e., is this ... always the same program as this? ...

The answer is mostly yes, sometimes no. Usually building an Observable has referential transparency, but there are some exceptions. You see, we've got two cases:

  1. cold observables, for which each subscriber gets its own data-source (when .subscribe happens)
  2. hot observables, for which the same data-source is shared between multiple subscribers

The need for a hot observable happens on expensive producers. For example an open file handle, like a persistent network socket might be an expensive data source and you might want to keep it active for the whole lifecycle of the application and you might want to share its data for whatever listener you might have. Picture an expensive RabbitMQ connection that then needs to be hooked to whatever client happens to connect on your web-sockets :-)

So cold observables are always referentially transparent, because cold observables are nothing more than factories, so the creation of this observable has zero side effects:

val a = Observable.intervalAtFixedRate(1.second)
  .scan(0)(_ + _).filter(_ % 2 == 0)

val sub1 = a.subscribe(obs1) // <-- gets own datasource, count starts immediately
val sub2 = a.subscribe(obs2) // <-- gets own datasource, count starts immediately

But on the other hand you can transform a cold observable into a hot one:

val hot = Observable.intervalAtFixedRate(1.second)
  .scan(0)(_ + _).filter(_ % 2 == 0)
  .publish() // <-- and now we are hot

val sub1 = hot.subscribe(obs1) // <-- shared datasource, does not start yet
val sub2 = hot.subscribe(obs2) // <-- shared datasource, does not start yet
val cancelable = a.connect() <-- now it starts

That publish operator has turned our cold Observable into a hot one and that means it has to maintain an internal list of subscribers and possibly a internal cache (depends). Of course, the cool thing is that the transformations themselves are still pure:

// these expressions yield the same result, even though the source is hot
val a = hot.map(_ + 2).filter(_ % 2 == 0) 
val b = hot.map(_ + 2).filter(_ % 2 == 0) 

Now for example Monifu also has the concept of a Channel, which is a way to create observables in an imperative fashion and without worrying about back-pressure concerns. So you can do this:

val channel = PublishChannel[Int](OverflowStrategy.DropOld(1000))
channel.pushNext(1)
channel.pushNext(2)
channel.pushNext(3)
channel.pushComplete()

A Channel resembles a Promise somewhat, meaning that it's an Observable that has an imperative write interface that you can use to push values in it. It's very useful when integrating with other people's data-sources, like for example when you want to listen to mouse events in the browser, you need this imperative interface to push those events somehow. This channel is also a hot observable by definition (it cannot exist otherwise). Obviously, if you create multiple PublishChannel instances, you'll get different objects with state. But again, the data transformations are pure, so this is fine ...

// these expressions yield the same result, even though the channel is hot
val a = channel.map(_ + 2).filter(_ % 2 == 0) 
val b = channel.map(_ + 2).filter(_ % 2 == 0) 

But all of these hot observables are expensive and happen at the edges and rarely. Why? Because a hot observable needs to have its lifecycle managed, much like a file handle. Otherwise you end up with connection or memory leaks. And so people don't build hot observables unless they need it. And then when an Observable instance falls in your lap from somewhere, as a consumer of it you do not care about its type, you just apply operators on it and it will work just fine, the transformations themselves being pure.

BTW, in sharing a signal with multiple observers is where the design of Monifu (and of Rx) diverges from something like Akka Stream, as I tried explaining in my comparison. Their design is to have this "fan out" to be very explicit. But in the Rx model you're working with the Observer pattern on steroids, so any observer can subscribe any time. As a matter of philosophy, personally I like this model more, because it mirrors the true nature of data streaming. My favorite analogy is to a river. A river just flows and is available and doesn't care who drinks from it. And I find it more user friendly to simply have such sources available and be able to subscribe whenever.

And in regards to FP what you see coming out of that river are basically snapshots at different moments in time, or values in other words. And you care about the values to be immutable and in case you're transforming the river, you care about those functions to be pure, but you know that the source is not pure and you know that somewhere downstream that river will flow into a sea of which you've got no control. And that's fine in practice, I think :-)

@ghost

This comment has been minimized.

ghost commented Dec 18, 2015

👍

@tpolecat

This comment has been minimized.

Member

tpolecat commented Dec 19, 2015

@alexandru thanks for taking the time to write such a detailed response. I think it answers my questions.

@milessabin

This comment has been minimized.

Member

milessabin commented Dec 19, 2015

Yes, thanks @alexandru, and thanks @tpolecat for asking helpful questions.

For me, the key topic in the discussion is how we segregate the effectful from the non-effectful parts of programs. Unsurprisingly, I think there's a lot of value in doing that with types, so something like Task/IO is the way to go. Unfortunately we don't currently have such a thing in Cats or any other current Typelevel project. I think that's something we're going to need sooner rather than later, otherwise we're going to be having this same conversation every time a project which uses input/output but doesn't already use Scalaz's Task/IO wants to get involved.

@alexandru would you be willing to explore evolving Monifu in a direction which makes a more statically enforced separation between effectful and non-effectful parts of the API? @tpolecat would that address your concerns, and would you be willing to help?

More generally, do we think that activity around this would usefully feed into the design of a Cats alternative to Task/IO?

@ghost

This comment has been minimized.

ghost commented Dec 19, 2015

More generally, do we think that activity around this would usefully feed into the design of a Cats alternative to Task/IO?

Yes, definitely - also see @non 's summary on this a while back https://github.com/non/cats/issues/32#issuecomment-141079117

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 19, 2015

Hi @milessabin, @tpolecat,

I'm very willing to explore ways to make a better separation between the effectful from the non-effectful parts and if anybody else wants to help that would be awesome. Yes, please do jump in :-) And I also wanted to explore how Monifu could better integrate Cats' types as an optional package. I mean, I went through a painful design process and made changes over time to make that happen (with Scalaz actually, but now Cats seems like a better idea). And will probably do more.

I do have a concern that you're picking the wrong role models. I mean, if you're talking about the Task in Scalaz, for one I don't see what Task does better, plus it has a broken API. You can see that from def run: A, or from its dependency on the ScheduledExecutorService from Java. I mean, I totally get why some people would prefer Task to Scala's Future, as Scala's Future is effectful when applying any of its operators, but guys, the design choices in Future are good sometimes and there's something to be said about Future working on top of Scala.js out of the box, while Task does not. You should also keep in mind that Monifu's Observable is a superset of Future and of Task, as you can think of Future as an Observable that emits a single element and then stops. And my view is that the more flexible and potent you make the model (e.g. one element versus a unidirectional stream versus actors) the more complex it becomes. In other words it's much easier to design a Task.

And the Scalaz IO is cool, but it was designed to separate the effectful from the non-effectful in the sense that's all it does, that's its raison d'être and it has no other. It has a much easier time, because it's not about asynchronous execution, it doesn't need to handle the notion of shared connections, it doesn't need to do buffering, it doesn't need to address back-pressure concerns, it doesn't need to handle multi-threading concerns.

Don't get me wrong, I just don't want to set some wrong expectations. I am for example interested in the design of scalaz-stream (currently suffering a rename from what I see). I could also help with the design of a Task for Cats, preferably one that doesn't copy the one from Scalaz.

@adelbertc

This comment has been minimized.

Member

adelbertc commented Dec 20, 2015

For me I think one of the important distinguishing factors of Typelevel, at least the TL of a year or two ago, was the functional nature of the libraries. When writing pure functional code, I could pretty much turn to any of the Typelevel libraries and have either seamless or easy integration with the library and keep everything pure.

No formal documented process has been put in place for the new state of the Typelevel umbrella in terms of what projects get accepted, but in my mind the same principle holds true - the library should allow ease of integration into pure functional code. Direct wiring into say, Task or IO perhaps is not necessary, but some mechanism for doing so would be nice. I have no issues with the use of Observable as it stands so long as users are also given an API that say, uses scalaz-stream (or soon to be FS2).

If Monifu has a way of clearly separating out the effectful from the non-effectful parts and perhaps providing mechanisms (either directly or otherwise) to hook into something like scalaz-stream or some other source, I would love to see that.

A side note on Task#run, while the method is certainly there, in practice is is never actually used until the end of the program. One does not typically call Task#run scattered throughout the code. Other than that I'm not too sure why the API is "broken."

@milessabin

This comment has been minimized.

Member

milessabin commented Dec 20, 2015

I've created #5 for general policy discussion. This issue should focus on constructive discussion of the suitability of Monifu for joining Typelevel, and the suitability of Typelevel membership for Monifu.

@milessabin

This comment has been minimized.

Member

milessabin commented Dec 20, 2015

@adelbertc my understanding is that Task#run is unimplementable on Scala.js. Describing that as "broken" is maybe not appropriate, but it's clearly problematic and something we need to address before insisting that projects which want to be portable use it.

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 20, 2015

@adelbertc, @milessabin do excuse my strong language. Me bringing up Task's design was a defensive reaction. I'll try explaining my reasoning below, over 2000 words until now and counting, please bear with me for another 700 :))

One typically does Observable.subscribe at the edges of the program and doesn't have them scattered throughout the code. As said .subscribe() is much like .unsafePerformIO, and I must say that Monifu has been the driver for the adoption of FP principles in a team of 20 people, because it is forcing people to model pure transformations operating on immutable data-structures like nothing else in our stack, competing for attention not with Task or Future or IO, but with Akka actors. Basically Monifu's usage is replacing actors that have internal maps and timers aggregating signals and send messages back and forth in an unholy dance of mutation and non-determinism that can make grown men cry.

To set the right expectations, Monifu is not going to expose the API of FS2. Not because I'm stubborn, but because these libraries end up mirroring their types and you can't just hide an Observable into a FS2 Process, or vice-versa. Obviously Monifu and FS2 have different priorities that have merits and of course it depends on needs. If FS2 will ever join the Typelevel family, then I don't see why it couldn't coexist and interoperate with Monifu. I mean if you're worried that Monifu will not interoperate cleanly with the rest of the Typelevel libraries, I can assure you that it will, as this has been one priority all along.

Back to Task, it's not that its API is this huge mistake, as it's very much understandable why it exposes run() and run() does make some sense. However I do think Task.run calls do get scattered throughout the code because Task is used to model discrete events / side-effects that often happen repeatedly for the whole duration of the program and so if those effects are repeating, you can't help scattering them all over the place. Or maybe you end up with recursive loops (flatMap(repeat)) that trigger other very real and not so FP friendly side-effects. And that's because people are being people, your competition being actors with maps, timers and asynchronous callbacks capturing your mutable internal context.

In other words I believe Task, in a very similar way to Scala's Future or with Akka actors, ends up being overused in absence of facilities for handling unidirectional streams.

Keeping this in mind, I think Task.run proved to be a bad idea because Task is used for modeling both synchronous and asynchronous effects, often not triggered quite at the edge and not having a clean API separation is harmful, much in the same way that you treasure a clean separation between the effectful and the non-effectful. Because in absence of an M:N platform that can fake synchronous calls or that can block the underlying threads, it's not possible to implement it, but also because blocking threads on the JVM has performance implications and is error prone due to the reality of how threads and thread pools work. This is another thing that Scala's Future got right: execution is always asynchronous, Await.result is not part of Future's API and when you do call Await.result then the underlying pool gets notified by means of Scala's BlockContext to maybe add more threads. And the nice touch is that Await.result requires a non-optional timeout: Duration to be specified, signaling to the user that this call is expensive, without resorting to vocabulary tricks, such as placing unsafe in the name of that function. API design is really hard and it's the nice touches like these that make the difference :-)

Going back into the feelings burdened and philosophical territory, also related to issue #5 and guys please don't take this personally in any way - it is my personal feeling that the tension against reactive-streams.org and the Rx model is that this producer/consumer communication protocol would never happen in a language that strives to maximize purity, such as Haskell. I can understand that, but the Rx model and reactive-streams.org are playing to the strengths of the underlying impure platform, instead of working in spite of it, being all about encapsulating the ugly non-determinism in a nice API. Monifu's Observable is defined by its subscribe(observer), much like Scala's Future is defined by onComplete(callback). You see, when working with Scala's Future you're being hidden the ugly truth that all of Future's operators are being built on onComplete. And you know, that's fine and if you want to single out the problems that Future has, its definition being based on onComplete is probably not one of them.

But you know, if you feel that Monifu is not a good cultural fit, I'll very much accept that, no hard feelings.
And sorry again for my ramblings.

@milessabin

This comment has been minimized.

Member

milessabin commented Dec 20, 2015

@alexandru I'd like to apologise for the discussion around your project getting embroiled in a wider Typelevel policy discussion ... please bear with us.

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 20, 2015

I think one problem with this issue is that it's hard to reason about frameworks for manipulating streams, as they are newer. So to better understand the philosophy and give you a better analogy, I've written a proposal for a Monifu Task, so please compare:

Cheers,

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 23, 2015

Btw, I'm very excited about this :-)

I now have a planned integration with cats-core on the radar: monix/monix#89. Was always on the back of my head, but was thinking of Scalaz originally and there have been reasons for why that was off-putting (e.g. incompatible with my ideals, so I've been procrastinating on it).

@larsrh

This comment has been minimized.

Member

larsrh commented Dec 23, 2015

I'm not very knowledgeable in the field of asynchronous systems, so please take my opinion with a pinch of salt. Monifu certainly sounds interesting to me and I can envision using it in my projects. To me, it looks like it gives me tools to write "almost" pure code on top of impure code, e.g. coming from third-party libraries (I'm assuming I can easily implement Observable for other libraries). Also, the fact that it runs on scala.js is a big plus in my book. @alexandru's remarks about when subscribe is called and how it relates to a functional programming style make sense to me.

In summary, a half 👍 from me (for lack of expertise). @tpolecat, do you have any remaining concerns?

Finally, thanks @alexandru for your patience, it is much appreciated. I'm always happy about new prospective members – I certainly didn't expect so much interest when I registered the domain about three years ago!

@stew

This comment has been minimized.

Member

stew commented Dec 24, 2015

I'm trying to figure out what all this is, I must say that all kinds of alarm bells are ringing for me. In trying to figure out what all this reactive streams stuff is, I've been prompted to read the Reactive Manifesto multiple times (I have semi-seriously discussed with co-workers how we would feel upon learning that someone had signed this thing). I keep seeing phrases like "asynchronous non-blocking backpressure" but I can't figure out what this means, and I keep thinking of Erik Meijer's talk about this: https://www.youtube.com/watch?v=pOl4E8x3fmw&t=155 ; I also read stuff in the reactive streams specification that says I have to do manditory null checks and get more turned off. This all seems very side-effecty and object oriented.The more I read about reactive streams, they seem like the type of thing I would try to warm my friends to avoid.

SO, If I were to be forced to deal with some reactive streams, monifu seems like a great way of dealing with them. But should I be dealing with reactive streams at all? To me this is a difficult line to draw. Are we, by endorsing a library for reactive streams, endorsing reactive streams? I'm not sure.

And I'd like to thank Miles for his comment and say that I also would like to apologize to alexandru, as this has clearly become a bigger more general discussion, which has turned into "what is typelevel" and not "is monifu good software", I think to me monifu seems to be well reasoned software, and the kind of thing I would certainly want to use if I was trying to operate in the space it is operating in.

@alexandru

This comment has been minimized.

Member

alexandru commented Dec 24, 2015

@larsrh thanks :-)

@stew I understand your concerns man :-) Unfortunately the folks writing the "Reactive Manifesto" have overloaded "reactive" as a term. But the "reactive" in Monifu comes from functional reactive programming (FRP) and has nothing to do with the Reactive Manifesto which is just marketing fluff.

On "asynchronous non-blocking back-pressure", well it's a buzzword, though synchronous/blocking calls have natural back-pressure (the blocking part), therefore talking about back-pressure only makes sense in an asynchronous context. When flatMap-ing on a recursive Task, that's a form of back-pressure. And we keep talking about it because, well the protocol for doing it has to be a conscious design decision. And it's basically about having control on what happens when you've got a slow consumer with a faster producer - can we slow down the producer? Do we buffer? Can we drop events on overflow? Etc.

On "Reactive Streams", not sure what the specification has to do with "reactive", but the specification is modeling a pretty low-level communication protocol and low level communication protocols are messy. And it is meant for interoperability between libraries. As an example, consider Play Framework: until now, if you wanted to stream or consume data over websocket, you had to work with Iteratees/Enumerators. Fairly FP wouldn't you say? Iteratees/Enumerators also have "back-pressure" backed in, but you've got no control so it actually sucks if you're using other libraries. It also doesn't detect slow clients. Now imagine a badly behaved browser client that keeps making connections and ends up DOS-ing your server. To protect your server, you could say "only send events when the client communicates demand and cut connections that don't communicate demand (e.g. ping) at least once every 10 seconds". Congrats, you just invented something close to the reactive streams protocol. Also note that Monifu's internal protocol is higher level than the one in reactive streams.

As to why you'd need it? Well in the grand context of things, you could use Monifu to control side-effects (I kid you not). You could think of side effects as events that happen over time and you can't rewind time or make it go faster, but you can observe events and signals, which are just values and act on them in an FP manner. This is what FRP is about.

On a practical note, it's very useful when wanting to aggregate signals from different sources and act on them. At a project at work we've been monitoring and controlling power plants and that involves reading signals from different sources and modeling state machines in real time. And imagine we have to be really tolerant in our network communications, so if we are missing data, or if we receive noise, then we can't act on that immediately, as faulting a power plant erroneously costs money, but then we do need to act in case of faults to shift the responsibilities around and we are controlling these things in aggregate. We have been using Monifu for parts of that and it's been a good choice. Such problems are niche though, but then I'm not interested in frontends to databases :-)

And man, you can't have the lambda architecture without backpressure :-P

@mandubian

This comment has been minimized.

mandubian commented Dec 30, 2015

@stew I need to dig into monifu a bit more but I feel like @alexandru's comment is meaningful. Reactive manifesto was just a very low-level ground for interop between systems able to manage async push/pull mechanism (RX .net/java/JS & akkastream for example). But in akka-stream which is a good API compatible with reactive stuff, anything called "reactive" is really hidden and the API is very functional & typesafe oriented so just forget about reactive manifesto in our context ;)

@larsrh

This comment has been minimized.

Member

larsrh commented Jan 17, 2016

Reviewing this thread it seems to me that monifu/monix qualifies for incubator status. There are legitimate concerns about the "purity" of the library. But I think the (potential) benefits are far more interesting: First of all, monix has full Scala.js compatibility, and it is my understanding that scalaz-stream/fs2 has not (apparently it is also difficult/impossible with the current design). Additionally, non/cats#32 is open and it looks like having monix in Typelevel could help tackling that problem.

To summarize, I'm in favour for incubation. Any other objections I have missed?

@ghost ghost referenced this issue Jan 18, 2016

Closed

Scala.JS build with FS2? #500

@adelbertc

This comment has been minimized.

Member

adelbertc commented Jan 18, 2016

👍 from me for incubator status - the work being done on Task is interesting. In regards to the functional-ness of the library, if it's reasonable enough to use in a RT setting (short of wrapping everything in IO/Task) then I think it's fair. I would have a lot more concerns if it was inherently side-effecty.

On a side note looks like there's some discussion re:FS2.JS (functional-streams-for-scala/fs2#500)

@larsrh

This comment has been minimized.

Member

larsrh commented Jan 19, 2016

Okay then, incubator approved. I still need to write down the steps required to add the project to the new website, and then we can close this.

@stew stew closed this Jan 23, 2016

@stew stew reopened this Jan 23, 2016

@alexandru

This comment has been minimized.

Member

alexandru commented Jan 25, 2016

Thanks folks, this is awesome.

One request: please notify me in advance before adding the project to the website.

@larsrh

This comment has been minimized.

Member

larsrh commented Feb 27, 2016

Sorry for the delay. PR to add it to the website is pending: typelevel/typelevel.github.com#91.

@alexandru I'm waiting for your 👍 on the PR, as you requested 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment