On Observables vs Emitters

[benlesh]

Ahead of reading this: I'd greatly like to apologize from any perceived tone here. It's very hard to criticize someone else's hard work without it sounding a little bad in a few spots. That's really not my intent, so if reading any of this upsets anyone, I'm sorry, that's not my intent.

Responding to some of the claims about Observable

I want to clear up a few of these statements I found in this repo:

Observables are similarly push-based primitives

True

Observables do not handle multiple consumers (by design)

This is not true. Observables can handle individual and multiple consumers, as a biproduct of closures. Observables are an extremely low-level primitive, like a specialized function. For example:

const socket = new WebSocket('wss://echo.websocket.org');

// Utilizing an observable from a suggested change I had to the original proposal
const source = new Observable((next, error, complete, signal) => {
  socket.addEventListener('message', next);
  signal.addEventListener('abort', () => {
    socket.removeEventListener('message', next);
  });
});

// Consumer 1
source.subscribe(e => console.log('1', e.data));

// Consumer 2
source.subscribe(e => console.log('1', e.data));

// to get the echo
socket.onopen = () => {
  socket.send('test');
};

Now, assuming that we have observable in the language, it's possible to make all of this more ergonomic by having EventTarget, et al, use that observable:

// The above could be
const source = new Observable((next, error, complete, signal) => {
  // `on` returns an Observable, which can take the signal directly
  socket.on('message').subscribe(next, error, complete, signal);
});

// Which, of course, reduces to:
const source = socket.on('message');

Handling multiple consumers is a compositional feature of Observable. You're not locked into "always multicast" or "always unicast". So the above statement is wrong and should be removed, IMO.

Observables invent new error-handling semantics ("complete/error/done"), which becomes a barrier/challenge in integrating it with the rest of the language

Observables error-handling semantics are not much different than that of Promises. So I think "invent new" needs to be explained here in great detail, or it's really just editorializing. "becomes a barrier/challenge" is also something that needs to be demonstrated and justified, in detail, against the challenges of current, existing types, such as callbacks and promises without editorializing.

To be clear, this is a type that has been in many languages and in JavaScript for more than a decade. The error semantics here are well defined and understood. It's not a new type that was created for a new proposal.

Observables can be thought of as a higher level abstration than Emitter, one that internally generates a new Emitter on every subscription.

If Emitter is a "lower level abstraction" than Observable, it should lend itself to more use cases than Observable and cover more.

Observable:

• sync or async
• unicast or multicast
• lazy (optional creation of data producers when subscribed to)
• cancellable (consumers can signal disinterest to free up resources)
• deterministic resource management (created resources WILL automatically be torn down when the stream is completed, errored, OR when the consumer(s) is/are no longer interested in produced values)

Emitter:

• sync or async
• always multicast
• lazy-ish (does not create new producers when subscribed to)
• cancellable-ish (not all resources can be freed because no means of defining teardown)

While you might be able to create an Observable using functions an Emitter, I'm not sure it's going to be very efficient or useful to do so. However, I'm fully confident that with an Observable primitive in the language, an efficient Emitter library could be created as a third party thing.

Emitter would be closer to what is called a Subject in Observable-land. However, this is also not quite accurate as when you next onto an Emitter it's not just directly broadcasting to it's children, but rather lets the Emitter process the value.

This is inaccurate. Basically it is saying "it's not Subject, it's a subject and operators" in RxJS-speak. This totally works in RxJS, and has for 4-5 years now:

import { Subject } from 'rxjs';
import { map } from 'rxjs/operators';

const subject = new Subject().pipe(
  map(x => x + x)
);

subject.subscribe(x => console.log(x));

subject.next(1);  // logs "2"

... that said, I'm not sure it's a great feature:

1. It's been hard for people to understand
2. It's completely impossible to type in TypeScript or any typed variant of JavaScript (lacking higher-kinded types)
3. It often times results in people leaking implementation details. (Did you really want to give someone access to next?)

Emitter is unidirectional, all signals go downwards (whether that's values, or an Emitter resolving/rejecting like a Promise would). A child can never affect a parent by design, especially not implicitly. This gives developers guarantees which make them easier to reason about. Observables, and other libraries, are built on a bidirectional architecture.

Without a reference implementation, this is tough to say. This "bidirectionality" you're speaking of here is really just the result of composition. Each one must subscribe to the previous. However, outside of the act of subscription, all observation (the important part) is completely unidirectional. So this difference might be a bit of a misnomer, or at least needs clarified (both in terms of what Observable is doing, and in terms of exactly how Emitter is not going to do this.

This in part contributes to making them harder to understand/implement.

Again, this is highly editorialized and requires evidence or should be thrown out.

It seems this design is also a by-product of the original method-chaining API: when you have A.op().op().subscribe() you're subscribing on the last node, which means a signal has to travel back up somehow.

It's a byproduct of the compositional nature of functions more than anything. And a hallmark of the reusability of Observable. I don't see any how any type is going to be reusable without some sort of upward call. If you can compose an Emitter from other Emitters, the new emitter must call run (or whatever) on the other Emitters it's consuming. That's the "upwards call" you're claiming doesn't exist. Overall, I think this claim should be removed.

Issues with Emitter in general

I touched on a lot of this above. But I'd like to summarize my concerns a bit.

1. Emitter is decidedly less primitive than Observable. Observable is more of a specialized function with some guarantees around safety and resource management. Emitter, as outlined in this proposal, is effectively a library of features.
2. Emitter lacks a basic cancellation primitive. This is something the language could really use. Cancellation via until is a bit more complex than returning a Subscription, or even better, passing a cancellation token like AbortSignal or the like.
3. Emitter is highly unlike anything else in the standard. It's a very different API, overall. For example, until is a highly polymorphic function that takes a number, or a function, or a Promise, or an Emitter. Nothing else in JS does that, other than something like Array.from, which in my opinion is a bit different. With until, you get different behaviors based on different values passed. While RxJS also does this in a few spots, I don't think RxJS should be in the language either.
4. Emitter is an untested type. I can't even find a reference implementation to play with. Other contemporaries, like Observable, have been around for more than a decade and are currently powering vast swathes of the web.
5. Introduction of some of these compositional functions (run, for example) seems to fly in the face of better function composition proposals like the pipeline operator or bind operator.

On the grounds of number 4 alone, I would strongly discourage standardizing this type, until more evidence of its usefulness is available.

At the highest level of this proposal, I'd like to steer clear of the author's feelings on how difficult Observable is to understand in comparison to this new Emitter type. I honestly think they're both hard to understand, however, all evidence is that RxJS, which is a beefy, heavy-handed, kitchen-sink approach to Observable, is something people are catching onto more and more easily, and the popularity is growing rapidly. 12.5M downloads a week on npm, used in several large well-known projects (YouTube, Netflix, Slack, PS4, et al), someone even organized a conference about it. None of that is to say Observable is "better", but it is to say that it's certainly does not seem difficult for a large, and growing, number of people to understand. I taught myself observables, and I've been teaching them to other people, and frankly, I'm not that smart.

[benlesh]

Related, I think that the current Observable proposal, as it stands could be simplified quite a bit and I filed an issue there as well

[pemrouz]

Ahead of reading this: I'd greatly like to apologize from any perceived tone here. It's very hard to criticize someone else's hard work without it sounding a little bad in a few spots. That's really not my intent, so if reading any of this upsets anyone, I'm sorry, that's not my intent.

Really appreciate this foreward, the fact that you're conscious of how other people feel and the effort you put in. Since we've spoken in person before, I totally understand how kind you are as a person!

I also appreciate the detailed engagement, despite not entirely correctly, charitably or fairly characterising the proposal/effort. In it's current lengthy format, I think it would be fair to say it would be unproductive to attempt to respond to everything line-by-line here, but I propose the following breakdown:

• A number of comments relate to mischaracterisation of Observables. We're working on some new docs that includes a section showing how you might do the same with Emitter as particular popular third-party libraries. I'd be happy to review this with you and we can remove anything that's not correct. Most of this prose will removed from here in favour of examples that will hopefully make it less subjective to different interpretations.

• A number of comments relate to some bikeshedding. For example, untilNumber, untilPromise, vs having until(number), until(promise), etc. I think these are a great candidate to open as top-level issues and strongly recommend you try to convert as much into smaller, focused issues we can discuss on GitHub.

• A number of comments relate to more general points which can be broken out too e.g. function composition, which I agree with, can explain more about, and what the challenges are around that. I'm sure you are probably aware too, and if there are suggestions you'd like to propose, it might be good to list the pro's as well con's of the different solutions.

• A number of points about Emitter are not correct (e.g. cancellable-ish - .resolve(), deterministic resource management - finally lifecycle callback), but I assume this is largely due to my poor explanations here. Hopefully these will be clearer after new docs, and we can discuss offline or over a call to clarify/iterate.

• A number of comments relate to feedback. It's not a great approach to be dismissive of all and any critical feedback, minimizing it as only "the author's feelings", implying their "not that smart" if they don't understand Observables, or we begin to debate how popular the metrics actually are, or listing every tweet/company/person/framework that has used/not-used/praised/criticised/backed out of using RxJS.

• A number of comments relate to popularity. Everybody understands experimental feedback is important and part of the process. Right now at Stage 1 my approach is to ensure we agree on the major semantics so we later publish and promote something that is reflective of other member's view's too rather than just my own. There's already been lots of great changes. I'd personally much rather not publish anything at all, than go ahead and publish my own library, get adoption, and use that as a bargaining chip to try to force people to implement it. Popularity on it's own is also not the end goal, many libraries are not suitable for standardisation, get superseded (Angular 1), or often inspire language features, but not imported directly ($ vs querySelector/querySelectorAll, or .?).

Overall, I think you may be approaching this from the wrong perspective. There's already significant challenges to introducing any primitive here. Given that you've already revisited thinking about how you could change the Observable's proposal now, and it's even more similar to Emitter, it would be great to discuss the similarities beside the differences that remain, see which one's we can narrow done and work together on this proposal.

[benlesh]

I think really I just need to see an open source reference implementation.

[benlesh]

My primary concern: I think having a next method, and forcing multicast, makes Emitter too high-level and makes the type less useful as a building block. If you remove that though, it's an Observable, and really we should be cooperating on that proposal.

My secondary concern is that we'll be throwing away more than a decade of knowledge and use around a very primitive type that has been proven useful, in order to standardize something untested that the community has not found use for yet.

But again, I'd really like to see an implementation of it.

[lucasbasquerotto]

@benlesh I agree with a lot of what you said, but unless I haven't understood correctly, I have to disagree with:

However, outside of the act of subscription, all observation (the important part) is completely unidirectional.

If you see this answer I gave on SO about a problem of Error: no elements in sequence, you can see that what caused that was that first() was called (by the consumer), but the stream was closed because a takeUntil() (also called by a consumer) received an Observable, and the Subject that generated the Obsevable received an error when called next() (producer).

The error happens because first() needs to receive at least one item before the stream closes, but the error is not received in an argument in the subscribe(), but when next() is called. This means that the producer received an error due to something done by a consumer, so it doesn't seem unidirectional to me.

Of course, this is related to an RxJS implementation, and first() might not even be included in ECMAScript, even if Observable is included, so this doesn't mean that a Observable itself would be bidirectional; but in any case, given that RxJS ends up being the reference for Observables in the javascript world, and that if the Observable proposal is accepted, it will probably take the RxJS implementation as a reference.

[benlesh]

That's still related to the subscription. Not the emissions. first, take, takeUntil, et al, all cause teardown. Which travels back up the chain. But that's related to the operator chains and how they subscribe. Setting up an observable subscription to unsubscribe is purely optional.
What's really interesting is that individual piece can also be modeled as an observable, as it's unidirectional. You can pass a hot observable to a subscribe call as a cancellation token. I'm not strictly talking about the rxjs implementation. I'm talking about observables at a high-level.

[lucasbasquerotto]

I think it's fine to travel up the chain, as long the producer (subject) doesn't fail or something of the sort.

That said, it seems that the error is not actually in the next() (causing an error in the producer), and can actually be handled in the 2nd argument of the subscribe() method (I've updated my answer on SO).

(I still think that first() behaves in an unexpected way, but that's not about bidirectionality, and it can be circumvented with take(1))