New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding Websocket support to core #19308
Comments
Re: Inspector.
|
i'm not wholly against this but i would like to factor in how much stuff we put into all our release binaries. if we can come up with more creative ways for shipping stuff like this i'm totally a +1 (#19307) |
I am still -1 on this. |
@mscdex can you be explicit in your reasoning? |
@devsnek for the same reasons I gave in the linked issue. |
As discussed in #1010 and as a maintainer of |
Since this original post we've added http2. While you listed a bunch of protocols not all of them are supported natively by the browser. https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API |
@MylesBorins I don't think node should aim to become a (DOM-less) browser. Anyway, my "vote" and reasoning still stands. |
It's more about communicating with browsers, not becoming one. |
http2 is an argument against being too eager to absorb protocols into core. Neither its API nor its implementation are all that great; it would have benefited from iterating outside core for a while. I guess you could construe that as an argument in favor of websockets: third-party modules have existed for years and their APIs and implementations have pretty much crystallized by now. |
This is a great idea. While today Node is already an indispensable tool for web developers it does not enjoy the same full seat at the table of web browser tech advancement despite being held completely captive by it. Few agree on the controversial edicts like Node has a big opportunity to become a full fledged user agent (web browser) and first class support for web features will be a part of that. +1! |
PRs welcome. Re: websockets I'm still -1 for the time being. This is something that has been done quite well by userland and there are still unanswered open standards questions about http2+ws that require more thought and experimentation. |
When http2 was added, I think one of the considerations were that we could do a lot of low level stuff in C++ land that was hard or even impossible(?) to do efficiently in user-land. Is there a similar reason for wanting to bring WS into core, or is it just to have more features? |
@jasnell can you point me towards the open standards discussion regarding h2 + ws? |
The ietf httpbis working group is working on this .... https://tools.ietf.org/html/draft-ietf-httpbis-h2-websockets-00 It's still super early in the process tho there is some early implementation happening. |
Ideally there should be a clear general framework on how to decide whether something belongs to the core or not. I would consider points like
|
@kof I fully agree with all your points except this one – there is no reason to believe that code in Node core is better-written or better-maintained than userland code in general. I think a better question in its place would be “Is it hard to do effeciently without native addons?”. |
@addaleax The idea behind that point is: if it is in userland, it is likely there will be multiple competing implementations, which will result in attention not being fully focused on a single code base and as a result lower quality, just because less people are working on it or using it. I have no data to back it up. Userland is good when a feature needs experimentation. |
No.
Yes it is.
Yes.
No.
Not necessarily but optimizations in core may help. In
Yes I think the community will benefit from WebSocket in core. There are a lot of userland implementations. The most popular are ws, faye-websocket, and websocket. Combining the the best parts of each one of them to create a single module in core would be great imho. |
+1, mainly because @lpinca has been doing a great job in maintaining |
pinging @jcoglan and @theturtle32 |
So far I haven't seen any concrete comments on why implementing WS in core would be better than having it in user-land. I'm sure @lpinca have done a great job maintaining I just think there need to be concrete examples on how core can provide a better WS implementation than user land |
i don't think there's anyone saying we would do a better job. i think the argument is better integration and since we can ship it with a native backing users will get a perf boost without having to install anything. |
@devsnek Could you provide an example of how the integration would be better if WS was in core? Regarding native backing then n-api and pre-compiled binaries should remove the requirement of having users compile stuff (if I understood your argument correctly). |
It seems the main argument for adding ws to Core is to reduce binary dependencies. Since both native dependencies are buffer-based, I think they may be good candidates for adoption of WebAssembly. @lpinca has that been seen as a possibility? |
For an API, I'd like to propose the WhatWG WebSocket API, as much as we can implement the standard (I know there will be some inconsistencies around events). If we run into any serious problems with the API design we can attempt to work with the whatwg to update the standard. It seems like @TimothyGu has participated in this document along with @domenic. re: http/2 + ws, if we get an implementation going we can participate in trying to solve this problem!
I think implementing standards in core, especially ones implemented in the webplatform, can help the ecosystem focus on creating better abstractions on top of a protocol like WS, and avoid having to implement the base protocol themselves. If the various ecosystem modules share a kernel it allows there to be collaboration across projects with a shared interest, and potentially brings more people to help maintain core itself. |
@watson userland implementations work and I'm a big supporter of small core but HTTP(s) and many other modules are in core because they are so popular that it made sense to create a core module for them. The same is valid for WebSocket in my opinion. Quoting @domenic from nodejs/NG#10 (comment)
Is core interested in supporting WebSocket? That's an open question. I think it should. @TimothyGu we didn't explore that possibility but that's definitely a good idea. We already have prebuilt and n-api based binaries for native addons. Removing binary dependencies is not the reason why I would like to see WebSocket in core though. The question is, why is HTTP and HTTP2 in core and WebSocket isn't? They all are in the same league and they all can live in userland if wanted. On top of that a good part of the WebSocket implementation is already in core as the upgrade mechanism is already available and supported in core. @MylesBorins If we ever decide to experiment with WebSocket in core, API need to be discussed. I think we want to support piping data around so a websocket should be implemented as a |
I can't establish a vote for this, although would be a nice to have. I found that WebSocket implementation isn't that solid neither in the Browser. Is high cpu consuming and so far I couldn't see any implementation in a big scale (which I would have guess they would be using it); eg: Facebook, Twitter and Gmail, they rather relay on polling. I guess is mainly related on the cost created on the Server side due being really hard to scale it (eg. Load balancing). Not sure Node wants to make his capacity in supporting it. |
Cross referencing bugs here: introducing websockets into the core could potentially help with issues like socketio/socket.io#3212 in the long run, where people are relying on binary extensions to improve websocket speed - which is evidently not a good idea. |
In the long run aren't users going to prefer http2 streaming capabilities over websockets? |
Huh? I haven't read anything to suggest that http2 will supersede websockets. http2's streaming is bidirectional but not symmetric - only the client can initiate requests. Unless something changed recently, server side push doesn't bubble responses up to JS code. Also (afaik) just like http, http2 doesn't make any guarantees about preserving the order of client->server messages. You could implement your own message-oriented streaming protocol on top of http2 using POST messages for C->S and server-sent events for S->C. But you will have to implement your own message ordering code. And you'll have load balancer problems. Websockets are sticky-by-default. That is to say, if you have a cluster of frontend servers with a load balancer, a websocket connection will stay connected to a single frontend machine for its lifetime. Configuring load balancers to replicate that behaviour on top of normal http requests is a black art. Corner me in a bar sometime and I'll tell you all about the nightmares we had getting sticky sessions working correctly through ELB a few years ago. (I understand some libraries try to silently migrate sessions. This introduces its own concurrency problems, and it doesn't work for all use cases.) So, no. In the long run I hope websockets are reimplemented on top of http2. Although to be honest most of http2's benefits won't make much difference for websockets anyway. Over http2 the initial WS connection & handshake will be faster (since it can reuse the TCP connection it loaded the page on), but once its established it won't really matter. When websockets are implemented in http2, it will still make sense to have websocket code inside nodejs. |
As others have pointed out, adoption is actually high.
This is a false negative; it may be a chicken and egg scenario where adoption seems lower (it's not) because the tooling is not easy to use and natively available. I personally believe this to be true. Adoption will be much higher with the right low-level API.
Off topic, but it could easily solve the "should we" paradigm. Too much to say on that here. Can it be picked up elsewhere?
EXACTLY this.
As someone who has had to implement WS on a few different projects, I am certain of two things: usage will increase dramatically over the coming years by particular verticals (storytelling engineers, for one). As well, WS are never easy to use. As @mikeal said, one of the things Node.js has had going for it from the beginning was the sense that it was built to be interoperable with the browser. Java, Ruby, Python, Go, and all the other low-level languages have never been easy to work with in terms of browser interoperability, and one of the explicit reasons is the difficulty in interacting with basic protocols such as http. Userland has built up a nice ecosystem around WS, but the tooling and ease-of-use factor is still a hell-a-difficult. Treat WS as the first-class citizen it deserves to be treated. This should have been done 5 years ago. |
Especially with the growing popularity of WAMP-based apps. +1 on this. |
From the about page
Node should definitely have a WebSocket support and Server-sent Events, TPC, UDP, HTTP, HTTP2. |
A lot of IT-student in germany have to write websocket-applications. From this standpoint alone, it should be native. nodejs feels incomplete, if it isnt. You just use it natively in a browser, but in node.js you first have to find out, its not supported, then you have to decide which userland implementation is ok and future proof? At this point, i would be annoyed. Because you will feel like nodejs is not the wonderland of server-communication you thought. The next thing as student is, you have to deal with a callback hell. How many give up at this point? Let alone, that javascript wasnt very loved in the past, but is developing (ES6) very fast and has the big advantage of learning only one language in front- and backend. If that wouldnt be the case, nodejs would have died long ago - but thats just my impression. |
Definitely. |
I have strong opposition to this. Node.js is used for far more than web applications. Adding bloat to the core hurts these other types of applications and their portability, while providing no significant benefit to those who want it. Keep Node.js light and keep WebSockets in solid modules like |
@bradisbell you have a valid point -- I just think WebSockets can be considered a core protocol of the web, very much like http2. |
@bradisbell this is probably the most valid point I've heard on this. I would favor, alternately, some sort of tree-shaking methodology native to Node in that case. I.e., only import what's actually imported. Obviously waaaaaay beyond the scope of this ticket (how would such a thing even be possible), but that would be my 2 cents. Best of both worlds, actually. Significantly lighten Node core, but still have core/official requires available on demand. |
What is the actual decision making process for a big change like this? It'll be controversial no matter what the decision is. But it would be nice to have a clear official resolution on which way node will move forward with this. Can we bump this to the TC or something? |
there's nothing stopping someone from making a pr. if that pr was blocked for a non-technical reason (like "ws shouldn't be in core") it could be elevated to the TSC |
@lpinca Mind create a PR ? Just found go has websocket too https://godoc.org/golang.org/x/net/websocket. In js ecosystem,new api System like Graphql subscription rely on this: https://github.com/apollographql/apollo-server/blob/678714293337be19163e7ae0f973c6d873cf3656/packages/apollo-server-core/package.json#L46. |
FYI everyone: There's a new spec If the decision is to add (We should also wait until this API stabilizes and there is buy-in from other browsers besides Chrome) |
FWIW I've recently added in |
+1 on this. So no one would forget it. |
A new year, a new +1. (As someone whose rollup bundle is currently breaking due to use of optional binaries in – the most excellent and wonderful – ws. goes to figure out a hack) |
@aral FWIW I'm on the Fastify stack now and this is pretty solid: https://github.com/fastify/fastify-websocket |
And another +1. No one should forget. |
I'm not sure if I can vote on this issue because I'm only a Node.js user but if I can there's another +1. |
+100 ;) |
+69 :DD |
+1 |
1 similar comment
+1 |
Interesting development in the world of deno: https://deno.com/blog/deploy-streams#websockets |
Hello guys |
MylesBorins commentedMar 13, 2018
The original thread where adding this was discussed #1010 was closed with a decision by the iojs TC to rather implement lower level buffer methods, but that was abandoned.
There is an open EPS to add the feature, but we have since abandoned the process.
Some of the people who originally were -1 changed their opinions in #1010 more recently. In fact, we already ship a partial implementation of ws in the inspector.
I think it might be worth us revisiting adding WS to core.
/cc @eugeneo @rauchg
The text was updated successfully, but these errors were encountered: