Wishlist: WebSocket

[chrysn]

Please consider exposing the browser's WebSocket API in rust-webplatform.

I did not yet try implementing this myself (still being busy on the other end of the socket), but if from Rust I could establish WS connections and send/receive binary data frames (probably as a JavaScript ArrayBuffer), that should allow smooth integration of the existing coap crate for CoAP over WebSockets communcation.

[chrysn]

I did put together a working implementation; it'll need some polishing, but at least demos the most basic functionality.

[tcr]

Hey @chrysn! I invite you to look at issue #22 if you're interested in potentially contributing to this library. Currently looking for maintainers or to combine efforts.

Would be interested in seeing your working implementation by the way!

[chrysn]

Thanks for the invitation @tcr, but I'm both rather new to Rust and to emscripten, so I wouldn't be very helpful in maintaining this library. (Have a look at the code to see why; this is about the first non-hello-world-ish code I've written in Rust).

My websocket branch now contains something usable for transferring binary data, but

• I had to duplicate a lot of code to support callbacks that are not an Event, and to pass data that's not a string. The arena has thus become a struct with vecs for whatever the as_int function needs to store. Instead of a single rust_caller, there are now rust_caller varieties for all required rust signatures. Of course, there needs to be a document.refs for each rust_caller variety. I'd be happy to write those parts in a more generic way, but I'm not sure what's the idiomatic approach here.

• Labelled "BIG FIXME" because it is: For passing binary data to JS, I need to keep the &[u8] around in the arena, but can't because it is not sized. This is probably easy to fix, but I haven't tried the various containment solutions yet. The current workaround is not to push it to the arena at all and hope that the js! (and thus .as_int) is called from somewhere where the array lives long enough to survive when its last reference is dropped in js!.

• Another BIG one: When receiving the ArrayBuffer from the websocket callback, I can't get an address of that buffer (which would be ideal to zero-copy the received data into a Rust &[u8]) -- that might easily be impossible to do, though, if that buffer is not a part of emscripten's addressable memory space. As a workaround, I'm _malloc'ing space to copy the buffer in what seems to be a convoluted back and forth between arrays and buffers, and worst is that I didn't find a function to free the memory again after the Rust call is over.

• My public method naming is a balancing act gone wrong between keeping with the WebSocket API naming (addEventListener) and having to suffix the kind of event (because a connect listener would only want to be called w/o arguments, while a string message or binary message listener would expect to be called with a String / &str or &[u8], respectively). (Sending around only Event objects does not feel like it'll work well in a language as strict as Rust, where JS can easily access the .data of any event and get an undefined if it was a connect event).

  Personally I'd probably even switch the callbacks to a Protocol style object where the public API is only a <P: WebSocketProtocol> document.websocket_create(protocol: P, url: &str) that will then call .connect() and .message_binary() etc. of the protocol object implementing the WebSocketProtocol trait (so no user visible callback registration any more at all), but that's coming from asynchronous Python programming, and might not fit the style of this library.

I've published a usage example at https://github.com/chrysn/wasm-websocket-demo .

[chrysn]

It would be nice to bypass JavaScript and the whole WebSocket DOM event registration and message event data extraction completely, but that would need to be supported by WebAssembly and AFAICT isn't right now. There is a mechanism in emscripten around emscripten_set_socket_message_callback, but that looks to me more like a mechanism that takes the JavaScript events and buffers data for reading with POSIX-style .read() operations into an application-provided buffer than a direct way of accessing a WebSocket from WebAssembly.

[tones111]

Not to derail any effort here, but I've also been working on emscripten socket support lately. You may find this pull request useful. In the comments is a link to a repo with wasm and asmjs examples.
tokio-rs/mio#612

[chrysn]

@tones111, i'd be happy to see my branch superseded by something different, but it seems to me that your mio integration adds emscripten's socket support (which is what I commented on in #23 (comment): providing something that looks like a TCP socket implemented using WebSocket).

What would be a great combination, though, is if my branch (or anything else providing the browser's WebSocket to Rust) could be made into a standard mio WebSocket client (maybe https://ws-rs.org/?), so that developers don't need to use different WebSocket APIs on different implementations.

[tones111]

My appologies if the mio integration doesn't align with what you're looking for, but I'm failing to understand why it wouldn't. The original post for this issue asks for the ability to...

from Rust I could establish WS connections and send/receive binary data frames

which is what I believe I've been able to accomplish via the Emscripten socket API. Emscripten sockets need a websocket server on the other end to communicate with. If you look at the traffic in wireshark you can see the HTTP negotiation and protocol upgrade. As I understand it Emscripten's sockets only operate with a "binary" protocol. Going this route skips having to implement anything in javascript (handled by Emscripten) and allows the client-side rust to focus on the data going over the socket.

If you want a concrete example my repo at https://github.com/tones111/mio-example builds up a server application using HTML (hyper) and websocket (ws-rs) and serves wasm to the browser. That wasm code is using a mio socket to talk to the port opened by ws-rs.

As far as I know Emscripten is currently the only way to get rust running in the browser. I imagine there will be other socket APIs as wasm or LLVM support matures, but you can't target something that doesn't exist yet.

Again, apologies if I'm misinterpreting the request.

[chrysn]

The Emscripten API you're using is sending binary frames, but it's expressing that process as a TCP socket. That means that no subprotocols are exposed, only binary frames can be exchanged and, (worst IMO) that emscripten would be free to join messages and send or deliver them as a merged (although they don't right now). That works fine if what one wants to substitute a TCP connection by terminating it at the server and tunneling it through a websocket to the browser, but not for using a WebSocket's potential.

With respoect to emscripten being required, AFAICT emscripten primarily provides an operating-system-like environment to programs running inside. That's nice to get println!() macros etc. working, but apart from that, it is my impression that rust-webplatoform tries not to depend too much on emscripten's features, the least of them on where it emulates POSIX APIs that a classical application might expect from its OS.

[tones111]

Thanks. I agree with you're points on emscripten's limitations. I'm hopeful that wasm will define apis that map more closely to the underlying functionality. Good luck with your implementation.

[chrysn]

@tcr: I only now read the tail of #22 and got what you meant with combined efforts. My approach would probably work equally well with stdweb (possibly even with less cross-language persistence trickery as the stdweb's js! seems to be more powerful). Shall I try to port it there?

[kpcyrd]

hey!

this issue and this repo1 show up when searching for "rust wasm websocket". Are there any pointers for somebody looking for wasm websockets in 2021? :)

Thanks!