Websockets protocol support

[rbtcollins]

(From #2)
Websockets (RFC6455, and a new I-D soon for hosting it on top of h2) is a distinct protocol from HTTP, but uses an HTTP handshake to initiate the connection (and to handle initial errors). There is support in gunicorn and uwsgi for websockets as an escape from WSGI today, but its not standardised. The gunicorn approach involves grabbing the raw socket, the uwsgi approach provides a dedicated API. Regardless of the outcome of #10 we should offer a consistent API so that folk can write to a spec rather than a specific deployment server.

Framework support - (Django)[https://github.com/stephenmcd/django-socketio]

While the protocol is fairly large, the API surface is shallow: we need to be able to accept a request and then send and receive messages. Or we need to complete the request with a 301/302/401/407/5xx (though I have no idea about the behaviour of e.g. mozilla's websockets client when non-101 status codes are received - there was quite a lot of debate about the merit of supporting that and browser implementors may ave chosen not to, but the spec does suggest it MAY work).

Once accepted we need to be able to send messages with either text or binary data (the spec supports both). We probably need to be able to support some wait/timeout facility (e.g. recv a message but if nothing received after 30 seconds return so that I can do something locally and perhaps send a message of my own).

Extensions to websockets are an interesting thing to consider. Here are three:

• http://msdn.microsoft.com/en-us/library/hh553781.aspx (allows disabling masking - its not negotiated on the wire, but set independently within an org, after which it operates as set - so this would perhaps fit as a ws extension to the environ).
• https://tools.ietf.org/html/draft-ietf-hybi-permessage-compression-18 - this is a regular extension in that it acts as a filter - as long as we can set the Sec-WebSocket-Extensions header and surface the RSV1 bit somehow (both for send and receive) this should be implementable on top of the message layer. E.g. we might say message.frames contains a list of the frames and their control flags on reads, or offer an extended recv API.
• https://tools.ietf.org/html/draft-ietf-hybi-websocket-multiplexing-11 - this draft has expired, likely because h2 offers the same facility. That said, I don't see how this can be implemented within 'userspace' or even middleware: its intent is to create multiple distinct websocket sessions all of which need to reenter the entire server stack - or one will have different server stacks serving multiplexed vs non-multiplexed connections.

Subprotocols are a key feature, but easy to support. Clearly we need to get and set the Sec-WebSocket-Protocol header during the handshake. Subprotocols don't change the framing at all - they define the application semantics expected - basically the same as ALPN but within the websockets transport.

Things we don't need:

• HTTP request bodies
• HTTP response bodies
• Access to the raw websocket frames (ping/ping/close etc)

[GrahamDumpleton]

Are WebSockets still principally used in a request/reply like manner, but with a persistent connection. Or is it quite normal to be generating lots of outgoing messages without receiving new incoming messages.

The concern I have always had about a high level API is if the receipt of new messages is blocking, then the ability to send many outgoing messages while still waiting for new ones requires multiple threads. That is, you either need multiple threads to deal with incoming and outgoing at the same time, or you need an internal thread (possibly per session), in the underlying server, which can be waiting on the incoming and then push it into the application rather than the application having to pull it.

You mention a timeout on getting a new incoming message as a means to do non blocking gets, but it is a poor substitute for a proper notification system.

So in practice how do WebSocket applications work as far as processing incoming messages and generating outgoing messages?

[GrahamDumpleton]

The other thing I would be interested in thoughts on is what makes sense as far as trying to perform performance monitoring of a WebSockets application?

Performance monitoring generally works best when it is a request/reply model where the request is pushed into the application and when it is done with that and issued any reply messages, it returns. That then marks bounds of tracking how long it too to handle.

When you have messages where there isn't necessarily a one to one relationship it gets very murky. Things are made more difficult when using a get (pull) style interface for incoming messages as the interface point itself doesn't give you anything to wrap to allow you to time how long processing of a message takes. The closest you could get is to start timing when the get request returns and stop timing when a new request to get another message is made. But then this only works on a single threaded consumer scenario. If you have a thread which is simply just pulling messages and then sticking them in an application specific internal queueing mechanism for handling, that doesn't work.

So what sort of monitoring might even be practical/useful in a WebSockets based application. Is it just too hard and any solution would have to be application specific all the time?

[GrahamDumpleton]

And just for the record, for mod_wsgi the current path to support WebSockets, at least in embedded mode, is to leverage the mod_websocket module for Apache. In using that, the interface exposed would one where you would say:

WSGIWebSocketScriptFile /some/path/app.py

In the provided Python script file you might have in its rawest form, as it maps directly to mod_websocket, the following:

def on_connect(server):
    return some_obj_for_session_to_pass_back_in_later

def on_message(server, session, msgtype, msg):
     …

def on_disconnect(server, session):
    …

The 'server' object would expose a method send() for sending back a message and a close() method.

The mod_websocket implementation has a restriction on send() or close() being used in on_connect as at that point the connection hasn't actually been completely set up.

You could make this prettier by having a single callable equivalent to on_connect() but which returns a object instance which serves as both the session object and has the methods defined on it.

class EchoServer(object):
  def __init__(self, server):
      self.server = server

  def handle_message(self, msgtype, msg):
      self.server.send(msgtype, msg)

  def cleanup_session(self):
      …

def create_session(server):
    return EchoServer(server)

Anyway, the point of this is to show that based on mod_websocket you would end up with a push style interface where the web server pushes new messages into the application.

The session object could still create a thread to allow it to send back messages at any time still, you just obviously have to be careful with multithreading around closing down the session.

This push style interface at least makes monitoring of incoming message handling time partly easy to instrument, but still doesn't deal with fact that it may actually be pushed onto a queue for internal handling. Any activities of a separate thread to consume the message or send responses would have to be separately instrumented for monitoring.

[GrahamDumpleton]

Thinking some more and it is possible to avoid needing two threads for session.

In Apache at least one thread is used inside of Apache to deal with the connection, when it passes that to the application object for the session, it can readily stick it in a common queue, with the session data, with a single process wide thread (or maybe more if want to have more consumers), to process messages coming in via any session.

So this is quite workable. The only issue with Apache of course is that it has to keep that thread dedicated to the connection under the covers.

If this push style API is hosted on top of a ASYNC server, you can still use the same API as described above and the internal server thread would just go off and do other stuff between messages.

In either case, the application side doesn't need to care. It just needs to make sure it always stores data against the session object and not thread local data as in ASYNC web server, same server level thread would be used on different sessions.

Now to contemplate how this API works where the application wants to operate in an ASYNC world as well.

[rbtcollins]

"Are WebSockets still principally used in a request/reply like manner, but with a persistent connection. "

No. They can be buts its not the intended use case. Think 0mq, AMQP or STOMP but for the web. So pub-sub patterns - either end sending messages as and when desired.

For instance, imagine a chat application: 5 people connected, 5 websockets. When A sends a message, the server notifies via (say) AMQP the other 4 workers (be they threads, processes, whatever) and they all then generate a message immediately to push the event to their browser. No polling, so it approaches the scalability of e.g. IRCd which AJAX polling simply cannot.

Or in OpenStack the horizon web UI wants to be able to show folk stuff happening to their servers, and today it has to poll - browser to API, and then API to backend API. Obviously there's a chunk of stuff to sort to make it all hang together nicely: websockets browser -> API, then pubsubhubbub callbacks from the API servers when state changes happen to servers, and finally hypervisors publishing messages about server changes.

"Or is it quite normal to be generating lots of outgoing messages without receiving new incoming messages."

Yup.

"The concern I have always had about a high level API is if the receipt of new messages is blocking, then the ability to send many outgoing messages while still waiting for new ones requires multiple threads. That is, you either need multiple threads to deal with incoming and outgoing at the same time, or you need an internal thread (possibly per session), in the underlying server, which can be waiting on the incoming and then push it into the application rather than the application having to pull it.

You mention a timeout on getting a new incoming message as a means to do non blocking gets, but it is a poor substitute for a proper notification system."

Agreed. In a non-blocking (e.g.) API (e.g. pep 3156) this is straight forward - you just have two things that can generate events - the push from the client and whatever internal thing one is managing.

However WSGI is today blocking, and if we want to be able to use common middleware for common things (e.g. extended logging, traceback collection and so on) we either need to make sure there's no IO at all in the common areas, or make all the areas have suitable blocking IO primitives.

So I think I'm suggesting that yes blocking isn't brilliant - but we need something, or we need to state that nonblock/async is the only offered API.

https://github.com/stephenmcd/django-socketio may be a useful read from the monitoring side - it suggests raising events on every message... if one had instrumented that, then you could record how long it took the event to be processed (and yes, if thats just popping it on an internal queue, shrug - at least you'll see ifthat queue blocks).

If we have a blocking push API, a middleware could record time to perform the push. The on_message one you sketch would permit that.

So the shape of this API would be:
app(environ, server)
-> server.start_response(status, headers, callbacks // protocol object) to accept and setup the connection [see e.g. subprotocol support for why thats desirable]

when start_response returns we unwind but we're now holding a reference to the protocol object/listed callbacks.

If the app has some event source - e.g. amqp, perhaps running in a thread, that source can call things on the server object to send messages.

[gvanrossum]

You might have a look at this implementation of websockets using asyncio (PEP 3156): https://github.com/aaugustin/websockets

[chadwhitacre]

Also: https://bitbucket.org/Jeffrey/gevent-websocket/.