update WebSockets implementation to support RFC 6455

This change allows websocket connections to be set up between browsers
and the yaws server. RFC 6455 for WebSocket connections is supported,
in addition to the hybi working group RFC drafts 10 to 17.

The quickest way to try this out is by compiling yaws as usual, then
visiting /websockets_example.yaws at the default local installation
host. This can be done using Google Chrome 14+, Firefox 7+ or any
other browser supporting WebSocket version 8 or above. Information
about getting started with WebSockets using this implementation is
given in /websockets.yaws.

This drops support for the older draft RFCs, specifically those of the
hixie working group which were previously supported by yaws but are
significantly different from the hybi working group's specification.

The interface for using WebSocket with yaws has changed
somewhat. Instead of spawning a websocket owner process which
maintains a server loop such as that shown in the old
websockets_endpoint.yaws, the application developer now implements a
callback module such as those in src/basic_echo_callback.erl or
src/advanced_echo_callback.erl -- the difference being that the
advanced callback style is only necessary if you need advanced
features of WebSocket such as fragmented messages. One suggested way
to deploy your callback module and its dependencies is as part of an
application in an OTP release, with yaws as a dependency. Rebar can be
used to build the dependencies, fetch and build yaws, and create a
release which will ensure the modules are in the path of the runtime
system.

Most behaviour tested by the Autobahn test suite 0.43 pass when
configured to connect to the /websockets_autobahn_endpoint.yaws and
/websockets_example_endpoint.yaws over an unencrypted
connection. Significantly, websocket connection closing is not
implemented and the socket is left to be cleaned up by the Runtime
System when either the connection is lost or the owning processes
dies. Secondly, certain cases where websocket frames are fragmented
within UTF-8 code points cause the check for valid text type messages
to incorrectly fail the connection.

Subprotocols are not currently supported.

Augment yaws.tex with a new WebSocket Protocol chapter (Steve
Vinoski).

doc/yaws.tex

@@ -107,13 +107,13 @@ \chapter{Introduction}
 \item Application Modules where virtual directory hierarchies can
 be made.
 \item Embedded mode
-\item WebSockets
+\item WebSockets (RFC 6455)
 \end{enumerate}
 
 \section{Prerequisites}
 This document requires that the reader:
 \begin{itemize}
-\item Is well acquainted with the \Erlang\  programming language
+\item Is well acquainted with the \Erlang\  programming language.
 \item Understands basic Web technologies.
 \end{itemize}
 
@@ -1663,6 +1663,10 @@ \section{All out/1 return values}
   is \verb+{header, H}+ with the effect of accumulating the HTTP
   header \verb+H+ for page \verb+Page+.
 
+\item \verb+{websocket, CallbackModule, Options}+ Tell \Yaws\  to use
+  \verb+CallbackModule+ as a WebSockets Protocol handler for traffic
+  on the client socket. See chapter \ref{websockets} for more details.
+
 \item \verb+[ListOfValues]+ It is possible to return a list of the
   above defined return values.  Any occurrence of
   \verb+stream_content+, \verb+get_more+, or \verb+page+ in this list
@@ -2672,5 +2676,142 @@ \section{Configuration Examples}
        </server>
 \end{verbatim}
 
+\chapter{WebSocket Protocol Support}
+\label{websockets}
+
+\Yaws\ supports the WebSocket Protocol (RFC 6455), which enables
+two-way communication between clients and web servers. \Yaws\ also
+provides support for working drafts of the WebSocket protocol,
+specifically drafts 10 to 17 of the hybi working group. No support for
+other drafts, such as those from the hixie working group, is provided.
+
+You can find example usage of the WebSocket Protocol in the file
+\verb+www/websockets_example.yaws+. This example, intended for use
+with any browser supporting RFC 6455, returns HTML and JavaScript that
+allow the client to establish a WebSocket connection to the
+server. These connections are handled by the code in
+\verb+www/websockets_example_endpoint.yaws+, which when invoked simply
+establishes \\ \verb+src/basic_echo_callback.erl+ as the WebSocket
+callback module for the connection.
+
+\section{WebSocket Callback Modules}
+
+A WebSocket callback module implements either the
+\verb+handle_message/1+ callback function or the
+\\ \verb+handle_message/2+ callback function, depending on whether
+it's a basic or advanced callback module.
+
+\subsection{Basic Callback Modules}
+
+The argument passed to \verb+handle_message/1+ callback function takes
+one of the following forms:
+
+\begin{itemize}
+
+\item \verb+{text, Text}+ --- the callback receives an unfragmented
+  text message.
+
+\item \verb+{binary, Message}+ --- the callback receives an
+  unfragmented binary message.
+
+\end{itemize}
+
+The \verb+handle_message/1+ callback function supplies one of the
+following as a return value:
+
+\begin{itemize}
+
+\item \verb+noreply+ --- do nothing, just wait for the next message.
+
+\item \verb+{reply, {Type, Data}}+ --- reply to the
+  message. \verb+Type+ must be either \verb+text+ or \verb+binary+ to
+  indicate the type of data in the reply message, and \verb+Data+ is
+  the reply message itself.
+
+\item \verb+{close, Reason}+ --- close the connection and exit the
+  handling process with \verb+Reason+. For a regular non-error close,
+  \verb+Reason+ should be the atom \verb+normal+.
+
+\end{itemize}
+
+To inform \Yaws\ of the details of your callback module, return
+\verb+{websocket, CallbackModule, Options}+ from your \verb+out/1+
+function, where \verb+CallbackModule+ is the name of your callback
+module and \verb+Options+ is a list of options. The following options
+are available:
+
+\begin{itemize}
+
+\item \verb+{callback, CallbackType}+ --- supply this atom to indicate
+  the type of the callback module. \\ \verb+CallbackType+ can be
+  either of the following:
+
+\begin{itemize}
+
+\item \verb+basic+ --- specify this to indicate your callback module
+  is the basic type. This is the default.
+
+\item \verb+{advanced, InitialState}+ --- specify this to indicate
+  your callback module is an advanced callback module. Here,
+  \verb+InitialState+ is the callback's initial state for handling
+  this client. See \ref{advanced_ws} for more details.
+
+\end{itemize}
+
+\item \verb+{origin, Origin}+ --- specify the \verb+Origin+ URL from
+  which messages will be accepted. This is useful for protecting
+  against cross-site attacks. This option defaults to \verb+any+,
+  meaning calls will be accepted from any origin.
+
+\end{itemize}
+
+\subsection{Advanced Callback Modules}
+\label{advanced_ws}
+
+Advanced callback modules---those that want to supply their own
+initial state and are prepared to handle fragmented messages
+themselves---supply a \verb+handle_message/2+ callback function.
+
+To indicate an advanced callback module, include
+\verb+{callback, {advanced, InitialState}}+ in the \verb+Options+ list
+when you return \verb+{websocket, CallbackModule, Options}+ from your
+\verb+out/1+ function, as described above.
+
+The arguments to the \verb+handle_message/2+ callback
+function are as follows:
+
+\begin{itemize}
+
+\item \verb+#ws_frame_info+ --- this record, defined in
+  \verb+include/yaws_api.hrl+, provides all details of a frame
+  section. See section 5 of RFC 6455 for details.
+
+\item \verb+State+ --- this is the callback module's current
+  state. The initial state is supplied when you return
+  \verb+{callback, {advanced, InitialState}}+ as part of the options
+  list you returned from your \verb+out/1+ function to establish the
+  WebSocket callback module.
+
+\end{itemize}
+
+The return values for the \verb+handle_message/2+ callback function
+can be any of the following:
+
+\begin{itemize}
+
+\item \verb+{noreply, State}+ --- do nothing, just wait for the next
+  message. \verb+State+ is the (possibly updated) state for the
+  callback module.
+
+\item \verb+{reply, Reply, State}+ --- reply to the received message
+  with \verb+Reply+, which is either \verb+{text, Data}+ or
+  \verb+{binary, Data}+. \verb+State+ is the (possibly updated) state
+  for the callback module.
+
+\item \verb+{close, Reason}+ --- close the connection and exit the
+  handling process with \verb+Reason+. For a regular non-error close,
+  \verb+Reason+ should be the atom \verb+normal+.
+
+\end{itemize}
 
 \end{document}

include/yaws_api.hrl

@@ -112,6 +112,33 @@
                        %%                 to append to the url
          }).
 
-
-
-
+%% Corresponds to the frame sections as in
+%% http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-08#section-4
+%% plus 'data' and 'ws_state'
+-record(ws_frame_info, {
+	  fin,
+	  rsv,
+	  opcode,
+	  masked,
+	  masking_key,
+	  length,
+	  payload,
+	  data,        %% The unmasked payload. Makes payload redundant.
+	  ws_state     %% The ws_state after unframing this frame.
+                       %% This is useful for the endpoint to know what type of
+                       %% fragment a potentially fragmented message is.
+	 }).
+
+%----------------------------------------------------------------------
+% The state of a WebSocket connection.
+% This is held by the ws owner process and passed in calls to yaws_api.
+%----------------------------------------------------------------------
+-type frag_type() :: text
+		   | binary
+		   | none.  %% The WebSocket is not expecting continuation
+                            %% of any fragmented message.
+-record(ws_state, {
+	  vsn :: integer(),                     % WebSocket version number
+	  sock,                                 % gen_tcp or gen_ssl socket
+	  frag_type :: frag_type()
+	 }).

src/advanced_echo_callback.erl

@@ -0,0 +1,74 @@
+%%%==============================================================
+%%% compiled using erlc -I include src/advanced_echo_callback.erl
+%%%==============================================================
+
+-module(advanced_echo_callback).
+
+-export([handle_message/2]).
+
+-include("yaws_api.hrl").
+
+%% define callback state to accumulate a fragmented WS message
+%% which we echo back when all fragments are in, returning to
+%% initial state.
+-record(state, {frag_type = none,               % fragment type
+                acc = <<>>}).                   % accumulate fragment data
+
+%% start of a fragmented message
+handle_message(#ws_frame_info{fin=0,
+                              opcode=FragType,
+                              data=Data},
+               #state{frag_type=none, acc = <<>>}) ->
+    {noreply, #state{frag_type=FragType, acc=Data}};
+
+%% non-final continuation of a fragmented message
+handle_message(#ws_frame_info{fin=0,
+                              data=Data,
+                              opcode=continuation},
+               #state{frag_type = FragType, acc = Acc}) ->
+    {noreply, #state{frag_type=FragType, acc = <<Acc/binary,Data/binary>>}};
+
+%% end of text fragmented message
+handle_message(#ws_frame_info{fin=1,
+                              opcode=continuation,
+                              data=Data},
+               #state{frag_type=text, acc=Acc}) ->
+    Unfragged = <<Acc/binary, Data/binary>>,
+    {reply, {text, Unfragged}, #state{frag_type=none, acc = <<>>}};
+
+%% one full non-fragmented message
+handle_message(#ws_frame_info{opcode=text, data=Data}, State) ->
+    {reply, {text, Data}, State};
+
+%% end of binary fragmented message
+handle_message(#ws_frame_info{fin=1,
+                              opcode=continuation,
+                              data=Data},
+               #state{frag_type=binary, acc=Acc}) ->
+    Unfragged = <<Acc/binary, Data/binary>>,
+    io:format("echoing back binary message~n",[]),
+    {reply, {binary, Unfragged}, #state{frag_type=none, acc = <<>>}};
+
+%% one full non-fragmented binary message
+handle_message(#ws_frame_info{opcode=binary,
+                              data=Data},
+               State) ->
+    io:format("echoing back binary message~n",[]),
+    {reply, {binary, Data}, State};
+
+handle_message(#ws_frame_info{opcode=ping,
+                              data=Data},
+               State) ->
+    io:format("replying pong to ping~n",[]),
+    {reply, {pong, Data}, State};
+
+handle_message(#ws_frame_info{opcode=pong}, State) ->
+    %% A response to an unsolicited pong frame is not expected.
+    %% http://tools.ietf.org/html/\
+    %%            draft-ietf-hybi-thewebsocketprotocol-08#section-4
+    io:format("ignoring unsolicited pong~n",[]),
+    {noreply, State};
+
+handle_message(#ws_frame_info{}=FrameInfo, State) ->
+    io:format("WS Endpoint Unhandled message: ~p~n~p~n", [FrameInfo, State]),
+    {close, {error, {unhandled_message, FrameInfo}}}.

src/basic_echo_callback.erl

@@ -0,0 +1,36 @@
+%%%===========================================================
+%%% compiled using erlc -I include src/basic_echo_callback.erl
+%%%===========================================================
+
+-module(basic_echo_callback).
+
+%% Export for websocket callbacks
+-export([handle_message/1]).
+
+%% Export for apply
+-export([say_hi/1]).
+
+handle_message({text, <<"bye">>}) ->
+    io:format("User said bye.~n", []),
+    {close, normal};
+
+handle_message({text, <<"something">>}) ->
+    io:format("Some action without a reply~n", []),
+    noreply;
+
+handle_message({text, <<"say hi later">>}) ->
+    io:format("saying hi in 3s.~n", []),
+    timer:apply_after(3000, ?MODULE, say_hi, [self()]),
+    {reply, {text, <<"I'll say hi in a bit...">>}};
+
+handle_message({text, Message}) ->
+    io:format("basic echo handler got ~p~n", [Message]),
+    {reply, {text, <<Message/binary>>}};
+
+handle_message({binary, Message}) ->
+    {reply, {binary, Message}}.
+
+
+say_hi(Pid) ->
+    io:format("asynchronous greeting~n", []),
+    yaws_api:websocket_send(Pid, {text, <<"hi there!">>}).

src/yaws_api.erl

@@ -37,8 +37,7 @@
          stream_chunk_end/1]).
 -export([stream_process_deliver/2, stream_process_deliver_chunk/2,
          stream_process_deliver_final_chunk/2, stream_process_end/2]).
--export([websocket_send/2, websocket_receive/1,
-         websocket_unframe_data/1, websocket_setopts/2]).
+-export([websocket_send/2]).
 -export([new_cookie_session/1, new_cookie_session/2, new_cookie_session/3,
          cookieval_to_opaque/1, request_url/1,
          print_cookie_sessions/0,
@@ -995,37 +994,9 @@ stream_process_end(Sock, YawsPid) ->
     YawsPid ! endofstreamcontent.
 
 
-websocket_send(Socket, IoList) ->
-    DataFrame = [0, IoList, 255],
-    case Socket of
-	{sslsocket,_,_} ->
-	    ssl:send(Socket, DataFrame);
-	_ ->
-	    gen_tcp:send(Socket, DataFrame)
-    end.
-
-websocket_receive(Socket) ->
-    R = case Socket of
-	    {sslsocket,_,_} ->
-		ssl:recv(Socket, 0);
-	    _ ->
-		gen_tcp:recv(Socket, 0)
-	end,
-    case R of
-	{ok, DataFrames} ->
-	    ReceivedMsgs = yaws_websockets:unframe_all(DataFrames, []),
-	    {ok, ReceivedMsgs};
-	_ -> R
-    end.
-
-websocket_unframe_data(DataFrameBin) ->
-    {ok, Msg, <<>>} = yaws_websockets:unframe_one(DataFrameBin),
-    Msg.
-
-websocket_setopts({sslsocket,_,_}=Socket, Opts) ->
-    ssl:setopts(Socket, Opts);
-websocket_setopts(Socket, Opts) ->
-    inet:setopts(Socket, Opts).
+%% Pid must the the process in control of the websocket connection.
+websocket_send(Pid, {Type, Data}) ->
+    yaws_websockets:send(Pid, {Type, Data}).
 
 
 %% Return new cookie string

src/yaws_server.erl

@@ -2677,10 +2677,10 @@ deliver_dyn_part(CliSock,                       % essential params
             Priv = deliver_accumulated(Arg, CliSock,
                                        no, undefined, stream),
             wait_for_streamcontent_pid(Priv, CliSock, Pid);
-        {websocket, OwnerPid, SocketMode} ->
+        {websocket, CallbackMod, Opts} ->
             %% The handshake passes control over the socket to OwnerPid
             %% and terminates the Yaws worker!
-            yaws_websockets:handshake(Arg, OwnerPid, SocketMode);
+            yaws_websockets:start(Arg, CallbackMod, Opts);
         _ ->
             DeliverCont(Arg)
     end.
@@ -3123,7 +3123,7 @@ handle_out_reply({streamcontent_from_pid, MimeType, Pid},
     yaws:outh_set_content_type(MimeType),
     {streamcontent_from_pid, MimeType, Pid};
 
-handle_out_reply({websocket, _OwnerPid, _SocketMode}=Reply,
+handle_out_reply({websocket, _CallbackMod, _Opts}=Reply,
                  _LineNo,_YawsFile, _UT, _ARG) ->
     yaws:accumulate_header({connection, erase}),
     Reply;