Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Websocket open callback. #99

Closed
wants to merge 5 commits into from

8 participants

@francois2metz

We can now pass some parameters when upgrading to websockets.

{websockets, CallbackMode, Opts, Params}

When the connection is upgraded, the websocket module now call:

CallbackMode:handle_message(open, Params)

Which allows you to keep some parameters from the initial request.
On the open callback, you can also send date over the connection.

handle_message(open, _Params) ->
    yaws_api:websocket_send(self(), {text, <<"Hello world">>}).

TODO:

  • handle compatibility with the previous implementation
  • should we handle return value? Like {close, Reason}?
  • update doc
  • add tests
  • gather feedback

It's just to start a discussion about #98.

francois2metz added some commits
@francois2metz francois2metz Websocket open callback.
We can now pass some parameters when upgrading to websockets.

    {websockets, CallbackMode, Opts, Params}

When the connection is upgraded, the websocket module now call:

    CallbackMode:handle_message(open, Params)

Which allows you to keep some parameters from the initial request.
On the open callback, you can also send date over the connection.

    handle_message(open, _Params) ->
        yaws_api:websocket_send(self(), {text, <<"Hello world">>}).

TODO:

  * handle compatibility with the previous implementation
  * should we handle return value? Like {close, Reason}?
  * update doc
  * add tests
  * gather feedback

Signed-off-by: François de Metz <francois@stormz.me>
b98c9ed
@francois2metz francois2metz Fix the unused param.
Signed-off-by: François de Metz <francois@stormz.me>
7472eec
@jbothma

What sort of data do you plan on passing along in Param? On another topic, I thought passing Arg could be useful for some backend, but we might waste time adding stuff we don't need yet.

CallbackModule:handle_message(open, Params)

right before entering the loop for the first time. I think it makes little difference since any send() would just queue a message for the next loop, but that's where I'd expect to find it.

francois2metz added some commits
@francois2metz francois2metz Rename handle_message/2 by handle_open/1.
Signed-off-by: François de Metz <francois@stormz.me>
8c31b07
@francois2metz francois2metz Add doc.
Signed-off-by: François de Metz <francois@stormz.me>
2a91dd1
@francois2metz

I renamed the open callback and updated the doc.

We probably take care of users who doesn't have defined
handle_open callback.

@jbothma

cool.

I like the idea of supporting modules without the open callback defined, but I don't know enough about callback APIs to know whether it's unwise. Anyone?

Two options I can think of are
1. checking if the module has the function - the cost doesn't matter since it's once per connection for the open callback
2. catch undef, rethrow other exceptions - not recommended I think

We should also keep the advanced callback in mind, but I don't see any need for it to be any different here.

@francois2metz

you can look on webmachine and cowboy rest implementation. they borgne check if some functions are exported.

it could also be a great improvement to handle simple and complex messages in the websocket module.

@jbothma

What do you mean with complex messages ?

@francois2metz francois2metz Test is handle_open is defined in the CallbackMod.
Signed-off-by: François de Metz <francois@stormz.me>
cde7af2
@francois2metz

Finally the handle_open callback is now optional.

We should now update handle_message to be able to pass params/state.

@f3r3nc

Is this pull planned to be merged?

@ethertricity

My use case is:

  1. The URL used to open the websocket contains pathinfo that I need to associate with the websocket.
  2. I wish to send a message to the client before the client sends its first message to me.

A good approach might be:

  1. Add an option {user, State} to the Options in the return value {websocket, Module, Options}
  2. Add both the Pid and the State arg to all callbacks
    • means no need to use process dictionary or ETS table keyed on pid)
  3. Disambiguate open, message and close callbacks:
    • handle_open(Pid, State)
    • handle_message(Pid, Message, State)
    • handle_close(Pid, State)
  4. Add a NewState element to the callback return tuples:
    • {noreply, NewState} (for both handle_open/2 and handle_message/3
    • {reply, Message, NewState}
    • {close, Reason}

Great software. Congratulations.

@aladhami5

when are you going to add Websocket open callback support to the main code???

@aladhami5 aladhami5 commented on the diff
src/yaws_websockets.erl
((5 lines not shown))
Handshake = handshake(ProtocolVersion, Arg, CliSock,
WebSocketLocation, Origin, Protocol),
gen_tcp:send(CliSock, Handshake), % TODO: use the yaws way of
% supporting normal
% and ssl sockets
{callback, CallbackType} = lists:keyfind(callback, 1, Opts),
+
+ case erlang:function_exported(CallbackMod, handle_open, 1) of

please make sure that the CallbackMod module is loaded before calling erlang:function_exported

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@capflam
Collaborator

Hi,

The websocket module was refactored so this request cannot be merged without some changes, but it could be done, it is not a big deal. Nevertheless I see an annoying problem, it breaks the compatibility with the actual version and I think it is not necessary.

To upgrade an HTTP connection to a websocket connection, an appmod (or a Yaws page) must return:

{websocket, CallbackMod, Options}

where Options is a proplist. So to pass some parameters to CallbackMod, we could add it in this proplist. Here is an example:

out(A) ->
    CallbackMod = basic_echo_callback,
    Params = ..., %% <-- Here Params can be anything
    Opts = [{origin, "http://" ++ (A#arg.headers)#headers.host}, {params, Params}],
    {websocket, CallbackMod, Opts}.

This way to do is fully compatible with actual version.

Next, the idea to pass some parameters to the callback module when the connection is upgraded to websocket could be extended. We could support other optional callback functions:

  • Callback:init(Params) -> {ok, CallbackInfo} | {error, Reason} :
    this function is called when the websocket process is started. CallbackInfo can be an arbitrary term. It is an opaque state.

  • Callback:terminate(Reason, CallbackInfo) -> ok :
    this function is called when the websocket process is stopped.

  • Callback:handle_open(CallbackInfo) -> {ok, NewCallbackInfo} | {error, Reason} :
    this function is called when the connection is upgraded.

  • Callback:handle_message(Msg, CallbackInfo) -> {reply, {Type, Data}, NewCallbackInfo} | {noreply, NewCallbackInfo} | close, Reason}
    this function is called when a message is received, if Callback is a basic callback module. If this function does not exist, we use the default function (and mandatory).

  • Callback:handle_message(Msg, CallbackState, CallbackInfo) -> {reply, {Type, Data}, NewCallbackState, NewCallbackInfo} | {noreply, NewCallbackState, NewCallbackInfo} | {close, Reason}
    this function is called when a message is received, if Callback is an advanced callback module. If this function does not exist, we use the default function (and mandatory).

I started to work on it, so every comments are welcome.

@pvieytes

Hi @capflam,
are you coding it? I'm very interested. If you are not doing it and you don't mind I could do it.

@capflam
Collaborator

Hi @pvieytes,

Yes, I will push my work in a branch later today (without testsuite neither documentation yet). I have rewritten an important part of yaws_websocket.erl to fix some bugs (and to add optional callback functions of course). So feedback will be greatly appreciated.

@pvieytes

ok,
it sounds pretty good.

@capflam
Collaborator

Ok, I pushed my changes in the branch websockets. I still have to write a proper documentation and an exhaustive testsuite. But, the commit log (which is quite long :) gives all needed information.
Because it is a big patch, all feedback and comments are welcome.

@pvieytes

Hi,
I also need a callback for direct messages to the process (handle_info). I don't know if you didn't add this functionality on purpose.

I want to monitor others processes so I need to accept messages like
{'DOWN', MonitorRef, Type, Object, Info}, and sometimes, I just want to change the callback state.

I made a pull request to your branch:

#135

@capflam
Collaborator

Hi,

Thanks ! I merged it and also homogenized the return values of the callback functions. Look my last commit for details.

@pvieytes

thanks.

@capflam
Collaborator

I pushed some changes. Apart from some bugfixes, I added the support of outgoing fragmented messages (commit: 38cb4ad).

@vinoski
Collaborator

I ran the websocket autobahn testsuite on this branch and there are a bunch of new failures. Christopher, I'll send you the failure info by email.

@pvieytes

Hi,
are the autobahn failures fixed? If you want to, I could try to help.

@vinoski
Collaborator

The current HEAD of yaws master shows 3 autobahn fuzzingclient failures, all related to UTF-8 handling, and these failures are quite old. The new failures I mentioned in my comment above are no longer present.

@capflam
Collaborator

Hi,

I was quite busy these last days, but my work on the websockets is almost finished. I just need to do a review of my changes. I'll try to work on it this week.

@capflam
Collaborator

A new version of the WebSockets module was pushed. It integrates the optional callback function handle_open/2. Feel free to test it. Let me know if you have any questions.

Thanks for your contribution !

@capflam capflam closed this
@capflam capflam referenced this pull request
Closed

Websocket PID #98

@andredupreez andredupreez referenced this pull request from a commit in patternmatched/yaws
Christopher Faulet Refactor WebSockets and add support of optional callback functions
Main changes:
  * Fix some bugs about UTF-8 encoding and messages fragmentation
  * Add support of optional callback functions
  * Add support of many startup options
  * Add support of outgoing fragmented messages
  * Add a websocket testsuite

                                 - * -
*** bug fixes ***

First of all, an huge part of yaws_websocket.erl was rewritten to fix bugs
about the messages fragmentation and the UTF-8 encoding of incoming text
messages:

  * UTF-8 encoding
    before, when a text message was fragmented, only the first frame was
    checked and partial UTF-8 sequences were not supported. Now, checks
    are done on each message part and a partial UTF-8 sequence at the end
    of a frame is accumulated and checked with the next frame (for basic
    callback only).

  * Messages fragmentation
    for basic callback modules, because of a buggy mapping between frames
    and messages, the messages fragmentation was almost unusable. To fix
    this, the message handling was rewritten.

Now, all tests in the autobahn testsuite[1] pass successfully.

                                 - * -
*** Optional callback functions ***

Then, from an idea of François de Metz[2], yaws_websocket module was
extended to support optional callback functions. See the documentation for
details (www/websockets.yaws).

Quickly, optional callback functions are:

  * Module:init/1           (for basic and advanced callback modules)
  * Module:terminate/2      (for basic and advanced callback modules)
  * Module:handle_open/2    (for basic and advanced callback modules)
  * Module:handle_info/2    (for basic and advanced callback modules)
  * Module:handle_message/2 (for basic callback modules only, used in place
                             of Module:handle_message/1)

Thanks to Pablo Vieytes[3] which added handle_info to optional callback
functions.
                                 - * -
*** Startup options ***

To start a websocket process a script must return the following term from
its out/1 function:

  {websocket, CallbackMod, Options}

where 'Options' is a (possibly empty) proplist. Following parameters are
supported:

  * {origin, Orig}
  * {callback, Type}
  * {keepalive, Boolean}
  * {keepalive_timeout, Tout}
  * {keepalive_grace_period, Time}
  * {drop_on_timeout, Boolean}
  * {close_timeout, Tout}
  * {close_if_unmasked, Boolean}
  * {max_frame_size, Int}
  * {max_message_size, Int}
  * {auto_fragment_message, Boolean}
  * {auto_fragment_threshold, Int}

See the documentation for details (www/websockets.yaws).

                                 - * -
*** Outgoing fragmented messages ***

A callback module can now send fragmented messages to clients using the
record #ws_frame{}:

 #ws_frame{fin     = true,  %% true | false
           rsv     = 0,
           opcode,          %% text | binary | continuation...
           payload = <<>>}. %% binary(), unmasked data

--
[1] http://autobahn.ws/testsuite
[2] klacke#99
[3] https://github.com/pvieytes
29a7989
@jgrinstead jgrinstead referenced this pull request from a commit in jgrinstead/yaws
Christopher Faulet Refactor WebSockets and add support of optional callback functions
Main changes:
  * Fix some bugs about UTF-8 encoding and messages fragmentation
  * Add support of optional callback functions
  * Add support of many startup options
  * Add support of outgoing fragmented messages
  * Add a websocket testsuite

                                 - * -
*** bug fixes ***

First of all, an huge part of yaws_websocket.erl was rewritten to fix bugs
about the messages fragmentation and the UTF-8 encoding of incoming text
messages:

  * UTF-8 encoding
    before, when a text message was fragmented, only the first frame was
    checked and partial UTF-8 sequences were not supported. Now, checks
    are done on each message part and a partial UTF-8 sequence at the end
    of a frame is accumulated and checked with the next frame (for basic
    callback only).

  * Messages fragmentation
    for basic callback modules, because of a buggy mapping between frames
    and messages, the messages fragmentation was almost unusable. To fix
    this, the message handling was rewritten.

Now, all tests in the autobahn testsuite[1] pass successfully.

                                 - * -
*** Optional callback functions ***

Then, from an idea of François de Metz[2], yaws_websocket module was
extended to support optional callback functions. See the documentation for
details (www/websockets.yaws).

Quickly, optional callback functions are:

  * Module:init/1           (for basic and advanced callback modules)
  * Module:terminate/2      (for basic and advanced callback modules)
  * Module:handle_open/2    (for basic and advanced callback modules)
  * Module:handle_info/2    (for basic and advanced callback modules)
  * Module:handle_message/2 (for basic callback modules only, used in place
                             of Module:handle_message/1)

Thanks to Pablo Vieytes[3] which added handle_info to optional callback
functions.
                                 - * -
*** Startup options ***

To start a websocket process a script must return the following term from
its out/1 function:

  {websocket, CallbackMod, Options}

where 'Options' is a (possibly empty) proplist. Following parameters are
supported:

  * {origin, Orig}
  * {callback, Type}
  * {keepalive, Boolean}
  * {keepalive_timeout, Tout}
  * {keepalive_grace_period, Time}
  * {drop_on_timeout, Boolean}
  * {close_timeout, Tout}
  * {close_if_unmasked, Boolean}
  * {max_frame_size, Int}
  * {max_message_size, Int}
  * {auto_fragment_message, Boolean}
  * {auto_fragment_threshold, Int}

See the documentation for details (www/websockets.yaws).

                                 - * -
*** Outgoing fragmented messages ***

A callback module can now send fragmented messages to clients using the
record #ws_frame{}:

 #ws_frame{fin     = true,  %% true | false
           rsv     = 0,
           opcode,          %% text | binary | continuation...
           payload = <<>>}. %% binary(), unmasked data

--
[1] http://autobahn.ws/testsuite
[2] klacke#99
[3] https://github.com/pvieytes
0958ce1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 16, 2012
  1. @francois2metz

    Websocket open callback.

    francois2metz authored
    We can now pass some parameters when upgrading to websockets.
    
        {websockets, CallbackMode, Opts, Params}
    
    When the connection is upgraded, the websocket module now call:
    
        CallbackMode:handle_message(open, Params)
    
    Which allows you to keep some parameters from the initial request.
    On the open callback, you can also send date over the connection.
    
        handle_message(open, _Params) ->
            yaws_api:websocket_send(self(), {text, <<"Hello world">>}).
    
    TODO:
    
      * handle compatibility with the previous implementation
      * should we handle return value? Like {close, Reason}?
      * update doc
      * add tests
      * gather feedback
    
    Signed-off-by: François de Metz <francois@stormz.me>
  2. @francois2metz

    Fix the unused param.

    francois2metz authored
    Signed-off-by: François de Metz <francois@stormz.me>
  3. @francois2metz

    Rename handle_message/2 by handle_open/1.

    francois2metz authored
    Signed-off-by: François de Metz <francois@stormz.me>
  4. @francois2metz

    Add doc.

    francois2metz authored
    Signed-off-by: François de Metz <francois@stormz.me>
Commits on Mar 21, 2012
  1. @francois2metz

    Test is handle_open is defined in the CallbackMod.

    francois2metz authored
    Signed-off-by: François de Metz <francois@stormz.me>
This page is out of date. Refresh to see the latest.
View
10 doc/yaws.tex
@@ -2738,6 +2738,9 @@ \section{WebSocket Callback Modules}
\\ \verb+handle_message/2+ callback function, depending on whether
it's a basic or advanced callback module.
+To be notified when the connection have been upgraded, you can implement,
+ \verb+handle_open/1+ callback.
+
\subsection{Basic Callback Modules}
The argument passed to \verb+handle_message/1+ callback function takes
@@ -2784,9 +2787,10 @@ \subsection{Basic Callback Modules}
\end{itemize}
To inform \Yaws\ of the details of your callback module, return
-\verb+{websocket, CallbackModule, Options}+ from your \verb+out/1+
+\verb+{websocket, CallbackModule, Options, Params}+ from your \verb+out/1+
function, where \verb+CallbackModule+ is the name of your callback
-module and \verb+Options+ is a list of options. The following options
+module, \verb+Options+ is a list of options and \verb+Params+ is an arbitrary
+param to be passed to \verb+handle_open/1+. The following options
are available:
\begin{itemize}
@@ -2823,7 +2827,7 @@ \subsection{Advanced Callback Modules}
To indicate an advanced callback module, include
\verb+{callback, {advanced, InitialState}}+ in the \\ \verb+Options+
-list when you return \verb+{websocket, CallbackModule, Options}+ from
+list when you return \verb+{websocket, CallbackModule, Options, Params}+ from
your \verb+out/1+ function, as described above.
The arguments to the \verb+handle_message/2+ callback
View
6 examples/src/basic_echo_callback.erl
@@ -5,11 +5,15 @@
-module(basic_echo_callback).
%% Export for websocket callbacks
--export([handle_message/1]).
+-export([handle_open/1, handle_message/1]).
%% Export for apply
-export([say_hi/1]).
+handle_open(_Params) ->
+ io:format("The connection was opened.~n", []),
+ noreply.
+
handle_message({text, <<"bye">>}) ->
io:format("User said bye.~n", []),
{close, normal};
View
6 src/yaws_server.erl
@@ -2655,10 +2655,10 @@ deliver_dyn_part(CliSock, % essential params
{streamcontent_from_pid, _, Pid} ->
Priv = deliver_accumulated(Arg, CliSock, no, undefined, stream),
wait_for_streamcontent_pid(Priv, CliSock, Pid);
- {websocket, CallbackMod, Opts} ->
+ {websocket, CallbackMod, Opts, Params} ->
%% The handshake passes control over the socket to OwnerPid
%% and terminates the Yaws worker!
- yaws_websockets:start(Arg, CallbackMod, Opts);
+ yaws_websockets:start(Arg, CallbackMod, Opts, Params);
_ ->
DeliverCont(Arg)
end.
@@ -3111,7 +3111,7 @@ handle_out_reply({streamcontent_from_pid, MimeType, Pid},
yaws:outh_set_content_type(MimeType),
{streamcontent_from_pid, MimeType, Pid};
-handle_out_reply({websocket, _CallbackMod, _Opts}=Reply,
+handle_out_reply({websocket, _CallbackMod, _Opts, _Params}=Reply,
_LineNo,_YawsFile, _UT, _ARG) ->
yaws:accumulate_header({connection, erase}),
Reply;
View
23 src/yaws_websockets.erl
@@ -18,17 +18,17 @@
-define(MAX_PAYLOAD, 16777216). %16MB
%% API
--export([start/3, send/2]).
+-export([start/4, send/2]).
%% Exported for spawn
--export([receive_control/4]).
+-export([receive_control/5]).
-start(Arg, CallbackMod, Opts) ->
+start(Arg, CallbackMod, Opts, Params) ->
SC = get(sc),
CliSock = Arg#arg.clisock,
PrepdOpts = preprocess_opts(Opts),
OwnerPid = spawn(?MODULE, receive_control,
- [Arg, SC, CallbackMod, PrepdOpts]),
+ [Arg, SC, CallbackMod, PrepdOpts, Params]),
CliSock = Arg#arg.clisock,
case SC#sconf.ssl of
undefined ->
@@ -73,15 +73,15 @@ preprocess_opts(GivenOpts) ->
{callback, basic}],
lists:foldl(Fun, GivenOpts, Defaults).
-receive_control(Arg, SC, CallbackMod, Opts) ->
+receive_control(Arg, SC, CallbackMod, Opts, Params) ->
receive
ok ->
- handshake(Arg, SC, CallbackMod, Opts);
+ handshake(Arg, SC, CallbackMod, Opts, Params);
{error, Reason} ->
exit(Reason)
end.
-handshake(Arg, SC, CallbackMod, Opts) ->
+handshake(Arg, SC, CallbackMod, Opts, Params) ->
CliSock = Arg#arg.clisock,
OriginOpt = lists:keyfind(origin, 1, Opts),
Origin = get_origin_header(Arg#arg.headers),
@@ -100,13 +100,20 @@ handshake(Arg, SC, CallbackMod, Opts) ->
undefined -> "ws://" ++ Host ++ Path;
_ -> "wss://" ++ Host ++ Path
end,
-
Handshake = handshake(ProtocolVersion, Arg, CliSock,
WebSocketLocation, Origin, Protocol),
gen_tcp:send(CliSock, Handshake), % TODO: use the yaws way of
% supporting normal
% and ssl sockets
{callback, CallbackType} = lists:keyfind(callback, 1, Opts),
+
+ case erlang:function_exported(CallbackMod, handle_open, 1) of

please make sure that the CallbackMod module is loaded before calling erlang:function_exported

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ true ->
+ CallbackMod:handle_open(Params);
+ false ->
+ ok
+ end,
+
WSState = #ws_state{sock = CliSock,
vsn = ProtocolVersion,
frag_type = none},
View
2  www/websockets_autobahn_endpoint.yaws
@@ -3,5 +3,5 @@ out(A) ->
CallbackMod = advanced_echo_callback,
InitialState = {state, none, <<>>},
Opts = [{callback, {advanced, InitialState}}],
- {websocket, CallbackMod, Opts}.
+ {websocket, CallbackMod, Opts, []}.
</erl>
View
2  www/websockets_example_endpoint.yaws
@@ -2,5 +2,5 @@
out(A) ->
CallbackMod = basic_echo_callback,
Opts = [{origin, "http://" ++ (A#arg.headers)#headers.host}],
- {websocket, CallbackMod, Opts}.
+ {websocket, CallbackMod, Opts, []}.
</erl>
Something went wrong with that request. Please try again.