Skip to content
Browse files

add reverse proxy intercept module capability

Users can now specify an interception module that can rewrite requests and
responses as they pass through the Yaws reverse proxy. See the
documentation for details (yaws.conf.5 man page or the yaws.pdf file).

Also add new set_header, get_header, and delete_header functions to the
yaws_api module to allow intercept modules and arg rewriters to more easily
examine and modify #headers{} records.

Add new tests for the new header manipulation functions and for the reverse
proxy interception feature.
  • Loading branch information...
1 parent 92a1a27 commit cccc5789535a43bcd0bf6b40033926fb1644d1ba @vinoski vinoski committed
View
67 doc/yaws.tex
@@ -1218,6 +1218,18 @@ \subsection{Arg rewrite}
argument so that the login page can redirect the user to the original
page once the login procedure is finished.
+Within an arg rewrite function, examining and modifying HTTP headers can be
+achieved using the following functions from the \verb+yaws_api+ module:
+
+\begin{itemize}
+\item \verb+set_header/2+, \verb+set_header/3+
+\item \verb+get_header/2+, \verb+get_header/3+
+\item \verb+delete_header/2+
+\end{itemize}
+
+All functions operate on instances of the \verb+#headers{}+ record type
+defined in the \verb+yaws_api.hrl+ include file.
+
\subsection{Authenticating}
Now we're approaching the \verb+login.yaws+ page, the page that
@@ -2697,17 +2709,64 @@ \section{Server Part}
specific configuration data, see the explanation for the
\verb+<opaque>+ context.
-\item \verb+revproxy = Prefix Url+ ---
+\item \verb+revproxy = Prefix Url [intercept_mod Module]+ ---
Make \Yaws\ a reverse proxy. \verb+Prefix+ is a path inside our own
docroot and \verb+Url+ is a URL pointing to a website we
- want to "mount" under the path which is \verb+Prefix+.
+ want to "mount" under the \verb+Prefix+ path. For example:
- Example: \verb+revproxy = /tmp/foo http://yaws.hyber.org+
+ \verb+revproxy = /tmp/foo http://yaws.hyber.org+
This makes the hyber website appear under \verb+/tmp/foo+.
It is possible to have multiple reverse proxies inside the same
- server.
+ server by supplying multiple \verb+revproxy+ directives.
+
+ If the optional keyword \verb+intercept_mod+ and
+ \verb+Module+ are supplied in the \verb+revproxy+ directive,
+ then \verb+Module+ indicates a proxy request and response
+ interception module. When the reverse proxy receives a
+ request from a client, it will first pass
+ \verb+#http_request{}+ and \verb+#headers{}+ records
+ representing the client request to the intercept module's
+ \verb+rewrite_request/2+ function. The function must return a
+ 3-tuple of the following form:
+
+\begin{verbatim}
+{ok, #http_request{}, #headers{}}
+\end{verbatim}
+
+ where the record instances can be the original values passed
+ in or new values. The reverse proxy will use these record
+ instances when passing the request to the backend server.
+
+ Similarly, when the backend server returns a response, the
+ reverse proxy will call the intercept module's
+ \verb+rewrite_response/2+ function, passing it an
+ \verb+#http_response{}+ record and a \verb+#headers{}+
+ record. The function must return a 3-tuple of the following
+ form:
+
+\begin{verbatim}
+{ok, #http_response{}, #headers{}}
+\end{verbatim}
+
+ where the record instances can be the original values passed
+ in or new values. The reverse proxy will use these record
+ instances when returning the response to the client.
+
+ Any failure within an intercept module function results in
+ HTTP status code 500 (Internal Server Error) being returned
+ to the client.
+
+ Intercept modules can use the following functions from the
+ \verb+yaws_api+ module to retrieve, set, or delete HTTP
+ headers from \verb+#headers{}+ records:
+
+\begin{itemize}
+\item \verb+set_header/2+, \verb+set_header/3+
+\item \verb+get_header/2+, \verb+get_header/3+
+\item \verb+delete_header/2+
+\end{itemize}
\item \verb+fwdproxy = true|false+ ---
Make \Yaws\ a forward proxy. By enabling this option you can use
View
7 include/yaws.hrl
@@ -340,7 +340,12 @@
}).
-
+%% forward and reverse proxy config info
+-record(proxy_cfg, {
+ prefix,
+ url,
+ intercept_mod
+ }).
View
62 man/yaws.conf.5
@@ -150,7 +150,7 @@ of 0 effectively disables the process pool.
\fBlog_wrap_size = Integer\fR
The logs written by Yaws are all wrap logs, the default value at the size where
they wrap around and the original gets renamed to File.old is \fI1000000\fR, 1
-megabyte. This value can changed.
+megabyte. This value can be changed.
.br
If we set the value to 0 the logs will never wrap. If we want to use Yaws in
combination with a more traditional log wrapper such as logrotate, set the size
@@ -667,18 +667,62 @@ synchronize the startup with the Yaws server as well as getting hold of user
specific configuration data, see the explanation for the <opaque> context.
.TP
-\fBrevproxy = Prefix Url\fR
-Make Yaws a reverse proxy. The Prefix is a path inside our own docroot and the
-Url argument is an url pointing to a website we want to "mount" under the path
-which is Prefix.
+\fBrevproxy = Prefix Url [intercept_mod Module]\fR
+Make Yaws a reverse proxy. \fIPrefix\fR is a path inside our own docroot
+and \fIUrl\fB argument is a URL pointing to a website we want to "mount"
+under the \fIPrefix\fR path. This example:
-Example: revproxy = /tmp/foo http://yaws.hyber.org
+.nf
+ revproxy = /tmp/foo http://yaws.hyber.org
+.fi
-Makes the hyber website appear under /tmp/foo
+makes the hyber website appear under \fI/tmp/foo\fR.
It is possible to have multiple reverse proxies inside the same server.
-WARNING, this feature is yet not in production quality.
+You can optionally configure an interception module for each reverse proxy,
+allowing your application to examine and modify requests and HTTP headers
+as they pass through the proxy from client to backend server and also
+examine and modify responses and HTTP headers as they return from the
+backend server through the proxy to the client.
+
+You specify an interception module by including the optional
+\fIintercept_mod\fR keyword followed by \fIModule\fR, which should be the
+name of your interception module.
+
+An interception module is expected to export two functions:
+\fIrewrite_request/2\fR and \fIrewrite_response/2\fR. The two arguments
+passed to \fIrewrite_request/2\fR function are a \fI#http_request{}\fR record
+and a \fI#headers{}\fR record, whereas \fIrewrite_response/2\fR function
+takes a \fI#http_response{}\fR record and also a \fI#headers{}\fR record. You
+can find definitions for these record types in the \fIyaws_api.hrl\fR
+header file. Each function can examine each record instance and can either
+return each original instance or can return a modified copy of each
+instance in its response. The \fIrewrite_request/2\fR function should
+return a tuple of the following form:
+
+.nf
+ \fI{ok, #http_request{}, #headers{}}\fR
+.fi
+
+and the \fIrewrite_response/2\fR function should similarly return a tuple
+of the following form:
+
+.nf
+ \fI{ok, #http_response{}, #headers{}}\fR
+.fi
+
+A \fI#headers{}\fR record can easily be manipulated in an interceptor using
+the functions listed below:
+
+.nf
+ \fIyaws_api:set_header/2\fR, \fIyaws_api:set_header/3\fR
+ \fIyaws_api:get_header/2\fR, \fIyaws_api:get_header/3\fR
+ \fIyaws_api:delete_header/2\fR
+.fi
+
+Any failures in your interception module's functions will result in HTTP
+status code 500, indicating an internal server error.
.TP
\fBfwdproxy = true|false\fR
@@ -686,8 +730,6 @@ Make Yaws a forward proxy. By enabling this option you can use Yaws as a proxy
for outgoing web traffic, typically by configuring the proxy settings in a
web-browser to explicitly target Yaws as its proxy server.
-WARNING, this feature is yet not in production quality.
-
.TP
\fBservername = Name\fR
If we're virthosting everal servers and want to force a server to match specific
View
360 src/yaws_api.erl
@@ -69,63 +69,28 @@
-export([binding/1,binding_exists/1,
dir_listing/1, dir_listing/2, redirect_self/1]).
--export([arg_clisock/1
- , arg_client_ip_port/1
- , arg_headers/1
- , arg_req/1
- , arg_clidata/1
- , arg_server_path/1
- , arg_querydata/1
- , arg_appmoddata/1
- , arg_docroot/1
- , arg_docroot_mount/1
- , arg_fullpath/1
- , arg_cont/1
- , arg_state/1
- , arg_pid/1
- , arg_opaque/1
- , arg_appmod_prepath/1
- , arg_prepath/1
- , arg_pathinfo/1
-
- , http_request_method/1
- , http_request_path/1
- , http_request_version/1
-
- , http_response_version/1
- , http_response_status/1
- , http_response_phrase/1
-
- , headers_connection/1
- , headers_accept/1
- , headers_host/1
- , headers_if_modified_since/1
- , headers_if_match/1
- , headers_if_none_match/1
- , headers_if_range/1
- , headers_if_unmodified_since/1
- , headers_range/1
- , headers_referer/1
- , headers_user_agent/1
- , headers_accept_ranges/1
- , headers_cookie/1
- , headers_keep_alive/1
- , headers_location/1
- , headers_content_length/1
- , headers_content_type/1
- , headers_content_encoding/1
- , headers_authorization/1
- , headers_transfer_encoding/1
- , headers_x_forwarded_for/1
- , headers_other/1
-
- ]).
-
+-export([arg_clisock/1, arg_client_ip_port/1, arg_headers/1, arg_req/1,
+ arg_clidata/1, arg_server_path/1, arg_querydata/1, arg_appmoddata/1,
+ arg_docroot/1, arg_docroot_mount/1, arg_fullpath/1, arg_cont/1,
+ arg_state/1, arg_pid/1, arg_opaque/1, arg_appmod_prepath/1, arg_prepath/1,
+ arg_pathinfo/1]).
+-export([http_request_method/1, http_request_path/1, http_request_version/1,
+ http_response_version/1, http_response_status/1, http_response_phrase/1,
+ headers_connection/1, headers_accept/1, headers_host/1,
+ headers_if_modified_since/1, headers_if_match/1, headers_if_none_match/1,
+ headers_if_range/1, headers_if_unmodified_since/1, headers_range/1,
+ headers_referer/1, headers_user_agent/1, headers_accept_ranges/1,
+ headers_cookie/1, headers_keep_alive/1, headers_location/1,
+ headers_content_length/1, headers_content_type/1,
+ headers_content_encoding/1, headers_authorization/1,
+ headers_transfer_encoding/1, headers_x_forwarded_for/1, headers_other/1]).
+
+-export([set_header/2, set_header/3, get_header/2, get_header/3, delete_header/2]).
-import(lists, [flatten/1, reverse/1]).
-%% these are a bunch of function that are useful inside
-%% yaws scripts
+%% These are a bunch of accessor functions that are useful inside
+%% yaws scripts.
arg_clisock(#arg{clisock = X}) -> X.
arg_client_ip_port(#arg{client_ip_port = X}) -> X.
@@ -1178,6 +1143,293 @@ reformat_header(H) ->
end, H#headers.other).
+set_header(#headers{}=Hdrs, {Header, Value}) ->
+ set_header(Hdrs, Header, Value).
+
+set_header(#headers{}=Hdrs, connection, Value) ->
+ Hdrs#headers{connection = Value};
+set_header(#headers{}=Hdrs, {lower, "connection"}, Value) ->
+ Hdrs#headers{connection = Value};
+set_header(#headers{}=Hdrs, accept, Value) ->
+ Hdrs#headers{accept = Value};
+set_header(#headers{}=Hdrs, {lower, "accept"}, Value) ->
+ Hdrs#headers{accept = Value};
+set_header(#headers{}=Hdrs, host, Value) ->
+ Hdrs#headers{host = Value};
+set_header(#headers{}=Hdrs, {lower, "host"}, Value) ->
+ Hdrs#headers{host = Value};
+set_header(#headers{}=Hdrs, if_modified_since, Value) ->
+ Hdrs#headers{if_modified_since = Value};
+set_header(#headers{}=Hdrs, {lower, "if-modified-since"}, Value) ->
+ Hdrs#headers{if_modified_since = Value};
+set_header(#headers{}=Hdrs, if_match, Value) ->
+ Hdrs#headers{if_match = Value};
+set_header(#headers{}=Hdrs, {lower, "if-match"}, Value) ->
+ Hdrs#headers{if_match = Value};
+set_header(#headers{}=Hdrs, if_none_match, Value) ->
+ Hdrs#headers{if_none_match = Value};
+set_header(#headers{}=Hdrs, {lower, "if-none-match"}, Value) ->
+ Hdrs#headers{if_none_match = Value};
+set_header(#headers{}=Hdrs, if_range, Value) ->
+ Hdrs#headers{if_range = Value};
+set_header(#headers{}=Hdrs, {lower, "if-range"}, Value) ->
+ Hdrs#headers{if_range = Value};
+set_header(#headers{}=Hdrs, if_unmodified_since, Value) ->
+ Hdrs#headers{if_unmodified_since = Value};
+set_header(#headers{}=Hdrs, {lower, "if-unmodified-since"}, Value) ->
+ Hdrs#headers{if_unmodified_since = Value};
+set_header(#headers{}=Hdrs, range, Value) ->
+ Hdrs#headers{range = Value};
+set_header(#headers{}=Hdrs, {lower, "range"}, Value) ->
+ Hdrs#headers{range = Value};
+set_header(#headers{}=Hdrs, referer, Value) ->
+ Hdrs#headers{referer = Value};
+set_header(#headers{}=Hdrs, {lower, "referer"}, Value) ->
+ Hdrs#headers{referer = Value};
+set_header(#headers{}=Hdrs, user_agent, Value) ->
+ Hdrs#headers{user_agent = Value};
+set_header(#headers{}=Hdrs, {lower, "user-agent"}, Value) ->
+ Hdrs#headers{user_agent = Value};
+set_header(#headers{}=Hdrs, accept_ranges, Value) ->
+ Hdrs#headers{accept_ranges = Value};
+set_header(#headers{}=Hdrs, {lower, "accept-ranges"}, Value) ->
+ Hdrs#headers{accept_ranges = Value};
+set_header(#headers{}=Hdrs, cookie, Value) ->
+ Hdrs#headers{cookie = Value};
+set_header(#headers{}=Hdrs, {lower, "cookie"}, Value) ->
+ Hdrs#headers{cookie = Value};
+set_header(#headers{}=Hdrs, keep_alive, Value) ->
+ Hdrs#headers{keep_alive = Value};
+set_header(#headers{}=Hdrs, {lower, "keep-alive"}, Value) ->
+ Hdrs#headers{keep_alive = Value};
+set_header(#headers{}=Hdrs, location, Value) ->
+ Hdrs#headers{location = Value};
+set_header(#headers{}=Hdrs, {lower, "location"}, Value) ->
+ Hdrs#headers{location = Value};
+set_header(#headers{}=Hdrs, content_length, Value) ->
+ Hdrs#headers{content_length = Value};
+set_header(#headers{}=Hdrs, {lower, "content-length"}, Value) ->
+ Hdrs#headers{content_length = Value};
+set_header(#headers{}=Hdrs, content_type, Value) ->
+ Hdrs#headers{content_type = Value};
+set_header(#headers{}=Hdrs, {lower, "content-type"}, Value) ->
+ Hdrs#headers{content_type = Value};
+set_header(#headers{}=Hdrs, content_encoding, Value) ->
+ Hdrs#headers{content_encoding = Value};
+set_header(#headers{}=Hdrs, {lower, "content-encoding"}, Value) ->
+ Hdrs#headers{content_encoding = Value};
+set_header(#headers{}=Hdrs, authorization, Value) ->
+ Hdrs#headers{authorization = Value};
+set_header(#headers{}=Hdrs, {lower, "authorization"}, Value) ->
+ Hdrs#headers{authorization = Value};
+set_header(#headers{}=Hdrs, transfer_encoding, Value) ->
+ Hdrs#headers{transfer_encoding = Value};
+set_header(#headers{}=Hdrs, {lower, "transfer-encoding"}, Value) ->
+ Hdrs#headers{transfer_encoding = Value};
+set_header(#headers{}=Hdrs, x_forwarded_for, Value) ->
+ Hdrs#headers{x_forwarded_for = Value};
+set_header(#headers{}=Hdrs, {lower, "x-forwarded-for"}, Value) ->
+ Hdrs#headers{x_forwarded_for = Value};
+set_header(#headers{}=Hdrs, Header, Value) when is_atom(Header) ->
+ set_header(Hdrs, atom_to_list(Header), Value);
+set_header(#headers{}=Hdrs, Header, Value) when is_binary(Header) ->
+ set_header(Hdrs, binary_to_list(Header), Value);
+set_header(#headers{}=Hdrs, Header, Val) when is_binary(Val) ->
+ set_header(Hdrs, {lower, string:to_lower(Header)}, binary_to_list(Val));
+set_header(#headers{other=Other}=Hdrs, {lower, Header}, undefined) ->
+ Handler = fun(_, true, Acc) ->
+ Acc;
+ (HdrVal, false, Acc) ->
+ [HdrVal|Acc]
+ end,
+ NewOther = fold_others(Header, Handler, Other, []),
+ Hdrs#headers{other = lists:reverse(NewOther)};
+set_header(#headers{other=Other}=Hdrs, {lower, Header}, Val) ->
+ HdrName = erlang_header_name(Header),
+ Handler = fun({http_header, Int, _, Rsv, _}, true, {Acc, _}) ->
+ {[{http_header, Int, HdrName, Rsv, Val}|Acc],true};
+ (HdrVal, false, {Acc, Found}) ->
+ {[HdrVal|Acc], Found}
+ end,
+ {NewOther0, Found} = fold_others(Header, Handler, Other, {[], false}),
+ NewOther = case Found of
+ true ->
+ NewOther0;
+ false ->
+ [{http_header, 0, HdrName, undefined, Val}|NewOther0]
+ end,
+ Hdrs#headers{other = lists:reverse(NewOther)};
+set_header(#headers{}=Hdrs, Header, undefined) ->
+ set_header(Hdrs, {lower, string:to_lower(Header)}, undefined);
+set_header(#headers{}=Hdrs, Header, Value) ->
+ set_header(Hdrs, {lower, string:to_lower(Header)}, Value).
+
+get_header(#headers{}=Hdrs, connection) ->
+ Hdrs#headers.connection;
+get_header(#headers{}=Hdrs, {lower, "connection"}) ->
+ Hdrs#headers.connection;
+get_header(#headers{}=Hdrs, accept) ->
+ Hdrs#headers.accept;
+get_header(#headers{}=Hdrs, {lower, "accept"}) ->
+ Hdrs#headers.accept;
+get_header(#headers{}=Hdrs, host) ->
+ Hdrs#headers.host;
+get_header(#headers{}=Hdrs, {lower, "host"}) ->
+ Hdrs#headers.host;
+get_header(#headers{}=Hdrs, if_modified_since) ->
+ Hdrs#headers.if_modified_since;
+get_header(#headers{}=Hdrs, {lower, "if-modified-since"}) ->
+ Hdrs#headers.if_modified_since;
+get_header(#headers{}=Hdrs, if_match) ->
+ Hdrs#headers.if_match;
+get_header(#headers{}=Hdrs, {lower, "if-match"}) ->
+ Hdrs#headers.if_match;
+get_header(#headers{}=Hdrs, if_none_match) ->
+ Hdrs#headers.if_none_match;
+get_header(#headers{}=Hdrs, {lower, "if-none-match"}) ->
+ Hdrs#headers.if_none_match;
+get_header(#headers{}=Hdrs, if_range) ->
+ Hdrs#headers.if_range;
+get_header(#headers{}=Hdrs, {lower, "if-range"}) ->
+ Hdrs#headers.if_range;
+get_header(#headers{}=Hdrs, if_unmodified_since) ->
+ Hdrs#headers.if_unmodified_since;
+get_header(#headers{}=Hdrs, {lower, "if-unmodified-since"}) ->
+ Hdrs#headers.if_unmodified_since;
+get_header(#headers{}=Hdrs, range) ->
+ Hdrs#headers.range;
+get_header(#headers{}=Hdrs, {lower, "range"}) ->
+ Hdrs#headers.range;
+get_header(#headers{}=Hdrs, referer) ->
+ Hdrs#headers.referer;
+get_header(#headers{}=Hdrs, {lower, "referer"}) ->
+ Hdrs#headers.referer;
+get_header(#headers{}=Hdrs, user_agent) ->
+ Hdrs#headers.user_agent;
+get_header(#headers{}=Hdrs, {lower, "user-agent"}) ->
+ Hdrs#headers.user_agent;
+get_header(#headers{}=Hdrs, accept_ranges) ->
+ Hdrs#headers.accept_ranges;
+get_header(#headers{}=Hdrs, {lower, "accept-ranges"}) ->
+ Hdrs#headers.accept_ranges;
+get_header(#headers{}=Hdrs, cookie) ->
+ Hdrs#headers.cookie;
+get_header(#headers{}=Hdrs, {lower, "cookie"}) ->
+ Hdrs#headers.cookie;
+get_header(#headers{}=Hdrs, keep_alive) ->
+ Hdrs#headers.keep_alive;
+get_header(#headers{}=Hdrs, {lower, "keep-alive"}) ->
+ Hdrs#headers.keep_alive;
+get_header(#headers{}=Hdrs, location) ->
+ Hdrs#headers.location;
+get_header(#headers{}=Hdrs, {lower, "location"}) ->
+ Hdrs#headers.location;
+get_header(#headers{}=Hdrs, content_length) ->
+ Hdrs#headers.content_length;
+get_header(#headers{}=Hdrs, {lower, "content-length"}) ->
+ Hdrs#headers.content_length;
+get_header(#headers{}=Hdrs, content_type) ->
+ Hdrs#headers.content_type;
+get_header(#headers{}=Hdrs, {lower, "content-type"}) ->
+ Hdrs#headers.content_type;
+get_header(#headers{}=Hdrs, content_encoding) ->
+ Hdrs#headers.content_encoding;
+get_header(#headers{}=Hdrs, {lower, "content-encoding"}) ->
+ Hdrs#headers.content_encoding;
+get_header(#headers{}=Hdrs, authorization) ->
+ Hdrs#headers.authorization;
+get_header(#headers{}=Hdrs, {lower, "authorization"}) ->
+ Hdrs#headers.authorization;
+get_header(#headers{}=Hdrs, transfer_encoding) ->
+ Hdrs#headers.transfer_encoding;
+get_header(#headers{}=Hdrs, {lower, "transfer-encoding"}) ->
+ Hdrs#headers.transfer_encoding;
+get_header(#headers{}=Hdrs, x_forwarded_for) ->
+ Hdrs#headers.x_forwarded_for;
+get_header(#headers{}=Hdrs, {lower, "x-forwarded-for"}) ->
+ Hdrs#headers.x_forwarded_for;
+get_header(#headers{}=Hdrs, Header) when is_atom(Header) ->
+ get_header(Hdrs, atom_to_list(Header));
+get_header(#headers{}=Hdrs, Header) when is_binary(Header) ->
+ get_header(Hdrs, binary_to_list(Header));
+get_header(#headers{other = Other}, {lower, Header}) ->
+ Handler = fun({http_header, _, _, _, Value}, true, _Acc) ->
+ throw(Value);
+ (_, false, Acc) ->
+ Acc
+ end,
+ catch fold_others(Header, Handler, Other, undefined);
+get_header(#headers{}=Hdrs, Header) ->
+ get_header(Hdrs, {lower, string:to_lower(Header)}).
+
+get_header(#headers{}=Hdrs, Header, Default) ->
+ case get_header(Hdrs, Header) of
+ undefined ->
+ Default;
+ Value ->
+ Value
+ end.
+
+delete_header(#headers{}=Hdrs, Header) ->
+ set_header(Hdrs, Header, undefined).
+
+%% assumes that LowerHdr is already downcased
+fold_others(LowerHdr, Handler, Other, StartAcc) ->
+ lists:foldl(fun({http_header, _, Hdr, _, _}=HdrVal, Acc) ->
+ HdrNm = string:to_lower(
+ if
+ is_atom(Hdr) -> atom_to_list(Hdr);
+ is_binary(Hdr) -> binary_to_list(Hdr);
+ true -> Hdr
+ end),
+ Handler(HdrVal, HdrNm == LowerHdr, Acc)
+ end, StartAcc, Other).
+
+erlang_header_name("cache-control") -> 'Cache-Control';
+erlang_header_name("date") -> 'Date';
+erlang_header_name("pragma") -> 'Pragma';
+erlang_header_name("upgrade") -> 'Upgrade';
+erlang_header_name("via") -> 'Via';
+erlang_header_name("accept-charset") -> 'Accept-Charset';
+erlang_header_name("accept-encoding") -> 'Accept-Encoding';
+erlang_header_name("accept-language") -> 'Accept-Language';
+erlang_header_name("from") -> 'From';
+erlang_header_name("max-forwards") -> 'Max-Forwards';
+erlang_header_name("proxy-authorization") -> 'Proxy-Authorization';
+erlang_header_name("age") -> 'Age';
+erlang_header_name("proxy-authenticate") -> 'Proxy-Authenticate';
+erlang_header_name("public") -> 'Public';
+erlang_header_name("retry-after") -> 'Retry-After';
+erlang_header_name("server") -> 'Server';
+erlang_header_name("vary") -> 'Vary';
+erlang_header_name("warning") -> 'Warning';
+erlang_header_name("www-authenticate") -> 'Www-Authenticate';
+erlang_header_name("allow") -> 'Allow';
+erlang_header_name("content-base") -> 'Content-Base';
+erlang_header_name("content-encoding") -> 'Content-Encoding';
+erlang_header_name("content-language") -> 'Content-Language';
+erlang_header_name("content-location") -> 'Content-Location';
+erlang_header_name("content-md5") -> 'Content-Md5';
+erlang_header_name("content-range") -> 'Content-Range';
+erlang_header_name("etag") -> 'Etag';
+erlang_header_name("expires") -> 'Expires';
+erlang_header_name("last-modified") -> 'Last-Modified';
+erlang_header_name("set-cookie") -> 'Set-Cookie';
+erlang_header_name("set-cookie2") -> 'Set-Cookie2';
+erlang_header_name("proxy-connection") -> 'Proxy-Connection';
+erlang_header_name(Name) -> capitalize_header(Name).
+
+capitalize_header(Name) ->
+ capitalize_header2(Name, "").
+
+capitalize_header2([C | Rest], "") when C >= $a andalso C =< $z ->
+ capitalize_header2(Rest, [C - $a + $A]);
+capitalize_header2([$-, C | Rest], Result) when C >= $a andalso C =< $z ->
+ capitalize_header2(Rest, [C - $a + $A, $- | Result]);
+capitalize_header2([C | Rest], Result) ->
+ capitalize_header2(Rest, [C | Result]);
+capitalize_header2([], Result) ->
+ lists:reverse(Result).
reformat_request(#http_request{method = bad_request}) ->
View
53 src/yaws_config.erl
@@ -1161,21 +1161,17 @@ fload(FD, server, GC, C, Cs, Lno, Chars) ->
Suffixes)},
fload(FD, server, GC, C2, Cs, Lno+1, Next);
- ["revproxy", '=', Prefix, Url] ->
- case (catch yaws_api:parse_url(Url)) of
- {'EXIT', _} ->
- {error, ?F("Bad url at line ~p",[Lno])};
- URL when URL#url.path == "/" ->
- P = case lists:reverse(Prefix) of
- [$/|_Tail] ->
- Prefix;
- Other ->
- lists:reverse(Other)
- end,
- C2 = C#sconf{revproxy = [{P, URL} | C#sconf.revproxy]},
+ ["revproxy", '=' | Tail] ->
+ case parse_revproxy(Tail) of
+ {ok, RevProxy} ->
+ C2 = C#sconf{revproxy = [RevProxy | C#sconf.revproxy]},
fload(FD, server, GC, C2, Cs, Lno+1, Next);
- _URL ->
- {error, "Can't revproxy to an URL with a path "}
+ {error, url} ->
+ {error, ?F("Bad url at line ~p",[Lno])};
+ {error, syntax} ->
+ {error, ?F("Bad revproxy syntax at line ~p",[Lno])};
+ Error ->
+ Error
end;
["fwdproxy", '=', Bool] ->
@@ -1942,6 +1938,35 @@ parse_appmods([], Ack) ->
{ok, Ack}.
+parse_revproxy([Prefix, Url]) ->
+ parse_revproxy_url(Prefix, Url);
+parse_revproxy([Prefix, Url, "intercept_mod", InterceptMod]) ->
+ case parse_revproxy_url(Prefix, Url) of
+ {ok, RP} ->
+ {ok, RP#proxy_cfg{intercept_mod = list_to_atom(InterceptMod)}};
+ Error ->
+ Error
+ end;
+parse_revproxy(_Other) ->
+ {error, syntax}.
+
+parse_revproxy_url(Prefix, Url) ->
+ case (catch yaws_api:parse_url(Url)) of
+ {'EXIT', _} ->
+ {error, url};
+ URL when URL#url.path == "/" ->
+ P = case lists:reverse(Prefix) of
+ [$/|_Tail] ->
+ Prefix;
+ Other ->
+ lists:reverse(Other)
+ end,
+ {ok, #proxy_cfg{prefix=P, url=URL}};
+ _URL ->
+ {error, "Can't revproxy to a URL with a path "}
+ end.
+
+
parse_expires(['<', MimeType, ',' , Expire, '>' | Tail], Ack) ->
{Type, Value} =
case string:tokens(Expire, "+") of
View
70 src/yaws_revproxy.erl
@@ -34,27 +34,30 @@
resp, %% response received from the server
headers, %% and associated headers
srvdata, %% the server data
- is_chunked}). %% true if the response is chunked
+ is_chunked, %% true if the response is chunked
+ intercept_mod %% revproxy request/response intercept module
+ }).
%% TODO: Activate proxy keep-alive with a new option ?
-define(proxy_keepalive, false).
-%% Initialize the connection to the backend server. If an error occured, return
+%% Initialize the connection to the backend server. If an error occurred, return
%% an error 404.
-out(#arg{req=Req, headers=Hdrs, state={Prefix,URL}}=Arg) ->
+out(#arg{req=Req, headers=Hdrs, state=#proxy_cfg{url=URL}=State}=Arg) ->
case connect(URL) of
{ok, Sock, Type} ->
?Debug("Connection established on ~p: Socket=~p, Type=~p~n",
[URL, Sock, Type]),
- RPState = #revproxy{srvsock= Sock,
- type = Type,
- state = sendheaders,
- prefix = Prefix,
- url = URL,
- r_meth = Req#http_request.method,
- r_host = Hdrs#headers.host},
+ RPState = #revproxy{srvsock = Sock,
+ type = Type,
+ state = sendheaders,
+ prefix = State#proxy_cfg.prefix,
+ url = URL,
+ r_meth = Req#http_request.method,
+ r_host = Hdrs#headers.host,
+ intercept_mod = State#proxy_cfg.intercept_mod},
out(Arg#arg{state=RPState});
_ERR ->
?Debug("Connection failed: ~p~n", [_ERR]),
@@ -64,13 +67,29 @@ out(#arg{req=Req, headers=Hdrs, state={Prefix,URL}}=Arg) ->
%% Send the client request to the server then check if the request content is
%% chunked or not
-out(#arg{state=RPState}=Arg) when RPState#revproxy.state == sendheaders ->
+out(#arg{state=#revproxy{}=RPState}=Arg) when RPState#revproxy.state == sendheaders ->
?Debug("Send request headers to backend server: ~n"
" - ~s~n", [?format_record(Arg#arg.req, http_request)]),
- Hdrs = Arg#arg.headers,
- ReqStr = yaws_api:reformat_request(rewrite_request(RPState, Arg#arg.req)),
- HdrsStr = yaws:headers_to_str(rewrite_client_headers(RPState, Hdrs)),
+ Req = rewrite_request(RPState, Arg#arg.req),
+ Hdrs0 = Arg#arg.headers,
+ Hdrs = rewrite_client_headers(RPState, Hdrs0),
+ {NewReq, NewHdrs} = case RPState#revproxy.intercept_mod of
+ undefined ->
+ {Req, Hdrs};
+ InterceptMod ->
+ case catch InterceptMod:rewrite_request(Req, Hdrs) of
+ {ok, NewReq0, NewHdrs0} ->
+ {NewReq0, NewHdrs0};
+ InterceptError ->
+ error_logger:error_msg(
+ "revproxy intercept module ~p:rewrite_request failed: ~p~n",
+ [InterceptMod, InterceptError]),
+ exit({error, intercept_mod})
+ end
+ end,
+ ReqStr = yaws_api:reformat_request(NewReq),
+ HdrsStr = yaws:headers_to_str(NewHdrs),
case send(RPState, [ReqStr, "\r\n", HdrsStr, "\r\n"]) of
ok ->
RPState1 = if
@@ -94,7 +113,7 @@ out(#arg{state=RPState}=Arg) when RPState#revproxy.state == sendheaders ->
%% Send the request content to the server. Here the content is not chunked. But
-%% it can be splitted because of 'partial_post_size' value.
+%% it can be split because of 'partial_post_size' value.
out(#arg{state=RPState}=Arg) when RPState#revproxy.state == sendcontent ->
case Arg#arg.clidata of
{partial, Bin} ->
@@ -184,10 +203,25 @@ out(#arg{state=RPState}=Arg) when RPState#revproxy.state == recvheaders ->
close(RPState),
outXXX(500, Arg);
- {Resp, RespHdrs} when is_record(Resp, http_response) ->
+ {Resp0, RespHdrs0} when is_record(Resp0, http_response) ->
?Debug("Response headers received from backend server:~n"
- " - ~s~n - ~s~n", [?format_record(Resp, http_response),
- ?format_record(RespHdrs, headers)]),
+ " - ~s~n - ~s~n", [?format_record(Resp0, http_response),
+ ?format_record(RespHdrs0, headers)]),
+
+ {Resp, RespHdrs} = case RPState#revproxy.intercept_mod of
+ undefined ->
+ {Resp0, RespHdrs0};
+ InterceptMod ->
+ case catch InterceptMod:rewrite_response(Resp0, RespHdrs0) of
+ {ok, NewResp, NewRespHdrs} ->
+ {NewResp, NewRespHdrs};
+ InterceptError ->
+ error_logger:error_msg(
+ "revproxy intercept module ~p:rewrite_response failure: ~p~n",
+ [InterceptMod, InterceptError]),
+ exit({error, intercept_mod})
+ end
+ end,
{CliConn, SrvConn} = get_connection_status(
(Arg#arg.req)#http_request.version,
View
6 src/yaws_server.erl
@@ -2002,15 +2002,15 @@ is_revproxy(ARG, Path, SC = #sconf{revproxy = RevConf}) ->
{false, _} ->
is_revproxy1(Path, RevConf);
{true, _} ->
- {true, {"/", fwdproxy_url(ARG)}}
+ {true, #proxy_cfg{prefix="/", url=fwdproxy_url(ARG)}}
end.
is_revproxy1(_,[]) ->
false;
-is_revproxy1(Path, [{Prefix, URL} | Tail]) ->
+is_revproxy1(Path, [#proxy_cfg{prefix=Prefix}=RevConf | Tail]) ->
case yaws:is_prefix(Prefix, Path) of
{true,_} ->
- {true, {Prefix,URL}};
+ {true, RevConf};
false ->
is_revproxy1(Path, Tail)
end.
View
19 src/yaws_websockets.erl
@@ -677,23 +677,8 @@ get_nonce_header(Headers) ->
query_header(HeaderName, Headers) ->
query_header(HeaderName, Headers, undefined).
-query_header(Header, #headers{other=L}, Default) ->
- lists:foldl(fun({http_header,_,K0,_,V}, undefined) ->
- K = case is_atom(K0) of
- true ->
- atom_to_list(K0);
- false ->
- K0
- end,
- case string:to_lower(K) of
- Header ->
- V;
- _ ->
- Default
- end;
- (_, Acc) ->
- Acc
- end, Default, L).
+query_header(Header, Headers, Default) ->
+ yaws_api:get_header(Headers, Header, Default).
hash_nonce(Nonce) ->
Salted = Nonce ++ "258EAFA5-E914-47DA-95CA-C5AB0DC85B11",
View
10 test/conf/revproxy.conf
@@ -147,3 +147,13 @@ use_fdsrv = false
docroot = %YTOP%/www
revproxy = /revproxy http://localhost:8002
</server>
+
+<server localhost>
+ port = 8005
+ listen = 0.0.0.0
+ listen_backlog = 512
+ docroot = %YTOP%/www
+ revproxy = /revproxy1 http://localhost:8002 intercept_mod intercept1
+ revproxy = /revproxy2 http://localhost:8002 intercept_mod intercept2
+ revproxy = /revproxy3 http://localhost:8002 intercept_mod intercept3
+</server>
View
3 test/eunit/Makefile
@@ -4,7 +4,8 @@ OBJ := multipart_post_parsing.beam \
yaws_session_server_test.beam \
ehtml_test.beam \
embedded_yaws_id_dir.beam \
- cookies.beam
+ cookies.beam \
+ headers.beam
all: $(OBJ)
View
78 test/eunit/headers.erl
@@ -0,0 +1,78 @@
+-module(headers).
+-compile(export_all).
+-include("../../include/yaws_api.hrl").
+-include_lib("eunit/include/eunit.hrl").
+
+set_headers_test() ->
+ Value = "test value",
+ lists:foreach(fun({Hdr, StrHdr, Fun}) ->
+ NewHdrs = yaws_api:set_header(#headers{}, Hdr, Value),
+ Value = yaws_api:Fun(NewHdrs),
+ Value = yaws_api:get_header(NewHdrs, StrHdr),
+ 0 = length(NewHdrs#headers.other)
+ end, field_headers()),
+ Hdrs = yaws_api:set_header(create_headers(99), age, "123"),
+ 100 = length(Hdrs#headers.other),
+ ok.
+
+get_headers_test() ->
+ undefined = yaws_api:get_header(#headers{}, accept),
+ undefined = yaws_api:get_header(#headers{}, "Connection"),
+ none = yaws_api:get_header(#headers{}, range, none),
+ none = yaws_api:get_header(#headers{}, "No-Such-Header", none),
+ Hdrs = create_headers(100),
+ lists:foreach(fun(I) ->
+ L = integer_to_list(I),
+ Val = "value"++L,
+ Hdr = "X-HEADER-"++L,
+ Val = yaws_api:get_header(Hdrs, Hdr)
+ end, lists:seq(1,100)),
+ ok.
+
+delete_headers_test() ->
+ Headers = create_headers(10),
+ {0, #headers{}} = lists:foldl(
+ fun(I, {Size, Hdrs}) ->
+ Size = length(Hdrs#headers.other),
+ L = integer_to_list(I),
+ Hdr = "X-Header-"++L,
+ NHdrs = yaws_api:delete_header(Hdrs, Hdr),
+ NewSize = Size - 1,
+ NewSize = length(NHdrs#headers.other),
+ {NewSize, NHdrs}
+ end, {10, Headers}, lists:seq(1,10)),
+ ok.
+
+create_headers(N) ->
+ lists:foldl(fun({Hdr,Val}, Hdrs) ->
+ yaws_api:set_header(Hdrs, Hdr, Val)
+ end,
+ #headers{},
+ [begin
+ L = integer_to_list(I),
+ {"X-Header-"++L, "value"++L}
+ end || I <- lists:seq(1,N)]).
+
+%% these headers are the fields in the #headers{} record
+field_headers() ->
+ [{connection, "Connection", headers_connection},
+ {accept, "Accept", headers_accept},
+ {host, "Host", headers_host},
+ {if_modified_since,"If-Modified-Since", headers_if_modified_since},
+ {if_match, "If-Match", headers_if_match},
+ {if_none_match, "If-None-Match", headers_if_none_match},
+ {if_range, "if-Range", headers_if_range},
+ {if_unmodified_since, "If-Unmodified-Since", headers_if_unmodified_since},
+ {range, "Range", headers_range},
+ {referer, "Referer", headers_referer},
+ {user_agent, "User-Agent", headers_user_agent},
+ {accept_ranges, "Accept-Ranges", headers_accept_ranges},
+ {cookie, "Cookie", headers_cookie},
+ {keep_alive, "Keep-Alive", headers_keep_alive},
+ {location, "Location", headers_location},
+ {content_length, "Content-Length", headers_content_length},
+ {content_type, "Content-Type", headers_content_type},
+ {content_encoding, "Content-Encoding", headers_content_encoding},
+ {authorization, "Authorization", headers_authorization},
+ {transfer_encoding, "Transfer-Encoding", headers_transfer_encoding},
+ {x_forwarded_for, "X-Forwarded-For", headers_x_forwarded_for}].
View
13 test/t4/Makefile
@@ -2,8 +2,17 @@ include ../support/include.mk
.PHONY: all test debug clean
+T4BEAMS := app_test.beam \
+ rewritetest.beam \
+ posttest.beam \
+ streamtest.beam \
+ nolengthtest.beam \
+ intercept1.beam \
+ intercept2.beam \
+ intercept3.beam
+
#
-all: conf setup app_test.beam rewritetest.beam posttest.beam streamtest.beam nolengthtest.beam
+all: conf setup $(T4BEAMS)
@echo "all ok"
@@ -18,7 +27,7 @@ test: all start
dd if=/dev/zero of=../../www/2000.txt bs=1024 count=2000 >/dev/null 2>&1
dd if=/dev/zero of=../../www/3000.txt bs=1024 count=3000 >/dev/null 2>&1
dd if=/dev/zero of=../../www/10000.txt bs=1024 count=10000 >/dev/null 2>&1
- dd if=/dev/urandom of=www2/8388608.bin bs=1 count=8388608 >/dev/null 2>&1
+ dd if=/dev/urandom of=www2/8388608.bin bs=1024 count=8192 >/dev/null 2>&1
ul=`ulimit -n` ; \
val=`expr $$ul '<' $(ULIMIT)` ; \
if [ $$val = 1 ] ; then \
View
25 test/t4/app_test.erl
@@ -29,6 +29,9 @@ start() ->
rewrite_revproxy_test(),
large_content_revproxy_test(),
no_content_length_revproxy_test(),
+ failed_req_interception_revproxy_test(),
+ failed_resp_interception_revproxy_test(),
+ good_interception_revproxy_test(),
fwdproxy_test(),
ok
catch
@@ -220,6 +223,28 @@ no_content_length_revproxy_test() ->
?line Res = Body,
ok.
+failed_req_interception_revproxy_test() ->
+ io:format("failed_req_interception_revproxy_test\n", []),
+ Uri = "http://localhost:8005/revproxy1/failedreqinterception",
+ ?line {ok, "500", _, _} = ibrowse:send_req(Uri, [], get),
+ ok.
+
+failed_resp_interception_revproxy_test() ->
+ io:format("failed_resp_interception_revproxy_test\n", []),
+ Uri = "http://localhost:8005/revproxy2/failedrespinterception",
+ ?line {ok, "500", _, _} = ibrowse:send_req(Uri, [], get),
+ ok.
+
+good_interception_revproxy_test() ->
+ io:format("good_interception_revproxy_test\n", []),
+ Uri = "http://localhost:8005/revproxy3/hello.txt",
+ Res = "Hello, World!\n",
+
+ ?line {ok, "200", Hdrs, Body} = ibrowse:send_req(Uri, [], get),
+ ?line Body = Res,
+ ?line "true" = proplists:get_value("X-Test-Interception", Hdrs),
+ ok.
+
fwdproxy_test() ->
io:format("fwdproxy_test\n", []),
Uri1 = "http://localhost:8001/rewrite/hello.txt",
View
10 test/t4/intercept1.erl
@@ -0,0 +1,10 @@
+-module(intercept1).
+-export([rewrite_request/2, rewrite_response/2]).
+
+%% Test that returning an incorrect return value from rewrite_request
+%% is handled correctly by the reverse proxy.
+rewrite_request(_, _) ->
+ error.
+
+rewrite_response(Resp, Hdrs) ->
+ {ok, Resp, Hdrs}.
View
10 test/t4/intercept2.erl
@@ -0,0 +1,10 @@
+-module(intercept2).
+-export([rewrite_request/2, rewrite_response/2]).
+
+rewrite_request(Req, Hdrs) ->
+ {ok, Req, Hdrs}.
+
+%% Test that returning an incorrect return value from rewrite_request
+%% is handled correctly by the reverse proxy.
+rewrite_response(_, _) ->
+ error.
View
10 test/t4/intercept3.erl
@@ -0,0 +1,10 @@
+-module(intercept3).
+-export([rewrite_request/2, rewrite_response/2]).
+
+-include("../../include/yaws_api.hrl").
+
+rewrite_request(Req, Hdrs) ->
+ {ok, Req, yaws_api:set_header(Hdrs, accept, "text/plain")}.
+
+rewrite_response(Resp, Hdrs) ->
+ {ok, Resp, yaws_api:set_header(Hdrs, "X-Test-Interception", "true")}.

0 comments on commit cccc578

Please sign in to comment.
Something went wrong with that request. Please try again.