ibrowse-1.5.0. See README for details

README

@@ -1,5 +1,3 @@
-$Id: README,v 1.16 2008/05/21 15:28:11 chandrusf Exp $
-
 ibrowse is a HTTP client. The following are a list of features.
 	- RFC2616 compliant (AFAIK) 
 	- supports GET, POST, OPTIONS, HEAD, PUT, DELETE, TRACE, 
@@ -18,10 +16,37 @@ ibrowse is a HTTP client. The following are a list of features.
 
 ibrowse is available under two different licenses. LGPL and the BSD license.
 
-Comments to : Chandrashekhar.Mullaparthi@t-mobile.co.uk
+Comments to : Chandrashekhar.Mullaparthi@gmail.com
+
+Version : 1.5.0
+
+Latest version : git://github.com/cmullaparthi/ibrowse.git
 
 CONTRIBUTIONS & CHANGE HISTORY
 ==============================
+29-06-2009 - * Fixed following issues reported by Oscar Hellström
+               - Use {active, once} instead of {active, true}
+	       - Fix 'dodgy' timeout handling
+	       - Use binaries internally instead of lists to reduce memory
+                 consumption on 64 bit platforms. The default response format
+                 is still 'list' to maintain backwards compatibility. Use the
+                 option {response_format, binary} to get responses as binaries.
+	     * Fixed chunking bug (reported by Adam Kocoloski)
+	     * Added new option {inactivity_timeout, Milliseconds} to timeout
+               requests if no data is received on the link for the specified
+               interval. Useful when responses are large and links are flaky.
+	     * Added ibrowse:all_trace_off/0 to turn off all tracing
+	     * Change to the way responses to asynchronous requests are
+               returned. The following messages have been removed.
+                * {ibrowse_async_response, Req_id, {chunk_start, Chunk_size}}
+ 		* {ibrowse_async_response, Req_id, chunk_end}
+	     * Fixed Makefiles as part of Debian packaging 
+	       (thanks to Thomas Lindgren)
+             * Moved repository from Sourceforge to Github
+
+11-06-2009 - * Added option to control size of streamed chunks. Also added 
+               option for the client to receive responses in binary format.
+
 21-05-2008 - * Fixed bug in reading some options from the ibrowse.conf file.
                Reported by Erik Reitsma on the erlyaws mailing list
              * Fixed bug when cleaning up closing connections

doc/ibrowse.html

@@ -10,9 +10,9 @@
 
 <h1>Module ibrowse</h1>
 <ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>The ibrowse application implements an HTTP 1.1 client.
-<p>Copyright © 2005-2008 Chandrashekhar Mullaparthi</p>
+<p>Copyright © 2005-2009 Chandrashekhar Mullaparthi</p>
 
-<p><b>Version:</b> 1.4</p>
+<p><b>Version:</b> 1.5.0</p>
 <p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
 <p><b>Authors:</b> Chandrashekhar Mullaparthi (<a href="mailto:chandrashekhar dot mullaparthi at gmail dot com"><tt>chandrashekhar dot mullaparthi at gmail dot com</tt></a>).</p>
 
@@ -64,7 +64,8 @@ <h2><a name="description">Description</a></h2><p>The ibrowse application impleme
   speed achieved using only erlang has been good enough, so the
   driver isn't actually used.</p>
 <h2><a name="index">Function Index</a></h2>
-<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
+<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#all_trace_off-0">all_trace_off/0</a></td><td>Turn Off ALL tracing.</td></tr>
+<tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
 <tr><td valign="top"><a href="#get_config_value-1">get_config_value/1</a></td><td>Internal export.</td></tr>
 <tr><td valign="top"><a href="#get_config_value-2">get_config_value/2</a></td><td>Internal export.</td></tr>
 <tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
@@ -111,52 +112,57 @@ <h2><a name="index">Function Index</a></h2>
 
 <h2><a name="functions">Function Details</a></h2>
 
+<h3 class="function"><a name="all_trace_off-0">all_trace_off/0</a></h3>
+<div class="spec">
+<p><tt>all_trace_off() -&gt; ok</tt></p>
+</div><p>Turn Off ALL tracing</p>
+
 <h3 class="function"><a name="code_change-3">code_change/3</a></h3>
 <div class="spec">
-<p><tt>code_change() -&gt; term()</tt></p>
+<p><tt>code_change(OldVsn, State, Extra) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="get_config_value-1">get_config_value/1</a></h3>
 <div class="spec">
-<p><tt>get_config_value() -&gt; term()</tt></p>
+<p><tt>get_config_value(Key) -&gt; any()</tt></p>
 </div><p>Internal export</p>
 
 <h3 class="function"><a name="get_config_value-2">get_config_value/2</a></h3>
 <div class="spec">
-<p><tt>get_config_value() -&gt; term()</tt></p>
+<p><tt>get_config_value(Key, DefVal) -&gt; any()</tt></p>
 </div><p>Internal export</p>
 
 <h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
 <div class="spec">
-<p><tt>handle_call() -&gt; term()</tt></p>
+<p><tt>handle_call(Request, From, State) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
 <div class="spec">
-<p><tt>handle_cast() -&gt; term()</tt></p>
+<p><tt>handle_cast(Msg, State) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
 <div class="spec">
-<p><tt>handle_info() -&gt; term()</tt></p>
+<p><tt>handle_info(Info, State) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="init-1">init/1</a></h3>
 <div class="spec">
-<p><tt>init() -&gt; term()</tt></p>
+<p><tt>init(X1) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="rescan_config-0">rescan_config/0</a></h3>
 <div class="spec">
-<p><tt>rescan_config() -&gt; term()</tt></p>
+<p><tt>rescan_config() -&gt; any()</tt></p>
 </div><p>Clear current configuration for ibrowse and load from the file
   ibrowse.conf in the IBROWSE_EBIN/../priv directory. Current
   configuration is cleared only if the ibrowse.conf file is readable
   using file:consult/1</p>
 
 <h3 class="function"><a name="rescan_config-1">rescan_config/1</a></h3>
 <div class="spec">
-<p><tt>rescan_config() -&gt; term()</tt></p>
+<p><tt>rescan_config(File) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="send_req-3">send_req/3</a></h3>
@@ -193,20 +199,21 @@ <h3 class="function"><a name="send_req-5">send_req/5</a></h3>
 <div class="spec">
 <p><tt>send_req(Url::string(), Headers::<a href="#type-headerList">headerList()</a>, Method::<a href="#type-method">method()</a>, Body::<a href="#type-body">body()</a>, Options::<a href="#type-optionList">optionList()</a>) -&gt; <a href="#type-response">response()</a></tt>
 <ul class="definitions"><li><tt><a name="type-optionList">optionList()</a> = [<a href="#type-option">option()</a>]</tt></li>
-<li><tt><a name="type-option">option()</a> = {max_sessions, integer()} | {max_pipeline_size, integer()} | {trace, <a href="#type-boolean">boolean()</a>} | {is_ssl, <a href="#type-boolean">boolean()</a>} | {ssl_options, [SSLOpt]} | {pool_name, atom()} | {proxy_host, string()} | {proxy_port, integer()} | {proxy_user, string()} | {proxy_password, string()} | {use_absolute_uri, <a href="#type-boolean">boolean()</a>} | {basic_auth, {<a href="#type-username">username()</a>, <a href="#type-password">password()</a>}} | {cookie, string()} | {content_length, integer()} | {content_type, string()} | {save_response_to_file, <a href="#type-srtf">srtf()</a>} | {stream_to, <a href="#type-process">process()</a>} | {http_vsn, {MajorVsn, MinorVsn}} | {host_header, string()} | {transfer_encoding, {chunked, ChunkSize}}</tt></li>
+<li><tt><a name="type-option">option()</a> = {max_sessions, integer()} | {response_format, <a href="#type-response_format">response_format()</a>} | {stream_chunk_size, integer()} | {max_pipeline_size, integer()} | {trace, <a href="#type-boolean">boolean()</a>} | {is_ssl, <a href="#type-boolean">boolean()</a>} | {ssl_options, [SSLOpt]} | {pool_name, atom()} | {proxy_host, string()} | {proxy_port, integer()} | {proxy_user, string()} | {proxy_password, string()} | {use_absolute_uri, <a href="#type-boolean">boolean()</a>} | {basic_auth, {<a href="#type-username">username()</a>, <a href="#type-password">password()</a>}} | {cookie, string()} | {content_length, integer()} | {content_type, string()} | {save_response_to_file, <a href="#type-srtf">srtf()</a>} | {stream_to, <a href="#type-process">process()</a>} | {http_vsn, {MajorVsn, MinorVsn}} | {host_header, string()} | {inactivity_timeout, integer()} | {connect_timeout, integer()} | {transfer_encoding, {chunked, ChunkSize}}</tt></li>
 <li><tt><a name="type-process">process()</a> = pid() | atom()</tt></li>
 <li><tt><a name="type-username">username()</a> = string()</tt></li>
 <li><tt><a name="type-password">password()</a> = string()</tt></li>
 <li><tt>SSLOpt = term()</tt></li>
 <li><tt>ChunkSize = integer()</tt></li>
 <li><tt><a name="type-srtf">srtf()</a> = <a href="#type-boolean">boolean()</a> | <a href="#type-filename">filename()</a></tt></li>
 <li><tt><a name="type-filename">filename()</a> = string()</tt></li>
+<li><tt><a name="type-response_format">response_format()</a> = list | binary</tt></li>
 </ul></p>
 </div><p>Same as send_req/4.
   For a description of SSL Options, look in the ssl manpage. If the
   HTTP Version to use is not specified, the default is 1.1.
   <br>
-  <p>The <code>host_header</code> is useful in the case where ibrowse is
+  <p>The <code>host_header</code> option is useful in the case where ibrowse is
   connecting to a component such as <a href="http://www.stunnel.org">stunnel</a> which then sets up a
   secure connection to a webserver. In this case, the URL supplied to
   ibrowse must have the stunnel host/port details, but that won't
@@ -223,7 +230,35 @@ <h3 class="function"><a name="send_req-5">send_req/5</a></h3>
   <li>Whenever an error occurs in the processing of a request, ibrowse will return as much
   information as it has, such as HTTP Status Code and HTTP Headers. When this happens, the response
   is of the form <code>{error, {Reason, {stat_code, StatusCode}, HTTP_headers}}</code></li>
-  </ul></p>
+
+  <li>The <code>inactivity_timeout</code> option is useful when
+  dealing with large response bodies and/or slow links. In these
+  cases, it might be hard to estimate how long a request will take to
+  complete. In such cases, the client might want to timeout if no
+  data has been received on the link for a certain time interval.</li>
+
+  <li>
+  The <code>connect_timeout</code> option is to specify how long the
+  client process should wait for connection establishment. This is
+  useful in scenarios where connections to servers are usually setup
+  very fast, but responses might take much longer compared to
+  connection setup. In such cases, it is better for the calling
+  process to timeout faster if there is a problem (DNS lookup
+  delays/failures, network routing issues, etc). The total timeout
+  value specified for the request will enforced. To illustrate using
+  an example:
+  <code>
+  ibrowse:send_req("http://www.example.com/cgi-bin/request", [], get, [], [{connect_timeout, 100}], 1000).
+  </code>
+  In the above invocation, if the connection isn't established within
+  100 milliseconds, the request will fail with
+  <code>{error, conn_failed}</code>.<br>
+  If connection setup succeeds, the total time allowed for the
+  request to complete will be 1000 milliseconds minus the time taken
+  for connection setup.
+  </li>
+  </ul>
+ </p>
 
 <h3 class="function"><a name="send_req-6">send_req/6</a></h3>
 <div class="spec">
@@ -235,31 +270,31 @@ <h3 class="function"><a name="send_req-6">send_req/6</a></h3>
 
 <h3 class="function"><a name="send_req_direct-4">send_req_direct/4</a></h3>
 <div class="spec">
-<p><tt>send_req_direct() -&gt; term()</tt></p>
+<p><tt>send_req_direct(Conn_pid, Url, Headers, Method) -&gt; any()</tt></p>
 </div><p>Same as send_req/3 except that the first argument is the PID
   returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
 
 <h3 class="function"><a name="send_req_direct-5">send_req_direct/5</a></h3>
 <div class="spec">
-<p><tt>send_req_direct() -&gt; term()</tt></p>
+<p><tt>send_req_direct(Conn_pid, Url, Headers, Method, Body) -&gt; any()</tt></p>
 </div><p>Same as send_req/4 except that the first argument is the PID
   returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
 
 <h3 class="function"><a name="send_req_direct-6">send_req_direct/6</a></h3>
 <div class="spec">
-<p><tt>send_req_direct() -&gt; term()</tt></p>
+<p><tt>send_req_direct(Conn_pid, Url, Headers, Method, Body, Options) -&gt; any()</tt></p>
 </div><p>Same as send_req/5 except that the first argument is the PID
   returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
 
 <h3 class="function"><a name="send_req_direct-7">send_req_direct/7</a></h3>
 <div class="spec">
-<p><tt>send_req_direct() -&gt; term()</tt></p>
+<p><tt>send_req_direct(Conn_pid, Url, Headers, Method, Body, Options, Timeout) -&gt; any()</tt></p>
 </div><p>Same as send_req/6 except that the first argument is the PID
   returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
 
 <h3 class="function"><a name="set_dest-3">set_dest/3</a></h3>
 <div class="spec">
-<p><tt>set_dest() -&gt; term()</tt></p>
+<p><tt>set_dest(Host, Port, T) -&gt; any()</tt></p>
 </div><p>Deprecated. Use set_max_sessions/3 and set_max_pipeline_size/3
   for achieving the same effect.</p>
 
@@ -275,15 +310,15 @@ <h3 class="function"><a name="set_max_sessions-3">set_max_sessions/3</a></h3>
 
 <h3 class="function"><a name="show_dest_status-2">show_dest_status/2</a></h3>
 <div class="spec">
-<p><tt>show_dest_status() -&gt; term()</tt></p>
+<p><tt>show_dest_status(Host, Port) -&gt; any()</tt></p>
 </div><p>Shows some internal information about load balancing to a
   specified Host:Port. Info about workers spawned using
   spawn_worker_process/2 or spawn_link_worker_process/2 is not
   included.</p>
 
 <h3 class="function"><a name="spawn_link_worker_process-2">spawn_link_worker_process/2</a></h3>
 <div class="spec">
-<p><tt>spawn_link_worker_process() -&gt; term()</tt></p>
+<p><tt>spawn_link_worker_process(Host, Port) -&gt; any()</tt></p>
 </div><p>Same as spawn_worker_process/2 except the the calling process
   is linked to the worker process which is spawned.</p>
 
@@ -305,7 +340,7 @@ <h3 class="function"><a name="spawn_worker_process-2">spawn_worker_process/2</a>
 
 <h3 class="function"><a name="start-0">start/0</a></h3>
 <div class="spec">
-<p><tt>start() -&gt; term()</tt></p>
+<p><tt>start() -&gt; any()</tt></p>
 </div><p>Starts the ibrowse process without linking. Useful when testing using the shell</p>
 
 <h3 class="function"><a name="start_link-0">start_link/0</a></h3>
@@ -315,7 +350,7 @@ <h3 class="function"><a name="start_link-0">start_link/0</a></h3>
 
 <h3 class="function"><a name="stop-0">stop/0</a></h3>
 <div class="spec">
-<p><tt>stop() -&gt; term()</tt></p>
+<p><tt>stop() -&gt; any()</tt></p>
 </div><p>Stop the ibrowse process. Useful when testing using the shell.</p>
 
 <h3 class="function"><a name="stop_worker_process-1">stop_worker_process/1</a></h3>
@@ -327,28 +362,28 @@ <h3 class="function"><a name="stop_worker_process-1">stop_worker_process/1</a></
 
 <h3 class="function"><a name="terminate-2">terminate/2</a></h3>
 <div class="spec">
-<p><tt>terminate() -&gt; term()</tt></p>
+<p><tt>terminate(Reason, State) -&gt; any()</tt></p>
 </div>
 
 <h3 class="function"><a name="trace_off-0">trace_off/0</a></h3>
 <div class="spec">
-<p><tt>trace_off() -&gt; term()</tt></p>
+<p><tt>trace_off() -&gt; any()</tt></p>
 </div><p>Turn tracing off for the ibrowse process</p>
 
 <h3 class="function"><a name="trace_off-2">trace_off/2</a></h3>
 <div class="spec">
-<p><tt>trace_off(Host, Port) -&gt; term()</tt></p>
+<p><tt>trace_off(Host, Port) -&gt; ok</tt></p>
 </div><p>Turn tracing OFF for all connections to the specified HTTP
   server.</p>
 
 <h3 class="function"><a name="trace_on-0">trace_on/0</a></h3>
 <div class="spec">
-<p><tt>trace_on() -&gt; term()</tt></p>
+<p><tt>trace_on() -&gt; any()</tt></p>
 </div><p>Turn tracing on for the ibrowse process</p>
 
 <h3 class="function"><a name="trace_on-2">trace_on/2</a></h3>
 <div class="spec">
-<p><tt>trace_on(Host, Port) -&gt; term()</tt>
+<p><tt>trace_on(Host, Port) -&gt; ok</tt>
 <ul class="definitions"><li><tt>Host = string()</tt></li>
 <li><tt>Port = integer()</tt></li>
 </ul></p>
@@ -357,6 +392,6 @@ <h3 class="function"><a name="trace_on-2">trace_on/2</a></h3>
 <hr>
 
 <div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
-<p><i>Generated by EDoc, Mar 27 2008, 01:20:55.</i></p>
+<p><i>Generated by EDoc, Jun 30 2009, 23:44:01.</i></p>
 </body>
 </html>

ebin/ibrowse.app

@@ -1,12 +1,12 @@
 {application, ibrowse,
         [{description, "HTTP client application"},
-         {vsn, "1.4"},
+         {vsn, "1.5.0"},
          {modules, [ ibrowse, 
 		     ibrowse_http_client, 
 		     ibrowse_app, 
 		     ibrowse_sup, 
-		     ibrowse_lb, 
-		     ibrowse_lib ]},
+		     ibrowse_lib,
+		     ibrowse_lb ]},
          {registered, []},
          {applications, [kernel,stdlib,sasl]},
 	 {env, []},