forked from cmullaparthi/ibrowse
-
Notifications
You must be signed in to change notification settings - Fork 2
/
ibrowse.html
362 lines (315 loc) · 20.7 KB
/
ibrowse.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>Module ibrowse</title>
<link rel="stylesheet" type="text/css" href="stylesheet.css" title="EDoc">
</head>
<body bgcolor="white">
<div class="navbar"><a name="#navbar_top"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
<hr>
<h1>Module ibrowse</h1>
<ul class="index"><li><a href="#description">Description</a></li><li><a href="#index">Function Index</a></li><li><a href="#functions">Function Details</a></li></ul>The ibrowse application implements an HTTP 1.1 client.
<p>Copyright © 2005-2008 Chandrashekhar Mullaparthi</p>
<p><b>Version:</b> 1.4</p>
<p><b>Behaviours:</b> <a href="gen_server.html"><tt>gen_server</tt></a>.</p>
<p><b>Authors:</b> Chandrashekhar Mullaparthi (<a href="mailto:chandrashekhar dot mullaparthi at gmail dot com"><tt>chandrashekhar dot mullaparthi at gmail dot com</tt></a>).</p>
<h2><a name="description">Description</a></h2><p>The ibrowse application implements an HTTP 1.1 client. This
module implements the API of the HTTP client. There is one named
process called 'ibrowse' which assists in load balancing and maintaining configuration. There is one load balancing process per unique webserver. There is
one process to handle one TCP connection to a webserver
(implemented in the module ibrowse_http_client). Multiple connections to a
webserver are setup based on the settings for each webserver. The
ibrowse process also determines which connection to pipeline a
certain request on. The functions to call are send_req/3,
send_req/4, send_req/5, send_req/6.</p>
<p>Here are a few sample invocations.</p>
<p><code>
ibrowse:send_req("http://intranet/messenger/", [], get).
<br><br>
ibrowse:send_req("http://www.google.com/", [], get, [],
[{proxy_user, "XXXXX"},
{proxy_password, "XXXXX"},
{proxy_host, "proxy"},
{proxy_port, 8080}], 1000).
<br><br>
ibrowse:send_req("http://www.erlang.org/download/otp_src_R10B-3.tar.gz", [], get, [],
[{proxy_user, "XXXXX"},
{proxy_password, "XXXXX"},
{proxy_host, "proxy"},
{proxy_port, 8080},
{save_response_to_file, true}], 1000).
<br><br>
ibrowse:send_req("http://www.erlang.org", [], head).
<br><br>
ibrowse:send_req("http://www.sun.com", [], options).
<br><br>
ibrowse:send_req("http://www.bbc.co.uk", [], trace).
<br><br>
ibrowse:send_req("http://www.google.com", [], get, [],
[{stream_to, self()}]).
</code></p>
<p>A driver exists which implements URL encoding in C, but the
speed achieved using only erlang has been good enough, so the
driver isn't actually used.</p>
<h2><a name="index">Function Index</a></h2>
<table width="100%" border="1" cellspacing="0" cellpadding="2" summary="function index"><tr><td valign="top"><a href="#code_change-3">code_change/3</a></td><td></td></tr>
<tr><td valign="top"><a href="#get_config_value-1">get_config_value/1</a></td><td>Internal export.</td></tr>
<tr><td valign="top"><a href="#get_config_value-2">get_config_value/2</a></td><td>Internal export.</td></tr>
<tr><td valign="top"><a href="#handle_call-3">handle_call/3</a></td><td></td></tr>
<tr><td valign="top"><a href="#handle_cast-2">handle_cast/2</a></td><td></td></tr>
<tr><td valign="top"><a href="#handle_info-2">handle_info/2</a></td><td></td></tr>
<tr><td valign="top"><a href="#init-1">init/1</a></td><td></td></tr>
<tr><td valign="top"><a href="#rescan_config-0">rescan_config/0</a></td><td>Clear current configuration for ibrowse and load from the file
ibrowse.conf in the IBROWSE_EBIN/../priv directory.</td></tr>
<tr><td valign="top"><a href="#rescan_config-1">rescan_config/1</a></td><td></td></tr>
<tr><td valign="top"><a href="#send_req-3">send_req/3</a></td><td>This is the basic function to send a HTTP request.</td></tr>
<tr><td valign="top"><a href="#send_req-4">send_req/4</a></td><td>Same as send_req/3.</td></tr>
<tr><td valign="top"><a href="#send_req-5">send_req/5</a></td><td>Same as send_req/4.</td></tr>
<tr><td valign="top"><a href="#send_req-6">send_req/6</a></td><td>Same as send_req/5.</td></tr>
<tr><td valign="top"><a href="#send_req_direct-4">send_req_direct/4</a></td><td>Same as send_req/3 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2.</td></tr>
<tr><td valign="top"><a href="#send_req_direct-5">send_req_direct/5</a></td><td>Same as send_req/4 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2.</td></tr>
<tr><td valign="top"><a href="#send_req_direct-6">send_req_direct/6</a></td><td>Same as send_req/5 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2.</td></tr>
<tr><td valign="top"><a href="#send_req_direct-7">send_req_direct/7</a></td><td>Same as send_req/6 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2.</td></tr>
<tr><td valign="top"><a href="#set_dest-3">set_dest/3</a></td><td>Deprecated.</td></tr>
<tr><td valign="top"><a href="#set_max_pipeline_size-3">set_max_pipeline_size/3</a></td><td>Set the maximum pipeline size for each connection to a specific Host:Port.</td></tr>
<tr><td valign="top"><a href="#set_max_sessions-3">set_max_sessions/3</a></td><td>Set the maximum number of connections allowed to a specific Host:Port.</td></tr>
<tr><td valign="top"><a href="#show_dest_status-2">show_dest_status/2</a></td><td>Shows some internal information about load balancing to a
specified Host:Port.</td></tr>
<tr><td valign="top"><a href="#spawn_link_worker_process-2">spawn_link_worker_process/2</a></td><td>Same as spawn_worker_process/2 except the the calling process
is linked to the worker process which is spawned.</td></tr>
<tr><td valign="top"><a href="#spawn_worker_process-2">spawn_worker_process/2</a></td><td>Creates a HTTP client process to the specified Host:Port which
is not part of the load balancing pool.</td></tr>
<tr><td valign="top"><a href="#start-0">start/0</a></td><td>Starts the ibrowse process without linking.</td></tr>
<tr><td valign="top"><a href="#start_link-0">start_link/0</a></td><td>Starts the ibrowse process linked to the calling process.</td></tr>
<tr><td valign="top"><a href="#stop-0">stop/0</a></td><td>Stop the ibrowse process.</td></tr>
<tr><td valign="top"><a href="#stop_worker_process-1">stop_worker_process/1</a></td><td>Terminate a worker process spawned using
spawn_worker_process/2 or spawn_link_worker_process/2.</td></tr>
<tr><td valign="top"><a href="#terminate-2">terminate/2</a></td><td></td></tr>
<tr><td valign="top"><a href="#trace_off-0">trace_off/0</a></td><td>Turn tracing off for the ibrowse process.</td></tr>
<tr><td valign="top"><a href="#trace_off-2">trace_off/2</a></td><td>Turn tracing OFF for all connections to the specified HTTP
server.</td></tr>
<tr><td valign="top"><a href="#trace_on-0">trace_on/0</a></td><td>Turn tracing on for the ibrowse process.</td></tr>
<tr><td valign="top"><a href="#trace_on-2">trace_on/2</a></td><td>Turn tracing on for all connections to the specified HTTP
server.</td></tr>
</table>
<h2><a name="functions">Function Details</a></h2>
<h3 class="function"><a name="code_change-3">code_change/3</a></h3>
<div class="spec">
<p><tt>code_change() -> term()</tt></p>
</div>
<h3 class="function"><a name="get_config_value-1">get_config_value/1</a></h3>
<div class="spec">
<p><tt>get_config_value() -> term()</tt></p>
</div><p>Internal export</p>
<h3 class="function"><a name="get_config_value-2">get_config_value/2</a></h3>
<div class="spec">
<p><tt>get_config_value() -> term()</tt></p>
</div><p>Internal export</p>
<h3 class="function"><a name="handle_call-3">handle_call/3</a></h3>
<div class="spec">
<p><tt>handle_call() -> term()</tt></p>
</div>
<h3 class="function"><a name="handle_cast-2">handle_cast/2</a></h3>
<div class="spec">
<p><tt>handle_cast() -> term()</tt></p>
</div>
<h3 class="function"><a name="handle_info-2">handle_info/2</a></h3>
<div class="spec">
<p><tt>handle_info() -> term()</tt></p>
</div>
<h3 class="function"><a name="init-1">init/1</a></h3>
<div class="spec">
<p><tt>init() -> term()</tt></p>
</div>
<h3 class="function"><a name="rescan_config-0">rescan_config/0</a></h3>
<div class="spec">
<p><tt>rescan_config() -> term()</tt></p>
</div><p>Clear current configuration for ibrowse and load from the file
ibrowse.conf in the IBROWSE_EBIN/../priv directory. Current
configuration is cleared only if the ibrowse.conf file is readable
using file:consult/1</p>
<h3 class="function"><a name="rescan_config-1">rescan_config/1</a></h3>
<div class="spec">
<p><tt>rescan_config() -> term()</tt></p>
</div>
<h3 class="function"><a name="send_req-3">send_req/3</a></h3>
<div class="spec">
<p><tt>send_req(Url::string(), Headers::<a href="#type-headerList">headerList()</a>, Method::<a href="#type-method">method()</a>) -> <a href="#type-response">response()</a></tt>
<ul class="definitions"><li><tt><a name="type-headerList">headerList()</a> = [{<a href="#type-header">header()</a>, <a href="#type-value">value()</a>}]</tt></li>
<li><tt><a name="type-header">header()</a> = atom() | string()</tt></li>
<li><tt><a name="type-value">value()</a> = term()</tt></li>
<li><tt><a name="type-method">method()</a> = get | post | head | options | put | delete | trace | mkcol | propfind | proppatch | lock | unlock | move | copy</tt></li>
<li><tt>Status = string()</tt></li>
<li><tt>ResponseHeaders = [<a href="#type-respHeader">respHeader()</a>]</tt></li>
<li><tt><a name="type-respHeader">respHeader()</a> = {<a href="#type-headerName">headerName()</a>, <a href="#type-headerValue">headerValue()</a>}</tt></li>
<li><tt><a name="type-headerName">headerName()</a> = string()</tt></li>
<li><tt><a name="type-headerValue">headerValue()</a> = string()</tt></li>
<li><tt><a name="type-response">response()</a> = {ok, Status, ResponseHeaders, ResponseBody} | {error, Reason}</tt></li>
<li><tt>ResponseBody = string() | {file, Filename}</tt></li>
<li><tt>Reason = term()</tt></li>
</ul></p>
</div><p>This is the basic function to send a HTTP request.
The Status return value indicates the HTTP status code returned by the webserver</p>
<h3 class="function"><a name="send_req-4">send_req/4</a></h3>
<div class="spec">
<p><tt>send_req(Url, Headers, Method::<a href="#type-method">method()</a>, Body::<a href="#type-body">body()</a>) -> <a href="#type-response">response()</a></tt>
<ul class="definitions"><li><tt><a name="type-body">body()</a> = [] | string() | binary() | <a href="#type-fun_arity_0">fun_arity_0()</a> | {<a href="#type-fun_arity_1">fun_arity_1()</a>, <a href="#type-initial_state">initial_state()</a>}</tt></li>
<li><tt><a name="type-initial_state">initial_state()</a> = term()</tt></li>
</ul></p>
</div><p>Same as send_req/3.
If a list is specified for the body it has to be a flat list. The body can also be a fun/0 or a fun/1. <br>
If fun/0, the connection handling process will repeatdely call the fun until it returns an error or eof. <pre>Fun() = {ok, Data} | eof</pre><br>
If fun/1, the connection handling process will repeatedly call the fun with the supplied state until it returns an error or eof. <pre>Fun(State) = {ok, Data} | {ok, Data, NewState} | eof</pre></p>
<h3 class="function"><a name="send_req-5">send_req/5</a></h3>
<div class="spec">
<p><tt>send_req(Url::string(), Headers::<a href="#type-headerList">headerList()</a>, Method::<a href="#type-method">method()</a>, Body::<a href="#type-body">body()</a>, Options::<a href="#type-optionList">optionList()</a>) -> <a href="#type-response">response()</a></tt>
<ul class="definitions"><li><tt><a name="type-optionList">optionList()</a> = [<a href="#type-option">option()</a>]</tt></li>
<li><tt><a name="type-option">option()</a> = {max_sessions, integer()} | {max_pipeline_size, integer()} | {trace, <a href="#type-boolean">boolean()</a>} | {is_ssl, <a href="#type-boolean">boolean()</a>} | {ssl_options, [SSLOpt]} | {pool_name, atom()} | {proxy_host, string()} | {proxy_port, integer()} | {proxy_user, string()} | {proxy_password, string()} | {use_absolute_uri, <a href="#type-boolean">boolean()</a>} | {basic_auth, {<a href="#type-username">username()</a>, <a href="#type-password">password()</a>}} | {cookie, string()} | {content_length, integer()} | {content_type, string()} | {save_response_to_file, <a href="#type-srtf">srtf()</a>} | {stream_to, <a href="#type-process">process()</a>} | {http_vsn, {MajorVsn, MinorVsn}} | {host_header, string()} | {transfer_encoding, {chunked, ChunkSize}}</tt></li>
<li><tt><a name="type-process">process()</a> = pid() | atom()</tt></li>
<li><tt><a name="type-username">username()</a> = string()</tt></li>
<li><tt><a name="type-password">password()</a> = string()</tt></li>
<li><tt>SSLOpt = term()</tt></li>
<li><tt>ChunkSize = integer()</tt></li>
<li><tt><a name="type-srtf">srtf()</a> = <a href="#type-boolean">boolean()</a> | <a href="#type-filename">filename()</a></tt></li>
<li><tt><a name="type-filename">filename()</a> = string()</tt></li>
</ul></p>
</div><p>Same as send_req/4.
For a description of SSL Options, look in the ssl manpage. If the
HTTP Version to use is not specified, the default is 1.1.
<br>
<p>The <code>host_header</code> is useful in the case where ibrowse is
connecting to a component such as <a href="http://www.stunnel.org">stunnel</a> which then sets up a
secure connection to a webserver. In this case, the URL supplied to
ibrowse must have the stunnel host/port details, but that won't
make sense to the destination webserver. This option can then be
used to specify what should go in the <code>Host</code> header in
the request.</p>
<ul>
<li>When both the options <code>save_response_to_file</code> and <code>stream_to</code>
are specified, the former takes precedence.</li>
<li>For the <code>save_response_to_file</code> option, the response body is saved to
file only if the status code is in the 200-299 range. If not, the response body is returned
as a string.</li>
<li>Whenever an error occurs in the processing of a request, ibrowse will return as much
information as it has, such as HTTP Status Code and HTTP Headers. When this happens, the response
is of the form <code>{error, {Reason, {stat_code, StatusCode}, HTTP_headers}}</code></li>
</ul></p>
<h3 class="function"><a name="send_req-6">send_req/6</a></h3>
<div class="spec">
<p><tt>send_req(Url, Headers::<a href="#type-headerList">headerList()</a>, Method::<a href="#type-method">method()</a>, Body::<a href="#type-body">body()</a>, Options::<a href="#type-optionList">optionList()</a>, Timeout) -> <a href="#type-response">response()</a></tt>
<ul class="definitions"><li><tt>Timeout = integer() | infinity</tt></li>
</ul></p>
</div><p>Same as send_req/5.
All timeout values are in milliseconds.</p>
<h3 class="function"><a name="send_req_direct-4">send_req_direct/4</a></h3>
<div class="spec">
<p><tt>send_req_direct() -> term()</tt></p>
</div><p>Same as send_req/3 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
<h3 class="function"><a name="send_req_direct-5">send_req_direct/5</a></h3>
<div class="spec">
<p><tt>send_req_direct() -> term()</tt></p>
</div><p>Same as send_req/4 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
<h3 class="function"><a name="send_req_direct-6">send_req_direct/6</a></h3>
<div class="spec">
<p><tt>send_req_direct() -> term()</tt></p>
</div><p>Same as send_req/5 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
<h3 class="function"><a name="send_req_direct-7">send_req_direct/7</a></h3>
<div class="spec">
<p><tt>send_req_direct() -> term()</tt></p>
</div><p>Same as send_req/6 except that the first argument is the PID
returned by spawn_worker_process/2 or spawn_link_worker_process/2</p>
<h3 class="function"><a name="set_dest-3">set_dest/3</a></h3>
<div class="spec">
<p><tt>set_dest() -> term()</tt></p>
</div><p>Deprecated. Use set_max_sessions/3 and set_max_pipeline_size/3
for achieving the same effect.</p>
<h3 class="function"><a name="set_max_pipeline_size-3">set_max_pipeline_size/3</a></h3>
<div class="spec">
<p><tt>set_max_pipeline_size(Host::string(), Port::integer(), Max::integer()) -> ok</tt></p>
</div><p>Set the maximum pipeline size for each connection to a specific Host:Port.</p>
<h3 class="function"><a name="set_max_sessions-3">set_max_sessions/3</a></h3>
<div class="spec">
<p><tt>set_max_sessions(Host::string(), Port::integer(), Max::integer()) -> ok</tt></p>
</div><p>Set the maximum number of connections allowed to a specific Host:Port.</p>
<h3 class="function"><a name="show_dest_status-2">show_dest_status/2</a></h3>
<div class="spec">
<p><tt>show_dest_status() -> term()</tt></p>
</div><p>Shows some internal information about load balancing to a
specified Host:Port. Info about workers spawned using
spawn_worker_process/2 or spawn_link_worker_process/2 is not
included.</p>
<h3 class="function"><a name="spawn_link_worker_process-2">spawn_link_worker_process/2</a></h3>
<div class="spec">
<p><tt>spawn_link_worker_process() -> term()</tt></p>
</div><p>Same as spawn_worker_process/2 except the the calling process
is linked to the worker process which is spawned.</p>
<h3 class="function"><a name="spawn_worker_process-2">spawn_worker_process/2</a></h3>
<div class="spec">
<p><tt>spawn_worker_process(Host::string(), Port::integer()) -> {ok, pid()}</tt></p>
</div><p>Creates a HTTP client process to the specified Host:Port which
is not part of the load balancing pool. This is useful in cases
where some requests to a webserver might take a long time whereas
some might take a very short time. To avoid getting these quick
requests stuck in the pipeline behind time consuming requests, use
this function to get a handle to a connection process. <br>
<b>Note:</b> Calling this function only creates a worker process. No connection
is setup. The connection attempt is made only when the first
request is sent via any of the send_req_direct/4,5,6,7 functions.<br>
<b>Note:</b> It is the responsibility of the calling process to control
pipeline size on such connections.
</p>
<h3 class="function"><a name="start-0">start/0</a></h3>
<div class="spec">
<p><tt>start() -> term()</tt></p>
</div><p>Starts the ibrowse process without linking. Useful when testing using the shell</p>
<h3 class="function"><a name="start_link-0">start_link/0</a></h3>
<div class="spec">
<p><tt>start_link() -> {ok, pid()}</tt></p>
</div><p>Starts the ibrowse process linked to the calling process. Usually invoked by the supervisor ibrowse_sup</p>
<h3 class="function"><a name="stop-0">stop/0</a></h3>
<div class="spec">
<p><tt>stop() -> term()</tt></p>
</div><p>Stop the ibrowse process. Useful when testing using the shell.</p>
<h3 class="function"><a name="stop_worker_process-1">stop_worker_process/1</a></h3>
<div class="spec">
<p><tt>stop_worker_process(Conn_pid::pid()) -> ok</tt></p>
</div><p>Terminate a worker process spawned using
spawn_worker_process/2 or spawn_link_worker_process/2. Requests in
progress will get the error response <pre>{error, closing_on_request}</pre></p>
<h3 class="function"><a name="terminate-2">terminate/2</a></h3>
<div class="spec">
<p><tt>terminate() -> term()</tt></p>
</div>
<h3 class="function"><a name="trace_off-0">trace_off/0</a></h3>
<div class="spec">
<p><tt>trace_off() -> term()</tt></p>
</div><p>Turn tracing off for the ibrowse process</p>
<h3 class="function"><a name="trace_off-2">trace_off/2</a></h3>
<div class="spec">
<p><tt>trace_off(Host, Port) -> term()</tt></p>
</div><p>Turn tracing OFF for all connections to the specified HTTP
server.</p>
<h3 class="function"><a name="trace_on-0">trace_on/0</a></h3>
<div class="spec">
<p><tt>trace_on() -> term()</tt></p>
</div><p>Turn tracing on for the ibrowse process</p>
<h3 class="function"><a name="trace_on-2">trace_on/2</a></h3>
<div class="spec">
<p><tt>trace_on(Host, Port) -> term()</tt>
<ul class="definitions"><li><tt>Host = string()</tt></li>
<li><tt>Port = integer()</tt></li>
</ul></p>
</div><p>Turn tracing on for all connections to the specified HTTP
server. Host is whatever is specified as the domain name in the URL</p>
<hr>
<div class="navbar"><a name="#navbar_bottom"></a><table width="100%" border="0" cellspacing="0" cellpadding="2" summary="navigation bar"><tr><td><a href="overview-summary.html" target="overviewFrame">Overview</a></td><td><a href="http://www.erlang.org/"><img src="erlang.png" align="right" border="0" alt="erlang logo"></a></td></tr></table></div>
<p><i>Generated by EDoc, Mar 27 2008, 01:20:55.</i></p>
</body>
</html>