Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 206 lines (134 sloc) 12.358 kB
ff01dce @ferd Initial commit
authored
1 # Dispcount #
2
630d6f9 @ferd Adding README details and removing warnings
authored
3 Dispcount is an attempt at making more efficient resource dispatching than usual Erlang pool approaches based on a single process receiving messages from everyone and possibly getting overloaded when demand is too high, or at least seeing slower and slower response times. It is still a fairly young library and we expect it to get more stable with time.
ff01dce @ferd Initial commit
authored
4
5 ## When should I use dispcount? ##
6
7 There have been a few characteristics assumed to be present for the design of dispcount:
8
b9cd1bd @ferd Clarifying the statistical compromise
authored
9 - resources are limited, but the demand for them is far superior to their availability.
ff01dce @ferd Initial commit
authored
10 - requests for resources are *always* incoming
11 - because of the previous point, it is possible and prefered to simply not queue requests for busy resources, but instantly return. Newer requests will take their spot
12 - low latency to know whether or not a resource is available is more important than being able to get all queries to run.
13
630d6f9 @ferd Adding README details and removing warnings
authored
14 If you cannot afford to ignore a query and wish to eventually serve every one of them, dispcount might not be for you. Otherwise, you'll need to queue them yourself because all it does is grant you a resource or tell you it's busy.
ff01dce @ferd Initial commit
authored
15
b9cd1bd @ferd Clarifying the statistical compromise
authored
16 Also note that the dispatching of resources is done on a hashing basis and doesn't guarantee that all resources are to be allocated before showing a 'busy' response. As mentioned earlier, dispcount makes the assumption that there are limited resources and the demand is superior to their availability; the more requests for resources there are, the better the distribution should be. See 'how does it work' for more details.
17
ff01dce @ferd Initial commit
authored
18 ## How to build ##
19
20 `$ ./rebar compile`
21
22 ## Running tests ##
23
24 Run the small Common Test suite with:
25
a6e248c @ferd minimal fix on test running command
authored
26 `$ ./rebar compile && ./rebar ct`
ff01dce @ferd Initial commit
authored
27
28 ## How to use dispcount ##
29
30 First start the application:
31
32 `application:start(dispcount).`
33
34 When resources need to be dispatched, a dispatcher has to be started:
35
36 ok = dispcount:start_dispatch(
37 ref_dispatcher,
38 {ref_dispatch, []},
39 [{restart,permanent},{shutdown,4000},
40 {maxr,10},{maxt,60},{resources,10}]
41 )
42
43 The general form is:
44
45 ok = dispcount:start_dispatch(
46 DispatcherName,
47 {CallbackMod, Arg},
48 [{restart,Type},{shutdown,Timeout},
49 {maxr,X},{maxt,Y},{resources,Num}]
50 )
51
52 The `restart`, `shutdown`, `maxr`, and `maxt` values allow to configure the supervisor that will take care of that dispatcher. The `resources` value lets you set how many 'things' you want available. If you were handling HTTP sockets, you could use 200 connections by putting `{resources,200}`. Skip to the next section to see how to write your own dispatcher callback module.
53
54 The dispatcher is then put under the supervision structure of the dispcount application. To be able to further access resources, you need to fetch information related to the dispatcher:
55
56 {ok, Info} = dispcount:dispatcher_info(ref_dispatcher)
57
58 This is because we want to reduce the number of calls to configuration spots and centralized points in a node. As such, you should call this function in the supervisor of whatever is going to call the dispatcher, and share the value to all children if possible. That way, a basic blob of content is going to be shared without any cost to all processes.
59
60 Using this `Info` value, calls to checkout resources can be made:
61
62 case dispcount:checkout(Info) of
63 {ok, CheckinReference, Resource} ->
64 timer:sleep(10),
65 dispcount:checkin(Info, CheckinReference, Resource);
66 {error, busy} ->
67 give_up
68 end
69
f0412e8 @ferd Fixing issue #3 - adding timeouts to checkout
authored
70 And that's the gist of it. Note that `dispcount:checkout` can also take a timeout value as a second argument. The value is similar to whatever OTP behaviours accept (default: 5000ms, can be the atom `infinity`).
ff01dce @ferd Initial commit
authored
71
72 ## Writing a dispatcher callback module ##
73
74 Each dispatcher to allow to lend resources is written as a callback for a custom behaviour. Here's an example (tested) callback module that simply returns references:
75
76 -module(ref_dispatch).
77 -behaviour(dispcount).
78 -export([init/1, checkout/2, checkin/2, handle_info/2, dead/1,
79 terminate/2, code_change/3]).
80
81 init([]) ->
82 {ok, make_ref()}.
83
84 This one works a bit like a gen\_server. You have arguments passed and return `{ok,State}`. The state will then be carried around for the subsequent calls.
85
86 The next function is `checkout`:
87
88 checkout(_From, Ref) ->
89 {ok, Ref, undefined}.
90
4bebd74 @ferd damn them typoes
authored
91 By default, the behaviour takes care of making sure only valid requests for a checkout (resources aren't busy) are going through. The `_From` variable is the pid of the requester of a resource. This is useful if you need to change things like a socket's controlling process or a port's controller. Then, you only need to return a resource by doing `{ok, Resource, NewState}`, and the caller will see `{ok, Reference, Resource}`. The `Reference` is a token added in by dispcount and is needed to chick the resource back in. Other things to return are `{error, Reason, NewState}`, which will return `{error, Reason}` to the caller.
ff01dce @ferd Initial commit
authored
92
93 Finally, you can return `{stop, Reason, NewState}` to terminate the resource watcher. Note that this is risky because of how things work (see the relevant section for this later in this README).
94
95 To check resources back in, the behaviour needs to implement the following:
96
97 checkin(Ref, undefined) ->
98 {ok, Ref};
99 checkin(SomeRef, Ref) ->
100 {ignore, Ref}.
101
102 In this case, what happens is that we make sure that the resource that is being sent back to us is the right one. The first function clause makes sure that we only receive a reference after we've distributed one, and we then accept that one. If we receive extraneous references (maybe someone called the `checkin/3` function twice?), we ignore the result.
103
f53efec @ferd Fixing the doc for checkin calls.
authored
104 The second clause here is entirely optional and defensive programming. Note that checking a resource in is an asynchronous operation.
ff01dce @ferd Initial commit
authored
105
106 The next call is the `dead/1` function:
107
108 dead(undefined) ->
109 {ok, make_ref()}.
110
f4e4458 @ferd Fixing a minor typo (although aren't all typos major?!)
authored
111 `dead(State)` is called whenever the process that checked out a given resource has died. This is because dispcount automatically monitors them so you don't need to do it yourself. If it sees the resource owner died, it calls that function.
ff01dce @ferd Initial commit
authored
112
113 This lets you create a new instance of a resource to distribute later on, if required or possible. As an example, if we were to use a permanent connection to a database as a resource, then this is where we'd set a new connection up and then keep going as if nothing went wrong.
114
115 You can also receive unexpected messages to your process, if you felt like implementing your own side-protocols or whatever:
116
117 handle_info(_Msg, State) ->
118 {ok, State}.
119
120 And finally, you benefit from a traditional OTP `terminate/2` function, and the related `code_change/3`.
121
122 terminate(_Reason, _State) ->
123 ok.
124
125 code_change(_OldVsn, State, _Extra) ->
126 {ok, State}.
127
128 Here's a similar callback module to handle HTTP sockets (untested):
129
130 -module(http_dispatch).
131 -behaviour(dispcount).
132 -export([init/1, checkout/2, checkin/2, handle_info/2, dead/1, terminate/2, code_change/3]).
133
134 -record(state, {resource, given=false, port}).
135
136 init([{port,Num}]) ->
137 {ok,Socket} = gen_tcp:connect({127,0,0,1}, Num, [binary]),
138 {ok, #state{resource=Socket, port=Num}}.
139
140 %% just in case, but that should never happen anyway :V I'm paranoid!
141 checkout(_From, State = #state{given=true}) ->
142 {error, busy, State};
143 checkout(From, State = #state{resource=Socket}) ->
144 gen_tcp:controlling_process(Socket, From),
bb96186 @ferd Fixing demo example in README
authored
145 %% We give our own pid back so that the client can make this
146 %% callback module the controlling process again before
147 %% handing it back.
148 {ok, {self(), Socket}, State#state{given=true}}.
ff01dce @ferd Initial commit
authored
149
150 checkin(Socket, State = #state{resource=Socket, given=true}) ->
bb96186 @ferd Fixing demo example in README
authored
151 %% This assumes the client made us the controlling process again.
152 %% This might be done via a client lib wrapping dispcount calls of
153 %% some sort.
ff01dce @ferd Initial commit
authored
154 {ok, State#state{given=false}};
155 checkin(_Socket, State) ->
156 %% The socket doesn't match the one we had -- an error happened somewhere
157 {ignore, State}.
158
159 dead(State) ->
160 %% aw shoot, someone lost our resource, we gotta create a new one:
161 {ok, NewSocket} = gen_tcp:connect({127,0,0,1}, State#state.port, [binary]),
162 {ok, State#state{resource=NewSocket,given=false}}.
163 %% alternatively:
164 %% {stop, Reason, State}
165
166 handle_info(_Msg, State) ->
167 %% something unexpected with the TCP connection if we set it to active,once???
168 {ok, State}.
169
170 terminate(_Reason, _State) ->
171 %% let the GC clean the socket.
172 ok.
173
174 code_change(_OldVsn, State, _Extra) ->
175 {ok, State}.
176
177 ## How does it work ##
178
179 What killed most of the pool and dispatch systems I used before was the amount of messaging required to make things work. When many thousands of processes require information from a central point at once, performance would quickly degrade as soon as the protocol had some messaging involved at its core.
180
181 We'd see mailbox queue build-up, busy schedulers, and booming memory. Dispcount tries to solve the problem by using the ubiquitous Erlang optimization tool: ETS tables.
182
183 The core concept of dispcount is based on two ETS tables: a dispatch table (write-only) and a worker matchup table (read-only). Two tables because what costs the most performance with ETS in terms of concurrency is switching between reading and writing.
184
b9cd1bd @ferd Clarifying the statistical compromise
authored
185 In each of the table, `N` entries are added: one for each resource available, matching with a process that manages that resource (a *watcher*). Persistent hashing of the resources allows to dispatch queries uniformly to all of these watchers. Once you know which watcher your request is dedicated to, the dispatch table is called into action. The persistent hashing does mean that it is not possible to guarantee that all free resources will be allocated before 'busy' messages start showing, but only that at a sufficiently high level of demand, the distribution should be roughly equal to all watchers, obtaining a full dispatching of resources.
ff01dce @ferd Initial commit
authored
186
187 The dispatch table manages to allow both reads and writes while remaining write-only. The trick is to use the `ets:update_counter` functions, which atomically increment a counter and return the value, although the operation is only writing and communicating a minimum of information.
188
189 The gist of the idea is that you can only get the permission to message the watcher if you're the first one to increment the counter. Other processes that try to do it just instantly give up. This guarantees that a single caller at a time has the permission to message a given worker, a bit like a mutex, but implemented efficiently (for Erlang, that is).
190
191 Then the lookup table comes in action; because we have the permission to message a watcher, we look up its pid, and then send a message.
192
193 Whenever we check a resource back in or the process that acquired it dies, the counter is reset to 0 and a new request can come in and take its place.
194
195 Generally, this allows us to move the bottleneck of similar applications away from a single process and its mailbox, to an evenly distributed number of workers. Then the next bottleneck will be the ETS tables (both set with read and write concurrency options), which are somewhat less likely to be as much of a hot spot.
196
2f56d2b @ferd Notice for R14 compatibility added to README
authored
197 ## I get crashes in R14, help me! ##
198
199 The error you see is likely `{start_spec,{invalid_shutdown,infinity}}`. This is due to Erlang/OTP releases R14 (or generally, versions prior to R15) sometimes disliking the atom `infinity` in child specifications. If you have this problem, use the branch `r14` instead of `master`. It changes the `infinity` value to `120000` (arbitrarily long value), which should hopefully make things work smoothly (it did for me).
200
ff01dce @ferd Initial commit
authored
201 ## What's left to do? ##
202
203 - Adding a function call to allow the transfer of ownership from a process to another one to avoid messing with monitoring in the callback module.
204 - Testing to make sure the callback modules can be updated with OTP relups and appups. This is so far untested.
bb96186 @ferd Fixing demo example in README
authored
205 - Allowing dynamic resizing of pools.
Something went wrong with that request. Please try again.