Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 371 lines (321 sloc) 11.936 kB
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
1 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
17 # 1. Simply use ActiveRecord::Base.connection as in pre-connection-pooled
18 # ActiveRecord. Eventually, when you're done with the connection and
19 # wish it to be returned to the pool, you call
20 # ActiveRecord::Base.connection_pool.release_thread_connection.
21 # 2. Manually check out a connection from the pool with
22 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
23 # returning this connection to the pool when finished by calling
24 # ActiveRecord::Base.connection_pool.checkin(connection).
25 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
26 # obtains a connection, yields it as the sole argument to the block,
27 # and returns it to the pool after the block completes.
28 class AbstractConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
29 # Factory method for connection pools.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
30 # Determines pool type to use based on contents of connection
31 # specification. Additional options for connection specification:
32 #
33 # * +pool+: number indicating size of fixed connection pool to use
34 # * +wait_timeout+ (optional): number of seconds to block and wait
35 # for a connection before giving up and raising a timeout error.
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
36 def self.create(spec)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
37 if spec.config[:pool] && spec.config[:pool].to_i > 0
38 ConnectionPool.new(spec)
39 else
40 ConnectionPerThread.new(spec)
41 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
42 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
43
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
44 delegate :verification_timeout, :to => "::ActiveRecord::Base"
45 attr_reader :spec
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
46
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
47 def initialize(spec)
48 @spec = spec
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
49 # The cache of reserved connections mapped to threads
50 @reserved_connections = {}
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
51 # The mutex used to synchronize pool access
52 @connection_mutex = Monitor.new
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
53 end
54
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
55 # Retrieve the connection associated with the current thread, or call
56 # #checkout to obtain one if necessary.
57 #
58 # #connection can be called any number of times; the connection is
59 # held in a hash keyed by the thread id.
60 def connection
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
61 if conn = @reserved_connections[active_connection_name]
62 conn.verify!(verification_timeout)
63 conn
64 else
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
65 @reserved_connections[active_connection_name] = checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
66 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
67 end
68
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
69 # Signal that the thread is finished with the current connection.
70 # #release_thread_connection releases the connection-thread association
71 # and returns the connection to the pool.
72 def release_thread_connection
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
73 conn = @reserved_connections.delete(active_connection_name)
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
74 checkin conn if conn
75 end
76
77 # Reserve a connection, and yield it to a block. Ensure the connection is
78 # checked back in when finished.
79 def with_connection
80 conn = checkout
81 yield conn
82 ensure
83 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
84 end
85
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
86 # Returns true if a connection has already been opened.
87 def connected?
88 !connections.empty?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
89 end
90
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
91 # Disconnect all connections in the pool.
92 def disconnect!
93 @reserved_connections.each do |name,conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
94 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
95 end
96 connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
97 conn.disconnect!
98 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
99 @reserved_connections = {}
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
100 end
101
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
102 # Clears the cache which maps classes
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
103 def clear_reloadable_connections!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
104 @reserved_connections.each do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
105 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
106 end
107 @reserved_connections = {}
108 connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
109 if conn.requires_reloading?
110 conn.disconnect!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
111 remove_connection conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
112 end
113 end
114 end
115
116 # Verify active connections.
117 def verify_active_connections! #:nodoc:
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
118 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
119 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
120 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
121 connections.each do |connection|
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
122 connection.verify!(verification_timeout)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
123 end
124 end
125
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
126 # Check-out a database connection from the pool.
127 def checkout
128 raise NotImplementedError, "checkout is an abstract method"
129 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
130
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 # Check-in a database connection back into the pool.
132 def checkin(connection)
133 raise NotImplementedError, "checkin is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
134 end
135
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
136 def remove_connection(conn)
137 raise NotImplementedError, "remove_connection is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
138 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
139 private :remove_connection
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
140
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
141 # Array containing all connections (reserved or available) in the pool.
142 def connections
143 raise NotImplementedError, "connections is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
144 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
145 private :connections
146
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
147 synchronize :connection, :release_thread_connection,
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
148 :clear_reloadable_connections!, :verify_active_connections!,
149 :connected?, :disconnect!, :with => :@connection_mutex
150
151 private
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
152 def new_connection
153 config = spec.config.reverse_merge(:allow_concurrency => ActiveRecord::Base.allow_concurrency)
154 ActiveRecord::Base.send(spec.adapter_method, config)
155 end
156
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
157 def active_connection_name #:nodoc:
158 Thread.current.object_id
159 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
160
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
161 # Remove stale threads from the cache.
162 def remove_stale_cached_threads!(cache, &block)
163 keys = Set.new(cache.keys)
164
165 Thread.list.each do |thread|
166 keys.delete(thread.object_id) if thread.alive?
167 end
168 keys.each do |key|
169 next unless cache.has_key?(key)
170 block.call(key, cache[key])
171 cache.delete(key)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
172 end
173 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
174 end
175
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
176 # ConnectionPerThread is a simple implementation: always create/disconnect
177 # on checkout/checkin, and use the base class reserved connections hash to
178 # manage the per-thread connections.
179 class ConnectionPerThread < AbstractConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
180 def active_connection
181 @reserved_connections[active_connection_name]
182 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
183
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
184 def active_connections; @reserved_connections; end
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
185
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
186 def checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
187 new_connection
188 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
189
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
190 def checkin(conn)
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
191 conn.disconnect!
192 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
193
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
194 private
195 def connections
196 @reserved_connections.values
197 end
198
199 def remove_connection(conn)
200 @reserved_connections.delete_if {|k,v| v == conn}
201 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
202 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
203
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
204 # ConnectionPool provides a full, fixed-size connection pool with timed
205 # waits when the pool is exhausted.
206 class ConnectionPool < AbstractConnectionPool
207 def initialize(spec)
208 super
209 # default 5 second timeout
210 @timeout = spec.config[:wait_timeout] || 5
211 @size = spec.config[:pool].to_i
212 @queue = @connection_mutex.new_cond
213 @connections = []
214 @checked_out = []
215 end
216
217 def checkout
218 # Checkout an available connection
219 conn = @connection_mutex.synchronize do
220 if @connections.length < @size
221 checkout_new_connection
222 elsif @checked_out.size < @connections.size
223 checkout_existing_connection
224 end
225 end
226 return conn if conn
227
228 # No connections available; wait for one
229 @connection_mutex.synchronize do
230 if @queue.wait(@timeout)
231 checkout_existing_connection
232 else
233 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
234 end
235 end
236 end
237
238 def checkin(conn)
239 @connection_mutex.synchronize do
240 @checked_out -= conn
241 @queue.signal
242 end
243 end
244
245 private
246 def checkout_new_connection
247 c = new_connection
248 @connections << c
249 @checked_out << c
250 c
251 end
252
253 def checkout_existing_connection
254 c = [@connections - @checked_out].first
255 @checked_out << c
256 c
257 end
258
259 def connections
260 @connections
261 end
262
263 def remove_connection(conn)
264 @connections.delete conn
265 end
266 end
267
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
268 module ConnectionHandlerMethods
269 def initialize(pools = {})
270 @connection_pools = pools
271 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
272
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
273 def connection_pools
274 @connection_pools ||= {}
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
275 end
276
277 def establish_connection(name, spec)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
278 @connection_pools[name] = ConnectionAdapters::AbstractConnectionPool.create(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
279 end
280
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
281 # for internal use only and for testing;
282 # only works with ConnectionPerThread pool class
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
283 def active_connections #:nodoc:
284 @connection_pools.inject({}) do |hash,kv|
285 hash[kv.first] = kv.last.active_connection
286 hash.delete(kv.first) unless hash[kv.first]
287 hash
288 end
289 end
290
291 # Clears the cache which maps classes to connections.
292 def clear_active_connections!
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
293 @connection_pools.each_value {|pool| pool.release_thread_connection }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
294 end
295
296 # Clears the cache which maps classes
297 def clear_reloadable_connections!
298 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
299 end
300
301 def clear_all_connections!
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
302 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
303 end
304
305 # Verify active connections.
306 def verify_active_connections! #:nodoc:
307 @connection_pools.each_value {|pool| pool.verify_active_connections!}
308 end
309
310 # Locate the connection of the nearest super class. This can be an
311 # active or defined connection: if it is the latter, it will be
312 # opened and set as the active connection for the class it was defined
313 # for (not necessarily the current class).
314 def retrieve_connection(klass) #:nodoc:
315 pool = retrieve_connection_pool(klass)
316 (pool && pool.connection) or raise ConnectionNotEstablished
317 end
318
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
319 # Returns true if a connection that's accessible to this class has
320 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
321 def connected?(klass)
322 retrieve_connection_pool(klass).connected?
323 end
324
325 # Remove the connection for this class. This will close the active
326 # connection and the defined connection (if they exist). The result
327 # can be used as an argument for establish_connection, for easily
328 # re-establishing the connection.
329 def remove_connection(klass)
330 pool = @connection_pools[klass.name]
331 @connection_pools.delete_if { |key, value| value == pool }
332 pool.disconnect! if pool
333 pool.spec.config if pool
334 end
335
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
336 def retrieve_connection_pool(klass)
337 loop do
338 pool = @connection_pools[klass.name]
339 return pool if pool
340 return nil if ActiveRecord::Base == klass
341 klass = klass.superclass
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
342 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
343 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
344 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
345
346 # This connection handler is not thread-safe, as it does not protect access
347 # to the underlying connection pools.
348 class SingleThreadConnectionHandler
349 include ConnectionHandlerMethods
350 end
351
352 # This connection handler is thread-safe. Each access or modification of a thread
353 # pool is synchronized by an internal monitor.
354 class MultipleThreadConnectionHandler
355 attr_reader :connection_pools_lock
356 include ConnectionHandlerMethods
357
358 def initialize(pools = {})
359 super
360 @connection_pools_lock = Monitor.new
361 end
362
363 # Apply monitor to all public methods that access the pool.
364 synchronize :establish_connection, :retrieve_connection,
365 :connected?, :remove_connection, :active_connections,
366 :clear_active_connections!, :clear_reloadable_connections!,
367 :clear_all_connections!, :verify_active_connections!,
368 :with => :connection_pools_lock
369 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
370 end
371 end
Something went wrong with that request. Please try again.