Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 371 lines (321 sloc) 11.936 kb
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
1 require 'monitor'
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
17 # 1. Simply use ActiveRecord::Base.connection as in pre-connection-pooled
18 # ActiveRecord. Eventually, when you're done with the connection and
19 # wish it to be returned to the pool, you call
20 # ActiveRecord::Base.connection_pool.release_thread_connection.
21 # 2. Manually check out a connection from the pool with
22 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
23 # returning this connection to the pool when finished by calling
24 # ActiveRecord::Base.connection_pool.checkin(connection).
25 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
26 # obtains a connection, yields it as the sole argument to the block,
27 # and returns it to the pool after the block completes.
28 class AbstractConnectionPool
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
29 # Factory method for connection pools.
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
30 # Determines pool type to use based on contents of connection
31 # specification. Additional options for connection specification:
32 #
33 # * +pool+: number indicating size of fixed connection pool to use
34 # * +wait_timeout+ (optional): number of seconds to block and wait
35 # for a connection before giving up and raising a timeout error.
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
36 def self.create(spec)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
37 if spec.config[:pool] && spec.config[:pool].to_i > 0
38 ConnectionPool.new(spec)
39 else
40 ConnectionPerThread.new(spec)
41 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
42 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
43
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
44 delegate :verification_timeout, :to => "::ActiveRecord::Base"
45 attr_reader :spec
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
46
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
47 def initialize(spec)
48 @spec = spec
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
49 # The cache of reserved connections mapped to threads
50 @reserved_connections = {}
cab76ce Nick Sieger Add synchronization to connection pool also
nicksieger authored
51 # The mutex used to synchronize pool access
52 @connection_mutex = Monitor.new
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
53 end
54
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
55 # Retrieve the connection associated with the current thread, or call
56 # #checkout to obtain one if necessary.
57 #
58 # #connection can be called any number of times; the connection is
59 # held in a hash keyed by the thread id.
60 def connection
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
61 if conn = @reserved_connections[active_connection_name]
62 conn.verify!(verification_timeout)
63 conn
64 else
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
65 @reserved_connections[active_connection_name] = checkout
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
66 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
67 end
68
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
69 # Signal that the thread is finished with the current connection.
70 # #release_thread_connection releases the connection-thread association
71 # and returns the connection to the pool.
72 def release_thread_connection
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
73 conn = @reserved_connections.delete(active_connection_name)
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
74 checkin conn if conn
75 end
76
77 # Reserve a connection, and yield it to a block. Ensure the connection is
78 # checked back in when finished.
79 def with_connection
80 conn = checkout
81 yield conn
82 ensure
83 checkin conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
84 end
85
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
86 # Returns true if a connection has already been opened.
87 def connected?
88 !connections.empty?
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
89 end
90
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
91 # Disconnect all connections in the pool.
92 def disconnect!
93 @reserved_connections.each do |name,conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
94 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
95 end
96 connections.each do |conn|
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
97 conn.disconnect!
98 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
99 @reserved_connections = {}
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
100 end
101
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
102 # Clears the cache which maps classes
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
103 def clear_reloadable_connections!
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
104 @reserved_connections.each do |name, conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
105 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
106 end
107 @reserved_connections = {}
108 connections.each do |conn|
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
109 if conn.requires_reloading?
110 conn.disconnect!
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
111 remove_connection conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
112 end
113 end
114 end
115
116 # Verify active connections.
117 def verify_active_connections! #:nodoc:
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
118 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
119 checkin conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
120 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
121 connections.each do |connection|
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
122 connection.verify!(verification_timeout)
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
123 end
124 end
125
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
126 # Check-out a database connection from the pool.
127 def checkout
128 raise NotImplementedError, "checkout is an abstract method"
129 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
130
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 # Check-in a database connection back into the pool.
132 def checkin(connection)
133 raise NotImplementedError, "checkin is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
134 end
135
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
136 def remove_connection(conn)
137 raise NotImplementedError, "remove_connection is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
138 end
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
139 private :remove_connection
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
140
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
141 # Array containing all connections (reserved or available) in the pool.
142 def connections
143 raise NotImplementedError, "connections is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
144 end
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
145 private :connections
146
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
147 synchronize :connection, :release_thread_connection,
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
148 :clear_reloadable_connections!, :verify_active_connections!,
149 :connected?, :disconnect!, :with => :@connection_mutex
150
151 private
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
152 def new_connection
153 config = spec.config.reverse_merge(:allow_concurrency => ActiveRecord::Base.allow_concurrency)
154 ActiveRecord::Base.send(spec.adapter_method, config)
155 end
156
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
157 def active_connection_name #:nodoc:
158 Thread.current.object_id
159 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
160
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
161 # Remove stale threads from the cache.
162 def remove_stale_cached_threads!(cache, &block)
163 keys = Set.new(cache.keys)
164
165 Thread.list.each do |thread|
166 keys.delete(thread.object_id) if thread.alive?
167 end
168 keys.each do |key|
169 next unless cache.has_key?(key)
170 block.call(key, cache[key])
171 cache.delete(key)
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
172 end
173 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
174 end
175
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
176 # ConnectionPerThread is a simple implementation: always create/disconnect
177 # on checkout/checkin, and use the base class reserved connections hash to
178 # manage the per-thread connections.
179 class ConnectionPerThread < AbstractConnectionPool
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
180 def active_connection
181 @reserved_connections[active_connection_name]
182 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
183
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
184 def active_connections; @reserved_connections; end
cab76ce Nick Sieger Add synchronization to connection pool also
nicksieger authored
185
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
186 def checkout
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
187 new_connection
188 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
189
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
190 def checkin(conn)
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
191 conn.disconnect!
192 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
193
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
194 private
195 def connections
196 @reserved_connections.values
197 end
198
199 def remove_connection(conn)
200 @reserved_connections.delete_if {|k,v| v == conn}
201 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
202 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
203
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
204 # ConnectionPool provides a full, fixed-size connection pool with timed
205 # waits when the pool is exhausted.
206 class ConnectionPool < AbstractConnectionPool
207 def initialize(spec)
208 super
209 # default 5 second timeout
210 @timeout = spec.config[:wait_timeout] || 5
211 @size = spec.config[:pool].to_i
212 @queue = @connection_mutex.new_cond
213 @connections = []
214 @checked_out = []
215 end
216
217 def checkout
218 # Checkout an available connection
219 conn = @connection_mutex.synchronize do
220 if @connections.length < @size
221 checkout_new_connection
222 elsif @checked_out.size < @connections.size
223 checkout_existing_connection
224 end
225 end
226 return conn if conn
227
228 # No connections available; wait for one
229 @connection_mutex.synchronize do
230 if @queue.wait(@timeout)
231 checkout_existing_connection
232 else
233 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
234 end
235 end
236 end
237
238 def checkin(conn)
239 @connection_mutex.synchronize do
240 @checked_out -= conn
241 @queue.signal
242 end
243 end
244
245 private
246 def checkout_new_connection
247 c = new_connection
248 @connections << c
249 @checked_out << c
250 c
251 end
252
253 def checkout_existing_connection
254 c = [@connections - @checked_out].first
255 @checked_out << c
256 c
257 end
258
259 def connections
260 @connections
261 end
262
263 def remove_connection(conn)
264 @connections.delete conn
265 end
266 end
267
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
268 module ConnectionHandlerMethods
269 def initialize(pools = {})
270 @connection_pools = pools
271 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
272
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
273 def connection_pools
274 @connection_pools ||= {}
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
275 end
276
277 def establish_connection(name, spec)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
278 @connection_pools[name] = ConnectionAdapters::AbstractConnectionPool.create(spec)
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
279 end
280
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
281 # for internal use only and for testing;
282 # only works with ConnectionPerThread pool class
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
283 def active_connections #:nodoc:
284 @connection_pools.inject({}) do |hash,kv|
285 hash[kv.first] = kv.last.active_connection
286 hash.delete(kv.first) unless hash[kv.first]
287 hash
288 end
289 end
290
291 # Clears the cache which maps classes to connections.
292 def clear_active_connections!
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
293 @connection_pools.each_value {|pool| pool.release_thread_connection }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
294 end
295
296 # Clears the cache which maps classes
297 def clear_reloadable_connections!
298 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
299 end
300
301 def clear_all_connections!
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
302 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
303 end
304
305 # Verify active connections.
306 def verify_active_connections! #:nodoc:
307 @connection_pools.each_value {|pool| pool.verify_active_connections!}
308 end
309
310 # Locate the connection of the nearest super class. This can be an
311 # active or defined connection: if it is the latter, it will be
312 # opened and set as the active connection for the class it was defined
313 # for (not necessarily the current class).
314 def retrieve_connection(klass) #:nodoc:
315 pool = retrieve_connection_pool(klass)
316 (pool && pool.connection) or raise ConnectionNotEstablished
317 end
318
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
319 # Returns true if a connection that's accessible to this class has
320 # already been opened.
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
321 def connected?(klass)
322 retrieve_connection_pool(klass).connected?
323 end
324
325 # Remove the connection for this class. This will close the active
326 # connection and the defined connection (if they exist). The result
327 # can be used as an argument for establish_connection, for easily
328 # re-establishing the connection.
329 def remove_connection(klass)
330 pool = @connection_pools[klass.name]
331 @connection_pools.delete_if { |key, value| value == pool }
332 pool.disconnect! if pool
333 pool.spec.config if pool
334 end
335
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
336 def retrieve_connection_pool(klass)
337 loop do
338 pool = @connection_pools[klass.name]
339 return pool if pool
340 return nil if ActiveRecord::Base == klass
341 klass = klass.superclass
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
342 end
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
343 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
344 end
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
345
346 # This connection handler is not thread-safe, as it does not protect access
347 # to the underlying connection pools.
348 class SingleThreadConnectionHandler
349 include ConnectionHandlerMethods
350 end
351
352 # This connection handler is thread-safe. Each access or modification of a thread
353 # pool is synchronized by an internal monitor.
354 class MultipleThreadConnectionHandler
355 attr_reader :connection_pools_lock
356 include ConnectionHandlerMethods
357
358 def initialize(pools = {})
359 super
360 @connection_pools_lock = Monitor.new
361 end
362
363 # Apply monitor to all public methods that access the pool.
364 synchronize :establish_connection, :retrieve_connection,
365 :connected?, :remove_connection, :active_connections,
366 :clear_active_connections!, :clear_reloadable_connections!,
367 :clear_all_connections!, :verify_active_connections!,
368 :with => :connection_pools_lock
369 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
370 end
371 end
Something went wrong with that request. Please try again.