Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 386 lines (335 sloc) 12.638 kb
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
1 require 'monitor'
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
17 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
18 # earlier (pre-connection-pooling). Eventually, when you're done with
19 # the connection(s) and wish it to be returned to the pool, you call
20 # ActiveRecord::Base.clear_active_connections!. This will be the
21 # default behavior for ActiveRecord when used in conjunction with
22 # ActionPack's request handling cycle.
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
23 # 2. Manually check out a connection from the pool with
24 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
25 # returning this connection to the pool when finished by calling
26 # ActiveRecord::Base.connection_pool.checkin(connection).
27 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
28 # obtains a connection, yields it as the sole argument to the block,
29 # and returns it to the pool after the block completes.
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
30 class ConnectionPool
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
31 # Factory method for connection pools.
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
32 # Determines pool type to use based on contents of connection
33 # specification. Additional options for connection specification:
34 #
35 # * +pool+: number indicating size of fixed connection pool to use
36 # * +wait_timeout+ (optional): number of seconds to block and wait
37 # for a connection before giving up and raising a timeout error.
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
38 def self.create(spec)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
39 if spec.config[:pool] && spec.config[:pool].to_i > 0
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
40 FixedSizeConnectionPool.new(spec)
41 elsif spec.config[:jndi] # JRuby appserver datasource pool; passthrough
42 NewConnectionEveryTime.new(spec)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
43 else
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
44 CachedConnectionPerThread.new(spec)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
45 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
46 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
47
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
48 delegate :verification_timeout, :to => "::ActiveRecord::Base"
49 attr_reader :spec
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
50
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
51 def initialize(spec)
52 @spec = spec
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
53 # The cache of reserved connections mapped to threads
54 @reserved_connections = {}
cab76ce Nick Sieger Add synchronization to connection pool also
nicksieger authored
55 # The mutex used to synchronize pool access
56 @connection_mutex = Monitor.new
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
57 end
58
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
59 # Retrieve the connection associated with the current thread, or call
60 # #checkout to obtain one if necessary.
61 #
62 # #connection can be called any number of times; the connection is
63 # held in a hash keyed by the thread id.
64 def connection
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
65 if conn = @reserved_connections[active_connection_name]
66 conn.verify!(verification_timeout)
67 conn
68 else
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
69 @reserved_connections[active_connection_name] = checkout
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
70 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
71 end
72
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
73 # Signal that the thread is finished with the current connection.
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
74 # #release_connection releases the connection-thread association
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
75 # and returns the connection to the pool.
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
76 def release_connection
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
77 conn = @reserved_connections.delete(active_connection_name)
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
78 checkin conn if conn
79 end
80
81 # Reserve a connection, and yield it to a block. Ensure the connection is
82 # checked back in when finished.
83 def with_connection
84 conn = checkout
85 yield conn
86 ensure
87 checkin conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
88 end
89
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
90 # Returns true if a connection has already been opened.
91 def connected?
92 !connections.empty?
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
93 end
94
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
95 # Disconnect all connections in the pool.
96 def disconnect!
97 @reserved_connections.each do |name,conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
98 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
99 end
100 connections.each do |conn|
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
101 conn.disconnect!
102 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
103 @reserved_connections = {}
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
104 end
105
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
106 # Clears the cache which maps classes
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
107 def clear_reloadable_connections!
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
108 @reserved_connections.each do |name, conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
109 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
110 end
111 @reserved_connections = {}
112 connections.each do |conn|
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
113 if conn.requires_reloading?
114 conn.disconnect!
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
115 remove_connection conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
116 end
117 end
118 end
119
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
120 # Verify active connections and remove and disconnect connections
121 # associated with stale threads.
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
122 def verify_active_connections! #:nodoc:
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
123 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
124 checkin conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
125 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
126 connections.each do |connection|
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
127 connection.verify!(verification_timeout)
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
128 end
129 end
130
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 # Check-out a database connection from the pool.
132 def checkout
133 raise NotImplementedError, "checkout is an abstract method"
134 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
135
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
136 # Check-in a database connection back into the pool.
137 def checkin(connection)
138 raise NotImplementedError, "checkin is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
139 end
140
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
141 def remove_connection(conn) #:nodoc:
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
142 raise NotImplementedError, "remove_connection is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
143 end
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
144 private :remove_connection
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
145
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
146 def connections #:nodoc:
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
147 raise NotImplementedError, "connections is an abstract method"
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
148 end
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
149 private :connections
150
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
151 synchronize :connection, :release_connection,
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
152 :clear_reloadable_connections!, :verify_active_connections!,
153 :connected?, :disconnect!, :with => :@connection_mutex
154
155 private
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
156 def new_connection
157 config = spec.config.reverse_merge(:allow_concurrency => ActiveRecord::Base.allow_concurrency)
158 ActiveRecord::Base.send(spec.adapter_method, config)
159 end
160
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
161 def active_connection_name #:nodoc:
162 Thread.current.object_id
163 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
164
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
165 # Remove stale threads from the cache.
166 def remove_stale_cached_threads!(cache, &block)
167 keys = Set.new(cache.keys)
168
169 Thread.list.each do |thread|
170 keys.delete(thread.object_id) if thread.alive?
171 end
172 keys.each do |key|
173 next unless cache.has_key?(key)
174 block.call(key, cache[key])
175 cache.delete(key)
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
176 end
177 end
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
178 end
179
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
180 # NewConnectionEveryTime is a simple implementation: always
181 # create/disconnect on checkout/checkin.
182 class NewConnectionEveryTime < ConnectionPool
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
183 def active_connection
184 @reserved_connections[active_connection_name]
185 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
186
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
187 def active_connections; @reserved_connections; end
cab76ce Nick Sieger Add synchronization to connection pool also
nicksieger authored
188
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
189 def checkout
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
190 new_connection
191 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
192
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
193 def checkin(conn)
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
194 conn.disconnect!
195 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
196
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
197 private
198 def connections
199 @reserved_connections.values
200 end
201
202 def remove_connection(conn)
203 @reserved_connections.delete_if {|k,v| v == conn}
204 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
205 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
206
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
207 # CachedConnectionPerThread is a compatible pseudo-connection pool that
208 # caches connections per-thread. In order to hold onto threads in the same
209 # manner as ActiveRecord 2.1 and earlier, it only disconnects the
210 # connection when the connection is checked in, or when calling
211 # ActiveRecord::Base.clear_all_connections!, and not during
212 # #release_connection.
213 class CachedConnectionPerThread < NewConnectionEveryTime
214 def release_connection
215 # no-op; keep the connection
216 end
217 end
218
219 # FixedSizeConnectionPool provides a full, fixed-size connection pool with
220 # timed waits when the pool is exhausted.
221 class FixedSizeConnectionPool < ConnectionPool
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
222 def initialize(spec)
223 super
224 # default 5 second timeout
225 @timeout = spec.config[:wait_timeout] || 5
226 @size = spec.config[:pool].to_i
227 @queue = @connection_mutex.new_cond
228 @connections = []
229 @checked_out = []
230 end
231
232 def checkout
233 # Checkout an available connection
234 conn = @connection_mutex.synchronize do
1712e37 Nick Sieger Favor existing connections over new ones if available
nicksieger authored
235 if @checked_out.size < @connections.size
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
236 checkout_existing_connection
1712e37 Nick Sieger Favor existing connections over new ones if available
nicksieger authored
237 elsif @connections.size < @size
238 checkout_new_connection
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
239 end
240 end
241 return conn if conn
242
243 # No connections available; wait for one
244 @connection_mutex.synchronize do
245 if @queue.wait(@timeout)
246 checkout_existing_connection
247 else
248 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
249 end
250 end
251 end
252
253 def checkin(conn)
254 @connection_mutex.synchronize do
3ce64d4 Nick Sieger Fix checkin method, add a couple more tests
nicksieger authored
255 @checked_out.delete conn
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
256 @queue.signal
257 end
258 end
259
260 private
261 def checkout_new_connection
262 c = new_connection
263 @connections << c
264 @checked_out << c
265 c
266 end
267
268 def checkout_existing_connection
269 c = [@connections - @checked_out].first
270 @checked_out << c
271 c
272 end
273
274 def connections
275 @connections
276 end
277
278 def remove_connection(conn)
279 @connections.delete conn
280 end
281 end
282
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
283 module ConnectionHandlerMethods
284 def initialize(pools = {})
285 @connection_pools = pools
286 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
287
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
288 def connection_pools
289 @connection_pools ||= {}
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
290 end
291
292 def establish_connection(name, spec)
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
293 @connection_pools[name] = ConnectionAdapters::ConnectionPool.create(spec)
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
294 end
295
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
296 # for internal use only and for testing;
297 # only works with ConnectionPerThread pool class
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
298 def active_connections #:nodoc:
299 @connection_pools.inject({}) do |hash,kv|
300 hash[kv.first] = kv.last.active_connection
301 hash.delete(kv.first) unless hash[kv.first]
302 hash
303 end
304 end
305
306 # Clears the cache which maps classes to connections.
307 def clear_active_connections!
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
308 @connection_pools.each_value {|pool| pool.release_connection }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
309 end
310
311 # Clears the cache which maps classes
312 def clear_reloadable_connections!
313 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
314 end
315
316 def clear_all_connections!
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
317 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
318 end
319
320 # Verify active connections.
321 def verify_active_connections! #:nodoc:
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
322 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
323 end
324
325 # Locate the connection of the nearest super class. This can be an
326 # active or defined connection: if it is the latter, it will be
327 # opened and set as the active connection for the class it was defined
328 # for (not necessarily the current class).
329 def retrieve_connection(klass) #:nodoc:
330 pool = retrieve_connection_pool(klass)
331 (pool && pool.connection) or raise ConnectionNotEstablished
332 end
333
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
334 # Returns true if a connection that's accessible to this class has
335 # already been opened.
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
336 def connected?(klass)
337 retrieve_connection_pool(klass).connected?
338 end
339
340 # Remove the connection for this class. This will close the active
341 # connection and the defined connection (if they exist). The result
342 # can be used as an argument for establish_connection, for easily
343 # re-establishing the connection.
344 def remove_connection(klass)
345 pool = @connection_pools[klass.name]
346 @connection_pools.delete_if { |key, value| value == pool }
347 pool.disconnect! if pool
348 pool.spec.config if pool
349 end
350
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
351 def retrieve_connection_pool(klass)
352 loop do
353 pool = @connection_pools[klass.name]
354 return pool if pool
355 return nil if ActiveRecord::Base == klass
356 klass = klass.superclass
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
357 end
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
358 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
359 end
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
360
361 # This connection handler is not thread-safe, as it does not protect access
362 # to the underlying connection pools.
363 class SingleThreadConnectionHandler
364 include ConnectionHandlerMethods
365 end
366
367 # This connection handler is thread-safe. Each access or modification of a thread
368 # pool is synchronized by an internal monitor.
369 class MultipleThreadConnectionHandler
370 attr_reader :connection_pools_lock
371 include ConnectionHandlerMethods
372
373 def initialize(pools = {})
374 super
375 @connection_pools_lock = Monitor.new
376 end
377
378 # Apply monitor to all public methods that access the pool.
379 synchronize :establish_connection, :retrieve_connection,
380 :connected?, :remove_connection, :active_connections,
381 :clear_active_connections!, :clear_reloadable_connections!,
382 :clear_all_connections!, :verify_active_connections!,
383 :with => :connection_pools_lock
384 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
385 end
386 end
Something went wrong with that request. Please try again.