Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 386 lines (335 sloc) 12.638 kB
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
1 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
17 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
18 # earlier (pre-connection-pooling). Eventually, when you're done with
19 # the connection(s) and wish it to be returned to the pool, you call
20 # ActiveRecord::Base.clear_active_connections!. This will be the
21 # default behavior for ActiveRecord when used in conjunction with
22 # ActionPack's request handling cycle.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
23 # 2. Manually check out a connection from the pool with
24 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
25 # returning this connection to the pool when finished by calling
26 # ActiveRecord::Base.connection_pool.checkin(connection).
27 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
28 # obtains a connection, yields it as the sole argument to the block,
29 # and returns it to the pool after the block completes.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
30 class ConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
31 # Factory method for connection pools.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
32 # Determines pool type to use based on contents of connection
33 # specification. Additional options for connection specification:
34 #
35 # * +pool+: number indicating size of fixed connection pool to use
36 # * +wait_timeout+ (optional): number of seconds to block and wait
37 # for a connection before giving up and raising a timeout error.
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
38 def self.create(spec)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
39 if spec.config[:pool] && spec.config[:pool].to_i > 0
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
40 FixedSizeConnectionPool.new(spec)
41 elsif spec.config[:jndi] # JRuby appserver datasource pool; passthrough
42 NewConnectionEveryTime.new(spec)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
43 else
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
44 CachedConnectionPerThread.new(spec)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
45 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
46 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
47
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
48 delegate :verification_timeout, :to => "::ActiveRecord::Base"
49 attr_reader :spec
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
50
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
51 def initialize(spec)
52 @spec = spec
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
53 # The cache of reserved connections mapped to threads
54 @reserved_connections = {}
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
55 # The mutex used to synchronize pool access
56 @connection_mutex = Monitor.new
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
57 end
58
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
59 # Retrieve the connection associated with the current thread, or call
60 # #checkout to obtain one if necessary.
61 #
62 # #connection can be called any number of times; the connection is
63 # held in a hash keyed by the thread id.
64 def connection
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
65 if conn = @reserved_connections[active_connection_name]
66 conn.verify!(verification_timeout)
67 conn
68 else
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
69 @reserved_connections[active_connection_name] = checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
70 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
71 end
72
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
73 # Signal that the thread is finished with the current connection.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
74 # #release_connection releases the connection-thread association
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
75 # and returns the connection to the pool.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
76 def release_connection
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
77 conn = @reserved_connections.delete(active_connection_name)
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
78 checkin conn if conn
79 end
80
81 # Reserve a connection, and yield it to a block. Ensure the connection is
82 # checked back in when finished.
83 def with_connection
84 conn = checkout
85 yield conn
86 ensure
87 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
88 end
89
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
90 # Returns true if a connection has already been opened.
91 def connected?
92 !connections.empty?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
93 end
94
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
95 # Disconnect all connections in the pool.
96 def disconnect!
97 @reserved_connections.each do |name,conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
98 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
99 end
100 connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
101 conn.disconnect!
102 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
103 @reserved_connections = {}
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
104 end
105
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
106 # Clears the cache which maps classes
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
107 def clear_reloadable_connections!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
108 @reserved_connections.each do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
109 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
110 end
111 @reserved_connections = {}
112 connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
113 if conn.requires_reloading?
114 conn.disconnect!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
115 remove_connection conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
116 end
117 end
118 end
119
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
120 # Verify active connections and remove and disconnect connections
121 # associated with stale threads.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
122 def verify_active_connections! #:nodoc:
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
123 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
124 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
125 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
126 connections.each do |connection|
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
127 connection.verify!(verification_timeout)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
128 end
129 end
130
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 # Check-out a database connection from the pool.
132 def checkout
133 raise NotImplementedError, "checkout is an abstract method"
134 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
135
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
136 # Check-in a database connection back into the pool.
137 def checkin(connection)
138 raise NotImplementedError, "checkin is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
139 end
140
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
141 def remove_connection(conn) #:nodoc:
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
142 raise NotImplementedError, "remove_connection is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
143 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
144 private :remove_connection
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
145
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
146 def connections #:nodoc:
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
147 raise NotImplementedError, "connections is an abstract method"
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
148 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
149 private :connections
150
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
151 synchronize :connection, :release_connection,
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
152 :clear_reloadable_connections!, :verify_active_connections!,
153 :connected?, :disconnect!, :with => :@connection_mutex
154
155 private
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
156 def new_connection
157 config = spec.config.reverse_merge(:allow_concurrency => ActiveRecord::Base.allow_concurrency)
158 ActiveRecord::Base.send(spec.adapter_method, config)
159 end
160
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
161 def active_connection_name #:nodoc:
162 Thread.current.object_id
163 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
164
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
165 # Remove stale threads from the cache.
166 def remove_stale_cached_threads!(cache, &block)
167 keys = Set.new(cache.keys)
168
169 Thread.list.each do |thread|
170 keys.delete(thread.object_id) if thread.alive?
171 end
172 keys.each do |key|
173 next unless cache.has_key?(key)
174 block.call(key, cache[key])
175 cache.delete(key)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
176 end
177 end
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
178 end
179
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
180 # NewConnectionEveryTime is a simple implementation: always
181 # create/disconnect on checkout/checkin.
182 class NewConnectionEveryTime < ConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
183 def active_connection
184 @reserved_connections[active_connection_name]
185 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
186
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
187 def active_connections; @reserved_connections; end
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
188
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
189 def checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
190 new_connection
191 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
192
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
193 def checkin(conn)
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
194 conn.disconnect!
195 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
196
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
197 private
198 def connections
199 @reserved_connections.values
200 end
201
202 def remove_connection(conn)
203 @reserved_connections.delete_if {|k,v| v == conn}
204 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
205 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
206
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
207 # CachedConnectionPerThread is a compatible pseudo-connection pool that
208 # caches connections per-thread. In order to hold onto threads in the same
209 # manner as ActiveRecord 2.1 and earlier, it only disconnects the
210 # connection when the connection is checked in, or when calling
211 # ActiveRecord::Base.clear_all_connections!, and not during
212 # #release_connection.
213 class CachedConnectionPerThread < NewConnectionEveryTime
214 def release_connection
215 # no-op; keep the connection
216 end
217 end
218
219 # FixedSizeConnectionPool provides a full, fixed-size connection pool with
220 # timed waits when the pool is exhausted.
221 class FixedSizeConnectionPool < ConnectionPool
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
222 def initialize(spec)
223 super
224 # default 5 second timeout
225 @timeout = spec.config[:wait_timeout] || 5
226 @size = spec.config[:pool].to_i
227 @queue = @connection_mutex.new_cond
228 @connections = []
229 @checked_out = []
230 end
231
232 def checkout
233 # Checkout an available connection
234 conn = @connection_mutex.synchronize do
1712e37 @nicksieger Favor existing connections over new ones if available
nicksieger authored
235 if @checked_out.size < @connections.size
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
236 checkout_existing_connection
1712e37 @nicksieger Favor existing connections over new ones if available
nicksieger authored
237 elsif @connections.size < @size
238 checkout_new_connection
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
239 end
240 end
241 return conn if conn
242
243 # No connections available; wait for one
244 @connection_mutex.synchronize do
245 if @queue.wait(@timeout)
246 checkout_existing_connection
247 else
248 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
249 end
250 end
251 end
252
253 def checkin(conn)
254 @connection_mutex.synchronize do
3ce64d4 @nicksieger Fix checkin method, add a couple more tests
nicksieger authored
255 @checked_out.delete conn
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
256 @queue.signal
257 end
258 end
259
260 private
261 def checkout_new_connection
262 c = new_connection
263 @connections << c
264 @checked_out << c
265 c
266 end
267
268 def checkout_existing_connection
269 c = [@connections - @checked_out].first
270 @checked_out << c
271 c
272 end
273
274 def connections
275 @connections
276 end
277
278 def remove_connection(conn)
279 @connections.delete conn
280 end
281 end
282
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
283 module ConnectionHandlerMethods
284 def initialize(pools = {})
285 @connection_pools = pools
286 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
287
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
288 def connection_pools
289 @connection_pools ||= {}
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
290 end
291
292 def establish_connection(name, spec)
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
293 @connection_pools[name] = ConnectionAdapters::ConnectionPool.create(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
294 end
295
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
296 # for internal use only and for testing;
297 # only works with ConnectionPerThread pool class
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
298 def active_connections #:nodoc:
299 @connection_pools.inject({}) do |hash,kv|
300 hash[kv.first] = kv.last.active_connection
301 hash.delete(kv.first) unless hash[kv.first]
302 hash
303 end
304 end
305
306 # Clears the cache which maps classes to connections.
307 def clear_active_connections!
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
308 @connection_pools.each_value {|pool| pool.release_connection }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
309 end
310
311 # Clears the cache which maps classes
312 def clear_reloadable_connections!
313 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
314 end
315
316 def clear_all_connections!
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
317 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
318 end
319
320 # Verify active connections.
321 def verify_active_connections! #:nodoc:
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
322 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
323 end
324
325 # Locate the connection of the nearest super class. This can be an
326 # active or defined connection: if it is the latter, it will be
327 # opened and set as the active connection for the class it was defined
328 # for (not necessarily the current class).
329 def retrieve_connection(klass) #:nodoc:
330 pool = retrieve_connection_pool(klass)
331 (pool && pool.connection) or raise ConnectionNotEstablished
332 end
333
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
334 # Returns true if a connection that's accessible to this class has
335 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
336 def connected?(klass)
337 retrieve_connection_pool(klass).connected?
338 end
339
340 # Remove the connection for this class. This will close the active
341 # connection and the defined connection (if they exist). The result
342 # can be used as an argument for establish_connection, for easily
343 # re-establishing the connection.
344 def remove_connection(klass)
345 pool = @connection_pools[klass.name]
346 @connection_pools.delete_if { |key, value| value == pool }
347 pool.disconnect! if pool
348 pool.spec.config if pool
349 end
350
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
351 def retrieve_connection_pool(klass)
352 loop do
353 pool = @connection_pools[klass.name]
354 return pool if pool
355 return nil if ActiveRecord::Base == klass
356 klass = klass.superclass
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
357 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
358 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
359 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
360
361 # This connection handler is not thread-safe, as it does not protect access
362 # to the underlying connection pools.
363 class SingleThreadConnectionHandler
364 include ConnectionHandlerMethods
365 end
366
367 # This connection handler is thread-safe. Each access or modification of a thread
368 # pool is synchronized by an internal monitor.
369 class MultipleThreadConnectionHandler
370 attr_reader :connection_pools_lock
371 include ConnectionHandlerMethods
372
373 def initialize(pools = {})
374 super
375 @connection_pools_lock = Monitor.new
376 end
377
378 # Apply monitor to all public methods that access the pool.
379 synchronize :establish_connection, :retrieve_connection,
380 :connected?, :remove_connection, :active_connections,
381 :clear_active_connections!, :clear_reloadable_connections!,
382 :clear_all_connections!, :verify_active_connections!,
383 :with => :connection_pools_lock
384 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
385 end
386 end
Something went wrong with that request. Please try again.