Skip to content
This repository
Newer
Older
100644 386 lines (335 sloc) 12.64 kb
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
1 require 'monitor'
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
2 require 'set'
3
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
4 module ActiveRecord
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
10 module ConnectionAdapters
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
17 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
18 # earlier (pre-connection-pooling). Eventually, when you're done with
19 # the connection(s) and wish it to be returned to the pool, you call
20 # ActiveRecord::Base.clear_active_connections!. This will be the
21 # default behavior for ActiveRecord when used in conjunction with
22 # ActionPack's request handling cycle.
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
23 # 2. Manually check out a connection from the pool with
24 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
25 # returning this connection to the pool when finished by calling
26 # ActiveRecord::Base.connection_pool.checkin(connection).
27 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
28 # obtains a connection, yields it as the sole argument to the block,
29 # and returns it to the pool after the block completes.
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
30 class ConnectionPool
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
31 # Factory method for connection pools.
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
32 # Determines pool type to use based on contents of connection
33 # specification. Additional options for connection specification:
34 #
35 # * +pool+: number indicating size of fixed connection pool to use
36 # * +wait_timeout+ (optional): number of seconds to block and wait
37 # for a connection before giving up and raising a timeout error.
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
38 def self.create(spec)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
39 if spec.config[:pool] && spec.config[:pool].to_i > 0
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
40 FixedSizeConnectionPool.new(spec)
41 elsif spec.config[:jndi] # JRuby appserver datasource pool; passthrough
42 NewConnectionEveryTime.new(spec)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
43 else
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
44 CachedConnectionPerThread.new(spec)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
45 end
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
46 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
47
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
48 delegate :verification_timeout, :to => "::ActiveRecord::Base"
49 attr_reader :spec
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
50
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
51 def initialize(spec)
52 @spec = spec
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
53 # The cache of reserved connections mapped to threads
54 @reserved_connections = {}
cab76ce6 » nicksieger
2008-04-19 Add synchronization to connection pool also
55 # The mutex used to synchronize pool access
56 @connection_mutex = Monitor.new
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
57 end
58
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
59 # Retrieve the connection associated with the current thread, or call
60 # #checkout to obtain one if necessary.
61 #
62 # #connection can be called any number of times; the connection is
63 # held in a hash keyed by the thread id.
64 def connection
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
65 if conn = @reserved_connections[active_connection_name]
66 conn.verify!(verification_timeout)
67 conn
68 else
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
69 @reserved_connections[active_connection_name] = checkout
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
70 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
71 end
72
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
73 # Signal that the thread is finished with the current connection.
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
74 # #release_connection releases the connection-thread association
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
75 # and returns the connection to the pool.
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
76 def release_connection
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
77 conn = @reserved_connections.delete(active_connection_name)
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
78 checkin conn if conn
79 end
80
81 # Reserve a connection, and yield it to a block. Ensure the connection is
82 # checked back in when finished.
83 def with_connection
84 conn = checkout
85 yield conn
86 ensure
87 checkin conn
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
88 end
89
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
90 # Returns true if a connection has already been opened.
91 def connected?
92 !connections.empty?
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
93 end
94
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
95 # Disconnect all connections in the pool.
96 def disconnect!
97 @reserved_connections.each do |name,conn|
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
98 checkin conn
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
99 end
100 connections.each do |conn|
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
101 conn.disconnect!
102 end
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
103 @reserved_connections = {}
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
104 end
105
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
106 # Clears the cache which maps classes
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
107 def clear_reloadable_connections!
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
108 @reserved_connections.each do |name, conn|
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
109 checkin conn
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
110 end
111 @reserved_connections = {}
112 connections.each do |conn|
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
113 if conn.requires_reloading?
114 conn.disconnect!
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
115 remove_connection conn
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
116 end
117 end
118 end
119
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
120 # Verify active connections and remove and disconnect connections
121 # associated with stale threads.
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
122 def verify_active_connections! #:nodoc:
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
123 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
124 checkin conn
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
125 end
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
126 connections.each do |connection|
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
127 connection.verify!(verification_timeout)
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
128 end
129 end
130
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
131 # Check-out a database connection from the pool.
132 def checkout
133 raise NotImplementedError, "checkout is an abstract method"
134 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
135
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
136 # Check-in a database connection back into the pool.
137 def checkin(connection)
138 raise NotImplementedError, "checkin is an abstract method"
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
139 end
140
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
141 def remove_connection(conn) #:nodoc:
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
142 raise NotImplementedError, "remove_connection is an abstract method"
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
143 end
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
144 private :remove_connection
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
145
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
146 def connections #:nodoc:
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
147 raise NotImplementedError, "connections is an abstract method"
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
148 end
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
149 private :connections
150
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
151 synchronize :connection, :release_connection,
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
152 :clear_reloadable_connections!, :verify_active_connections!,
153 :connected?, :disconnect!, :with => :@connection_mutex
154
155 private
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
156 def new_connection
157 config = spec.config.reverse_merge(:allow_concurrency => ActiveRecord::Base.allow_concurrency)
158 ActiveRecord::Base.send(spec.adapter_method, config)
159 end
160
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
161 def active_connection_name #:nodoc:
162 Thread.current.object_id
163 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
164
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
165 # Remove stale threads from the cache.
166 def remove_stale_cached_threads!(cache, &block)
167 keys = Set.new(cache.keys)
168
169 Thread.list.each do |thread|
170 keys.delete(thread.object_id) if thread.alive?
171 end
172 keys.each do |key|
173 next unless cache.has_key?(key)
174 block.call(key, cache[key])
175 cache.delete(key)
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
176 end
177 end
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
178 end
179
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
180 # NewConnectionEveryTime is a simple implementation: always
181 # create/disconnect on checkout/checkin.
182 class NewConnectionEveryTime < ConnectionPool
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
183 def active_connection
184 @reserved_connections[active_connection_name]
185 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
186
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
187 def active_connections; @reserved_connections; end
cab76ce6 » nicksieger
2008-04-19 Add synchronization to connection pool also
188
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
189 def checkout
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
190 new_connection
191 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
192
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
193 def checkin(conn)
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
194 conn.disconnect!
195 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
196
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
197 private
198 def connections
199 @reserved_connections.values
200 end
201
202 def remove_connection(conn)
203 @reserved_connections.delete_if {|k,v| v == conn}
204 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
205 end
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
206
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
207 # CachedConnectionPerThread is a compatible pseudo-connection pool that
208 # caches connections per-thread. In order to hold onto threads in the same
209 # manner as ActiveRecord 2.1 and earlier, it only disconnects the
210 # connection when the connection is checked in, or when calling
211 # ActiveRecord::Base.clear_all_connections!, and not during
212 # #release_connection.
213 class CachedConnectionPerThread < NewConnectionEveryTime
214 def release_connection
215 # no-op; keep the connection
216 end
217 end
218
219 # FixedSizeConnectionPool provides a full, fixed-size connection pool with
220 # timed waits when the pool is exhausted.
221 class FixedSizeConnectionPool < ConnectionPool
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
222 def initialize(spec)
223 super
224 # default 5 second timeout
225 @timeout = spec.config[:wait_timeout] || 5
226 @size = spec.config[:pool].to_i
227 @queue = @connection_mutex.new_cond
228 @connections = []
229 @checked_out = []
230 end
231
232 def checkout
233 # Checkout an available connection
234 conn = @connection_mutex.synchronize do
235 if @connections.length < @size
236 checkout_new_connection
237 elsif @checked_out.size < @connections.size
238 checkout_existing_connection
239 end
240 end
241 return conn if conn
242
243 # No connections available; wait for one
244 @connection_mutex.synchronize do
245 if @queue.wait(@timeout)
246 checkout_existing_connection
247 else
248 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
249 end
250 end
251 end
252
253 def checkin(conn)
254 @connection_mutex.synchronize do
3ce64d4f » nicksieger
2008-08-07 Fix checkin method, add a couple more tests
255 @checked_out.delete conn
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
256 @queue.signal
257 end
258 end
259
260 private
261 def checkout_new_connection
262 c = new_connection
263 @connections << c
264 @checked_out << c
265 c
266 end
267
268 def checkout_existing_connection
269 c = [@connections - @checked_out].first
270 @checked_out << c
271 c
272 end
273
274 def connections
275 @connections
276 end
277
278 def remove_connection(conn)
279 @connections.delete conn
280 end
281 end
282
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
283 module ConnectionHandlerMethods
284 def initialize(pools = {})
285 @connection_pools = pools
286 end
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
287
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
288 def connection_pools
289 @connection_pools ||= {}
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
290 end
291
292 def establish_connection(name, spec)
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
293 @connection_pools[name] = ConnectionAdapters::ConnectionPool.create(spec)
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
294 end
295
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
296 # for internal use only and for testing;
297 # only works with ConnectionPerThread pool class
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
298 def active_connections #:nodoc:
299 @connection_pools.inject({}) do |hash,kv|
300 hash[kv.first] = kv.last.active_connection
301 hash.delete(kv.first) unless hash[kv.first]
302 hash
303 end
304 end
305
306 # Clears the cache which maps classes to connections.
307 def clear_active_connections!
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
308 @connection_pools.each_value {|pool| pool.release_connection }
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
309 end
310
311 # Clears the cache which maps classes
312 def clear_reloadable_connections!
313 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
314 end
315
316 def clear_all_connections!
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
317 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
318 end
319
320 # Verify active connections.
321 def verify_active_connections! #:nodoc:
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
322 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
323 end
324
325 # Locate the connection of the nearest super class. This can be an
326 # active or defined connection: if it is the latter, it will be
327 # opened and set as the active connection for the class it was defined
328 # for (not necessarily the current class).
329 def retrieve_connection(klass) #:nodoc:
330 pool = retrieve_connection_pool(klass)
331 (pool && pool.connection) or raise ConnectionNotEstablished
332 end
333
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
334 # Returns true if a connection that's accessible to this class has
335 # already been opened.
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
336 def connected?(klass)
337 retrieve_connection_pool(klass).connected?
338 end
339
340 # Remove the connection for this class. This will close the active
341 # connection and the defined connection (if they exist). The result
342 # can be used as an argument for establish_connection, for easily
343 # re-establishing the connection.
344 def remove_connection(klass)
345 pool = @connection_pools[klass.name]
346 @connection_pools.delete_if { |key, value| value == pool }
347 pool.disconnect! if pool
348 pool.spec.config if pool
349 end
350
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
351 def retrieve_connection_pool(klass)
352 loop do
353 pool = @connection_pools[klass.name]
354 return pool if pool
355 return nil if ActiveRecord::Base == klass
356 klass = klass.superclass
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
357 end
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
358 end
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
359 end
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
360
361 # This connection handler is not thread-safe, as it does not protect access
362 # to the underlying connection pools.
363 class SingleThreadConnectionHandler
364 include ConnectionHandlerMethods
365 end
366
367 # This connection handler is thread-safe. Each access or modification of a thread
368 # pool is synchronized by an internal monitor.
369 class MultipleThreadConnectionHandler
370 attr_reader :connection_pools_lock
371 include ConnectionHandlerMethods
372
373 def initialize(pools = {})
374 super
375 @connection_pools_lock = Monitor.new
376 end
377
378 # Apply monitor to all public methods that access the pool.
379 synchronize :establish_connection, :retrieve_connection,
380 :connected?, :remove_connection, :active_connections,
381 :clear_active_connections!, :clear_reloadable_connections!,
382 :clear_all_connections!, :verify_active_connections!,
383 :with => :connection_pools_lock
384 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
385 end
386 end
Something went wrong with that request. Please try again.