Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 351 lines (318 sloc) 12.85 kb
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
1 require 'monitor'
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
a293278 Pratik Merge docrails
lifo authored
14 # == Introduction
15 #
16 # A connection pool synchronizes thread access to a limited number of
17 # database connections. The basic idea is that each thread checks out a
18 # database connection from the pool, uses that connection, and checks the
19 # connection back in. ConnectionPool is completely thread-safe, and will
20 # ensure that a connection cannot be used by two threads at the same time,
21 # as long as ConnectionPool's contract is correctly followed. It will also
22 # handle cases in which there are more threads than connections: if all
23 # connections have been checked out, and a thread tries to checkout a
24 # connection anyway, then ConnectionPool will wait until some other thread
25 # has checked in a connection.
26 #
27 # == Obtaining (checking out) a connection
28 #
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
29 # Connections can be obtained and used from a connection pool in several
30 # ways:
31 #
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
32 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
33 # earlier (pre-connection-pooling). Eventually, when you're done with
34 # the connection(s) and wish it to be returned to the pool, you call
35 # ActiveRecord::Base.clear_active_connections!. This will be the
36 # default behavior for ActiveRecord when used in conjunction with
37 # ActionPack's request handling cycle.
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
38 # 2. Manually check out a connection from the pool with
39 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
40 # returning this connection to the pool when finished by calling
41 # ActiveRecord::Base.connection_pool.checkin(connection).
42 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
43 # obtains a connection, yields it as the sole argument to the block,
44 # and returns it to the pool after the block completes.
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
45 #
a293278 Pratik Merge docrails
lifo authored
46 # Connections in the pool are actually AbstractAdapter objects (or objects
47 # compatible with AbstractAdapter's interface).
48 #
49 # == Options
50 #
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
51 # There are two connection-pooling-related options that you can add to
52 # your database connection configuration:
53 #
54 # * +pool+: number indicating size of connection pool (default 5)
55 # * +wait_timeout+: number of seconds to block and wait for a connection
56 # before giving up and raising a timeout error (default 5 seconds).
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
57 class ConnectionPool
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
58 attr_reader :spec
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
59
a293278 Pratik Merge docrails
lifo authored
60 # Creates a new ConnectionPool object. +spec+ is a ConnectionSpecification
61 # object which describes database connection information (e.g. adapter,
62 # host name, username, password, etc), as well as the maximum size for
63 # this ConnectionPool.
64 #
65 # The default ConnectionPool maximum size is 5.
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
66 def initialize(spec)
67 @spec = spec
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
68 # The cache of reserved connections mapped to threads
69 @reserved_connections = {}
cab76ce Nick Sieger Add synchronization to connection pool also
nicksieger authored
70 # The mutex used to synchronize pool access
71 @connection_mutex = Monitor.new
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
72 @queue = @connection_mutex.new_cond
73 # default 5 second timeout
74 @timeout = spec.config[:wait_timeout] || 5
75 # default max pool size to 5
76 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
77 @connections = []
78 @checked_out = []
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
79 end
80
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
81 # Retrieve the connection associated with the current thread, or call
82 # #checkout to obtain one if necessary.
83 #
84 # #connection can be called any number of times; the connection is
85 # held in a hash keyed by the thread id.
86 def connection
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
87 if conn = @reserved_connections[current_connection_id]
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
88 conn
89 else
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
90 @reserved_connections[current_connection_id] = checkout
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
91 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
92 end
93
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
94 # Signal that the thread is finished with the current connection.
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
95 # #release_connection releases the connection-thread association
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
96 # and returns the connection to the pool.
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
97 def release_connection
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
98 conn = @reserved_connections.delete(current_connection_id)
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
99 checkin conn if conn
100 end
101
102 # Reserve a connection, and yield it to a block. Ensure the connection is
103 # checked back in when finished.
104 def with_connection
105 conn = checkout
106 yield conn
107 ensure
108 checkin conn
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
109 end
110
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
111 # Returns true if a connection has already been opened.
112 def connected?
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
113 !@connections.empty?
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
114 end
115
a293278 Pratik Merge docrails
lifo authored
116 # Disconnects all connections in the pool, and clears the pool.
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
117 def disconnect!
118 @reserved_connections.each do |name,conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
119 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
120 end
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
121 @reserved_connections = {}
122 @connections.each do |conn|
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
123 conn.disconnect!
124 end
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
125 @connections = []
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
126 end
127
50cd4bd Nick Sieger Introduce synchronization around connection pool access
nicksieger authored
128 # Clears the cache which maps classes
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
129 def clear_reloadable_connections!
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
130 @reserved_connections.each do |name, conn|
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 checkin conn
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
132 end
133 @reserved_connections = {}
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
134 @connections.each do |conn|
135 conn.disconnect! if conn.requires_reloading?
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
136 end
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
137 @connections = []
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
138 end
139
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
140 # Verify active connections and remove and disconnect connections
141 # associated with stale threads.
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
142 def verify_active_connections! #:nodoc:
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
143 clear_stale_cached_connections!
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
144 @connections.each do |connection|
7ba2872 Nick Sieger Deprecate verification_timeout and verify before reset
nicksieger authored
145 connection.verify!
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
146 end
147 end
148
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
149 # Return any checked-out connections back to the pool by threads that
150 # are no longer alive.
151 def clear_stale_cached_connections!
152 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
153 checkin conn
154 end
155 end
156
a293278 Pratik Merge docrails
lifo authored
157 # Check-out a database connection from the pool, indicating that you want
158 # to use it. You should call #checkin when you no longer need this.
159 #
160 # This is done by either returning an existing connection, or by creating
161 # a new connection. If the maximum number of connections for this pool has
162 # already been reached, but the pool is empty (i.e. they're all being used),
163 # then this method will wait until a thread has checked in a connection.
164 # The wait time is bounded however: if no connection can be checked out
165 # within the timeout specified for this pool, then a ConnectionTimeoutError
166 # exception will be raised.
167 #
168 # Returns: an AbstractAdapter object.
169 #
170 # Raises:
171 # - ConnectionTimeoutError: no connection can be obtained from the pool
172 # within the timeout period.
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
173 def checkout
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
174 # Checkout an available connection
175 @connection_mutex.synchronize do
21eb18a Aliaksey Kandratsenka (aka Aliaksei Kandratsenka) Fix race in ConnectionPool#checkout
alk authored
176 loop do
177 conn = if @checked_out.size < @connections.size
178 checkout_existing_connection
179 elsif @connections.size < @size
180 checkout_new_connection
181 end
182 return conn if conn
183 # No connections available; wait for one
184 if @queue.wait(@timeout)
185 next
186 else
8343611 Aliaksey Kandratsenka (aka Aliaksei Kandratsenka) made ConnectionPool#checkout more robust by trying to loot dead threads ...
alk authored
187 # try looting dead threads
188 clear_stale_cached_connections!
189 if @size == @checked_out.size
190 raise ConnectionTimeoutError, "could not obtain a database connection within #{@timeout} seconds. The pool size is currently #{@size}, perhaps you need to increase it?"
191 end
21eb18a Aliaksey Kandratsenka (aka Aliaksei Kandratsenka) Fix race in ConnectionPool#checkout
alk authored
192 end
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
193 end
194 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
195 end
196
a293278 Pratik Merge docrails
lifo authored
197 # Check-in a database connection back into the pool, indicating that you
198 # no longer need this connection.
199 #
200 # +conn+: an AbstractAdapter object, which was obtained by earlier by
201 # calling +checkout+ on this pool.
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
202 def checkin(conn)
203 @connection_mutex.synchronize do
204 conn.run_callbacks :checkin
205 @checked_out.delete conn
206 @queue.signal
207 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
208 end
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
209
113cc4e Nick Sieger Remove some synchronization that's probably overkill, assuming one doesn...
nicksieger authored
210 synchronize :clear_reloadable_connections!, :verify_active_connections!,
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
211 :connected?, :disconnect!, :with => :@connection_mutex
212
213 private
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
214 def new_connection
a3f12f5 Nick Sieger Default connection allow_concurrency to false (for PostgreSQL)
nicksieger authored
215 ActiveRecord::Base.send(spec.adapter_method, spec.config)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
216 end
217
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
218 def current_connection_id #:nodoc:
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
219 Thread.current.object_id
220 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
221
029952e Nick Sieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
222 # Remove stale threads from the cache.
223 def remove_stale_cached_threads!(cache, &block)
224 keys = Set.new(cache.keys)
225
226 Thread.list.each do |thread|
227 keys.delete(thread.object_id) if thread.alive?
228 end
229 keys.each do |key|
230 next unless cache.has_key?(key)
231 block.call(key, cache[key])
232 cache.delete(key)
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
233 end
234 end
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
235
236 def checkout_new_connection
237 c = new_connection
238 @connections << c
a96b7d4 Nick Sieger Add connection reset and verification upon each connection checkout
nicksieger authored
239 checkout_and_verify(c)
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
240 end
241
242 def checkout_existing_connection
d7d2d73 Nick Sieger Fix typo: was using brackets instead of parens. Must need more sleep.
nicksieger authored
243 c = (@connections - @checked_out).first
a96b7d4 Nick Sieger Add connection reset and verification upon each connection checkout
nicksieger authored
244 checkout_and_verify(c)
245 end
246
247 def checkout_and_verify(c)
7ba2872 Nick Sieger Deprecate verification_timeout and verify before reset
nicksieger authored
248 c.verify!
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
249 c.run_callbacks :checkout
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
250 @checked_out << c
251 c
252 end
253 end
254
a293278 Pratik Merge docrails
lifo authored
255 # ConnectionHandler is a collection of ConnectionPool objects. It is used
256 # for keeping separate connection pools for ActiveRecord models that connect
257 # to different databases.
258 #
259 # For example, suppose that you have 5 models, with the following hierarchy:
260 #
261 # |
262 # +-- Book
263 # | |
264 # | +-- ScaryBook
265 # | +-- GoodBook
266 # +-- Author
267 # +-- BankAccount
268 #
269 # Suppose that Book is to connect to a separate database (i.e. one other
270 # than the default database). Then Book, ScaryBook and GoodBook will all use
271 # the same connection pool. Likewise, Author and BankAccount will use the
272 # same connection pool. However, the connection pool used by Author/BankAccount
273 # is not the same as the one used by Book/ScaryBook/GoodBook.
274 #
275 # Normally there is only a single ConnectionHandler instance, accessible via
276 # ActiveRecord::Base.connection_handler. ActiveRecord models use this to
277 # determine that connection pool that they should use.
ca6d717 Nick Sieger Deprecate allow_concurrency and make it have no effect
nicksieger authored
278 class ConnectionHandler
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
279 def initialize(pools = {})
280 @connection_pools = pools
281 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
282
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
283 def connection_pools
284 @connection_pools ||= {}
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
285 end
286
287 def establish_connection(name, spec)
8e5e02b Nick Sieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
288 @connection_pools[name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
289 end
290
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
291 # Returns any connections in use by the current thread back to the pool,
292 # and also returns connections to the pool cached by threads that are no
293 # longer alive.
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
294 def clear_active_connections!
d07a6b1 Nick Sieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
295 @connection_pools.each_value do |pool|
296 pool.release_connection
297 pool.clear_stale_cached_connections!
298 end
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
299 end
300
301 # Clears the cache which maps classes
302 def clear_reloadable_connections!
303 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
304 end
305
306 def clear_all_connections!
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
307 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
308 end
309
310 # Verify active connections.
311 def verify_active_connections! #:nodoc:
817a07b Nick Sieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
312 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
313 end
314
315 # Locate the connection of the nearest super class. This can be an
316 # active or defined connection: if it is the latter, it will be
317 # opened and set as the active connection for the class it was defined
318 # for (not necessarily the current class).
319 def retrieve_connection(klass) #:nodoc:
320 pool = retrieve_connection_pool(klass)
321 (pool && pool.connection) or raise ConnectionNotEstablished
322 end
323
82fcd9d Nick Sieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
324 # Returns true if a connection that's accessible to this class has
325 # already been opened.
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
326 def connected?(klass)
0832bc6 Pratik Make sure ActiveRecord::Base.connected? doesn't raise an exception for d...
lifo authored
327 conn = retrieve_connection_pool(klass)
328 conn ? conn.connected? : false
ff97e9d Nick Sieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
329 end
330
331 # Remove the connection for this class. This will close the active
332 # connection and the defined connection (if they exist). The result
333 # can be used as an argument for establish_connection, for easily
334 # re-establishing the connection.
335 def remove_connection(klass)
336 pool = @connection_pools[klass.name]
337 @connection_pools.delete_if { |key, value| value == pool }
338 pool.disconnect! if pool
339 pool.spec.config if pool
340 end
341
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
342 def retrieve_connection_pool(klass)
3007545 Nick Sieger Minor tweak to retrieve_connection_pool -- recurse instead of loop
nicksieger authored
343 pool = @connection_pools[klass.name]
344 return pool if pool
345 return nil if ActiveRecord::Base == klass
346 retrieve_connection_pool klass.superclass
fe575dd Nick Sieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
347 end
72d959d Nick Sieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
348 end
6edaa26 Nick Sieger Initial conversion to connection pool
nicksieger authored
349 end
21eb18a Aliaksey Kandratsenka (aka Aliaksei Kandratsenka) Fix race in ConnectionPool#checkout
alk authored
350 end
Something went wrong with that request. Please try again.