Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 356 lines (318 sloc) 12.877 kb
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
1 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
a293278 @lifo Merge docrails
lifo authored
14 # == Introduction
15 #
16 # A connection pool synchronizes thread access to a limited number of
17 # database connections. The basic idea is that each thread checks out a
18 # database connection from the pool, uses that connection, and checks the
19 # connection back in. ConnectionPool is completely thread-safe, and will
20 # ensure that a connection cannot be used by two threads at the same time,
21 # as long as ConnectionPool's contract is correctly followed. It will also
22 # handle cases in which there are more threads than connections: if all
23 # connections have been checked out, and a thread tries to checkout a
24 # connection anyway, then ConnectionPool will wait until some other thread
25 # has checked in a connection.
26 #
27 # == Obtaining (checking out) a connection
28 #
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
29 # Connections can be obtained and used from a connection pool in several
30 # ways:
31 #
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
32 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
33 # earlier (pre-connection-pooling). Eventually, when you're done with
34 # the connection(s) and wish it to be returned to the pool, you call
35 # ActiveRecord::Base.clear_active_connections!. This will be the
36 # default behavior for ActiveRecord when used in conjunction with
37 # ActionPack's request handling cycle.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
38 # 2. Manually check out a connection from the pool with
39 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
40 # returning this connection to the pool when finished by calling
41 # ActiveRecord::Base.connection_pool.checkin(connection).
42 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
43 # obtains a connection, yields it as the sole argument to the block,
44 # and returns it to the pool after the block completes.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
45 #
a293278 @lifo Merge docrails
lifo authored
46 # Connections in the pool are actually AbstractAdapter objects (or objects
47 # compatible with AbstractAdapter's interface).
48 #
49 # == Options
50 #
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
51 # There are two connection-pooling-related options that you can add to
52 # your database connection configuration:
53 #
54 # * +pool+: number indicating size of connection pool (default 5)
55 # * +wait_timeout+: number of seconds to block and wait for a connection
56 # before giving up and raising a timeout error (default 5 seconds).
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
57 class ConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
58 attr_reader :spec
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
59
a293278 @lifo Merge docrails
lifo authored
60 # Creates a new ConnectionPool object. +spec+ is a ConnectionSpecification
61 # object which describes database connection information (e.g. adapter,
62 # host name, username, password, etc), as well as the maximum size for
63 # this ConnectionPool.
64 #
65 # The default ConnectionPool maximum size is 5.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
66 def initialize(spec)
67 @spec = spec
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
68
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
69 # The cache of reserved connections mapped to threads
70 @reserved_connections = {}
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
71
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
72 # The mutex used to synchronize pool access
73 @connection_mutex = Monitor.new
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
74 @queue = @connection_mutex.new_cond
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
75
76 # default 5 second timeout unless on ruby 1.9
77 @timeout =
78 if RUBY_VERSION < '1.9'
79 spec.config[:wait_timeout] || 5
80 end
81
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
82 # default max pool size to 5
83 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
84
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
85 @connections = []
86 @checked_out = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
87 end
88
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
89 # Retrieve the connection associated with the current thread, or call
90 # #checkout to obtain one if necessary.
91 #
92 # #connection can be called any number of times; the connection is
93 # held in a hash keyed by the thread id.
94 def connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
95 if conn = @reserved_connections[current_connection_id]
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
96 conn
97 else
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
98 @reserved_connections[current_connection_id] = checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
99 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
100 end
101
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
102 # Signal that the thread is finished with the current connection.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
103 # #release_connection releases the connection-thread association
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
104 # and returns the connection to the pool.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
105 def release_connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
106 conn = @reserved_connections.delete(current_connection_id)
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
107 checkin conn if conn
108 end
109
110 # Reserve a connection, and yield it to a block. Ensure the connection is
111 # checked back in when finished.
112 def with_connection
113 conn = checkout
114 yield conn
115 ensure
116 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
117 end
118
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
119 # Returns true if a connection has already been opened.
120 def connected?
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
121 !@connections.empty?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
122 end
123
a293278 @lifo Merge docrails
lifo authored
124 # Disconnects all connections in the pool, and clears the pool.
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
125 def disconnect!
126 @reserved_connections.each do |name,conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
127 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
128 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
129 @reserved_connections = {}
130 @connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
131 conn.disconnect!
132 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
133 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
134 end
135
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
136 # Clears the cache which maps classes
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
137 def clear_reloadable_connections!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
138 @reserved_connections.each do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
139 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
140 end
141 @reserved_connections = {}
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
142 @connections.each do |conn|
143 conn.disconnect! if conn.requires_reloading?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
144 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
145 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
146 end
147
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
148 # Verify active connections and remove and disconnect connections
149 # associated with stale threads.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
150 def verify_active_connections! #:nodoc:
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
151 clear_stale_cached_connections!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
152 @connections.each do |connection|
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
153 connection.verify!
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
154 end
155 end
156
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
157 # Return any checked-out connections back to the pool by threads that
158 # are no longer alive.
159 def clear_stale_cached_connections!
160 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
161 checkin conn
162 end
163 end
164
a293278 @lifo Merge docrails
lifo authored
165 # Check-out a database connection from the pool, indicating that you want
166 # to use it. You should call #checkin when you no longer need this.
167 #
168 # This is done by either returning an existing connection, or by creating
169 # a new connection. If the maximum number of connections for this pool has
170 # already been reached, but the pool is empty (i.e. they're all being used),
171 # then this method will wait until a thread has checked in a connection.
172 # The wait time is bounded however: if no connection can be checked out
173 # within the timeout specified for this pool, then a ConnectionTimeoutError
174 # exception will be raised.
175 #
176 # Returns: an AbstractAdapter object.
177 #
178 # Raises:
179 # - ConnectionTimeoutError: no connection can be obtained from the pool
180 # within the timeout period.
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
181 def checkout
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
182 # Checkout an available connection
183 @connection_mutex.synchronize do
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
184 loop do
185 conn = if @checked_out.size < @connections.size
186 checkout_existing_connection
187 elsif @connections.size < @size
188 checkout_new_connection
189 end
190 return conn if conn
191 # No connections available; wait for one
192 if @queue.wait(@timeout)
193 next
194 else
8343611 @alk made ConnectionPool#checkout more robust by trying to loot dead threads ...
alk authored
195 # try looting dead threads
196 clear_stale_cached_connections!
197 if @size == @checked_out.size
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
198 raise ConnectionTimeoutError, "could not obtain a database connection#{" within #{@timeout} seconds" if @timeout}. The max pool size is currently #{@size}; consider increasing it."
8343611 @alk made ConnectionPool#checkout more robust by trying to loot dead threads ...
alk authored
199 end
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
200 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
201 end
202 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
203 end
204
a293278 @lifo Merge docrails
lifo authored
205 # Check-in a database connection back into the pool, indicating that you
206 # no longer need this connection.
207 #
208 # +conn+: an AbstractAdapter object, which was obtained by earlier by
209 # calling +checkout+ on this pool.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
210 def checkin(conn)
211 @connection_mutex.synchronize do
212 conn.run_callbacks :checkin
213 @checked_out.delete conn
214 @queue.signal
215 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
216 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
217
113cc4e @nicksieger Remove some synchronization that's probably overkill, assuming one doesn...
nicksieger authored
218 synchronize :clear_reloadable_connections!, :verify_active_connections!,
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
219 :connected?, :disconnect!, :with => :@connection_mutex
220
221 private
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
222 def new_connection
a3f12f5 @nicksieger Default connection allow_concurrency to false (for PostgreSQL)
nicksieger authored
223 ActiveRecord::Base.send(spec.adapter_method, spec.config)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
224 end
225
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
226 def current_connection_id #:nodoc:
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
227 Thread.current.object_id
228 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
229
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
230 # Remove stale threads from the cache.
231 def remove_stale_cached_threads!(cache, &block)
232 keys = Set.new(cache.keys)
233
234 Thread.list.each do |thread|
235 keys.delete(thread.object_id) if thread.alive?
236 end
237 keys.each do |key|
238 next unless cache.has_key?(key)
239 block.call(key, cache[key])
240 cache.delete(key)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
241 end
242 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
243
244 def checkout_new_connection
245 c = new_connection
246 @connections << c
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
247 checkout_and_verify(c)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
248 end
249
250 def checkout_existing_connection
d7d2d73 @nicksieger Fix typo: was using brackets instead of parens. Must need more sleep.
nicksieger authored
251 c = (@connections - @checked_out).first
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
252 checkout_and_verify(c)
253 end
254
255 def checkout_and_verify(c)
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
256 c.verify!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
257 c.run_callbacks :checkout
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
258 @checked_out << c
259 c
260 end
261 end
262
a293278 @lifo Merge docrails
lifo authored
263 # ConnectionHandler is a collection of ConnectionPool objects. It is used
264 # for keeping separate connection pools for ActiveRecord models that connect
265 # to different databases.
266 #
267 # For example, suppose that you have 5 models, with the following hierarchy:
268 #
269 # |
270 # +-- Book
271 # | |
272 # | +-- ScaryBook
273 # | +-- GoodBook
274 # +-- Author
275 # +-- BankAccount
276 #
277 # Suppose that Book is to connect to a separate database (i.e. one other
278 # than the default database). Then Book, ScaryBook and GoodBook will all use
279 # the same connection pool. Likewise, Author and BankAccount will use the
280 # same connection pool. However, the connection pool used by Author/BankAccount
281 # is not the same as the one used by Book/ScaryBook/GoodBook.
282 #
283 # Normally there is only a single ConnectionHandler instance, accessible via
284 # ActiveRecord::Base.connection_handler. ActiveRecord models use this to
285 # determine that connection pool that they should use.
ca6d717 @nicksieger Deprecate allow_concurrency and make it have no effect
nicksieger authored
286 class ConnectionHandler
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
287 def initialize(pools = {})
288 @connection_pools = pools
289 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
290
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
291 def connection_pools
292 @connection_pools ||= {}
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
293 end
294
295 def establish_connection(name, spec)
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
296 @connection_pools[name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
297 end
298
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
299 # Returns any connections in use by the current thread back to the pool,
300 # and also returns connections to the pool cached by threads that are no
301 # longer alive.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
302 def clear_active_connections!
529c271 @nicksieger Simplify dispatcher callbacks to eliminate unnecessary stale thread purg...
nicksieger authored
303 @connection_pools.each_value {|pool| pool.release_connection }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
304 end
305
306 # Clears the cache which maps classes
307 def clear_reloadable_connections!
308 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
309 end
310
311 def clear_all_connections!
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
312 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
313 end
314
315 # Verify active connections.
316 def verify_active_connections! #:nodoc:
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
317 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
318 end
319
320 # Locate the connection of the nearest super class. This can be an
321 # active or defined connection: if it is the latter, it will be
322 # opened and set as the active connection for the class it was defined
323 # for (not necessarily the current class).
324 def retrieve_connection(klass) #:nodoc:
325 pool = retrieve_connection_pool(klass)
326 (pool && pool.connection) or raise ConnectionNotEstablished
327 end
328
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
329 # Returns true if a connection that's accessible to this class has
330 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
331 def connected?(klass)
0832bc6 @lifo Make sure ActiveRecord::Base.connected? doesn't raise an exception for d...
lifo authored
332 conn = retrieve_connection_pool(klass)
333 conn ? conn.connected? : false
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
334 end
335
336 # Remove the connection for this class. This will close the active
337 # connection and the defined connection (if they exist). The result
338 # can be used as an argument for establish_connection, for easily
339 # re-establishing the connection.
340 def remove_connection(klass)
341 pool = @connection_pools[klass.name]
342 @connection_pools.delete_if { |key, value| value == pool }
343 pool.disconnect! if pool
344 pool.spec.config if pool
345 end
346
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
347 def retrieve_connection_pool(klass)
3007545 @nicksieger Minor tweak to retrieve_connection_pool -- recurse instead of loop
nicksieger authored
348 pool = @connection_pools[klass.name]
349 return pool if pool
350 return nil if ActiveRecord::Base == klass
351 retrieve_connection_pool klass.superclass
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
352 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
353 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
354 end
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
355 end
Something went wrong with that request. Please try again.