Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 348 lines (315 sloc) 12.781 kB
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
1 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
a293278 @lifo Merge docrails
lifo authored
14 # == Introduction
15 #
16 # A connection pool synchronizes thread access to a limited number of
17 # database connections. The basic idea is that each thread checks out a
18 # database connection from the pool, uses that connection, and checks the
19 # connection back in. ConnectionPool is completely thread-safe, and will
20 # ensure that a connection cannot be used by two threads at the same time,
21 # as long as ConnectionPool's contract is correctly followed. It will also
22 # handle cases in which there are more threads than connections: if all
23 # connections have been checked out, and a thread tries to checkout a
24 # connection anyway, then ConnectionPool will wait until some other thread
25 # has checked in a connection.
26 #
27 # == Obtaining (checking out) a connection
28 #
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
29 # Connections can be obtained and used from a connection pool in several
30 # ways:
31 #
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
32 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
33 # earlier (pre-connection-pooling). Eventually, when you're done with
34 # the connection(s) and wish it to be returned to the pool, you call
35 # ActiveRecord::Base.clear_active_connections!. This will be the
36 # default behavior for ActiveRecord when used in conjunction with
37 # ActionPack's request handling cycle.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
38 # 2. Manually check out a connection from the pool with
39 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
40 # returning this connection to the pool when finished by calling
41 # ActiveRecord::Base.connection_pool.checkin(connection).
42 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
43 # obtains a connection, yields it as the sole argument to the block,
44 # and returns it to the pool after the block completes.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
45 #
a293278 @lifo Merge docrails
lifo authored
46 # Connections in the pool are actually AbstractAdapter objects (or objects
47 # compatible with AbstractAdapter's interface).
48 #
49 # == Options
50 #
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
51 # There are two connection-pooling-related options that you can add to
52 # your database connection configuration:
53 #
54 # * +pool+: number indicating size of connection pool (default 5)
55 # * +wait_timeout+: number of seconds to block and wait for a connection
56 # before giving up and raising a timeout error (default 5 seconds).
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
57 class ConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
58 attr_reader :spec
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
59
a293278 @lifo Merge docrails
lifo authored
60 # Creates a new ConnectionPool object. +spec+ is a ConnectionSpecification
61 # object which describes database connection information (e.g. adapter,
62 # host name, username, password, etc), as well as the maximum size for
63 # this ConnectionPool.
64 #
65 # The default ConnectionPool maximum size is 5.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
66 def initialize(spec)
67 @spec = spec
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
68 # The cache of reserved connections mapped to threads
69 @reserved_connections = {}
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
70 # The mutex used to synchronize pool access
71 @connection_mutex = Monitor.new
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
72 @queue = @connection_mutex.new_cond
73 # default 5 second timeout
74 @timeout = spec.config[:wait_timeout] || 5
75 # default max pool size to 5
76 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
77 @connections = []
78 @checked_out = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
79 end
80
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
81 # Retrieve the connection associated with the current thread, or call
82 # #checkout to obtain one if necessary.
83 #
84 # #connection can be called any number of times; the connection is
85 # held in a hash keyed by the thread id.
86 def connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
87 if conn = @reserved_connections[current_connection_id]
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
88 conn
89 else
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
90 @reserved_connections[current_connection_id] = checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
91 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
92 end
93
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
94 # Signal that the thread is finished with the current connection.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
95 # #release_connection releases the connection-thread association
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
96 # and returns the connection to the pool.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
97 def release_connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
98 conn = @reserved_connections.delete(current_connection_id)
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
99 checkin conn if conn
100 end
101
102 # Reserve a connection, and yield it to a block. Ensure the connection is
103 # checked back in when finished.
104 def with_connection
105 conn = checkout
106 yield conn
107 ensure
108 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
109 end
110
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
111 # Returns true if a connection has already been opened.
112 def connected?
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
113 !@connections.empty?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
114 end
115
a293278 @lifo Merge docrails
lifo authored
116 # Disconnects all connections in the pool, and clears the pool.
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
117 def disconnect!
118 @reserved_connections.each do |name,conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
119 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
120 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
121 @reserved_connections = {}
122 @connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
123 conn.disconnect!
124 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
125 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
126 end
127
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
128 # Clears the cache which maps classes
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
129 def clear_reloadable_connections!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
130 @reserved_connections.each do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
132 end
133 @reserved_connections = {}
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
134 @connections.each do |conn|
135 conn.disconnect! if conn.requires_reloading?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
136 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
137 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
138 end
139
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
140 # Verify active connections and remove and disconnect connections
141 # associated with stale threads.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
142 def verify_active_connections! #:nodoc:
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
143 clear_stale_cached_connections!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
144 @connections.each do |connection|
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
145 connection.verify!
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
146 end
147 end
148
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
149 # Return any checked-out connections back to the pool by threads that
150 # are no longer alive.
151 def clear_stale_cached_connections!
152 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
153 checkin conn
154 end
155 end
156
a293278 @lifo Merge docrails
lifo authored
157 # Check-out a database connection from the pool, indicating that you want
158 # to use it. You should call #checkin when you no longer need this.
159 #
160 # This is done by either returning an existing connection, or by creating
161 # a new connection. If the maximum number of connections for this pool has
162 # already been reached, but the pool is empty (i.e. they're all being used),
163 # then this method will wait until a thread has checked in a connection.
164 # The wait time is bounded however: if no connection can be checked out
165 # within the timeout specified for this pool, then a ConnectionTimeoutError
166 # exception will be raised.
167 #
168 # Returns: an AbstractAdapter object.
169 #
170 # Raises:
171 # - ConnectionTimeoutError: no connection can be obtained from the pool
172 # within the timeout period.
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
173 def checkout
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
174 # Checkout an available connection
175 @connection_mutex.synchronize do
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
176 loop do
177 conn = if @checked_out.size < @connections.size
178 checkout_existing_connection
179 elsif @connections.size < @size
180 checkout_new_connection
181 end
182 return conn if conn
183 # No connections available; wait for one
184 if @queue.wait(@timeout)
185 next
186 else
8343611 @alk made ConnectionPool#checkout more robust by trying to loot dead threa…
alk authored
187 # try looting dead threads
188 clear_stale_cached_connections!
189 if @size == @checked_out.size
190 raise ConnectionTimeoutError, "could not obtain a database connection within #{@timeout} seconds. The pool size is currently #{@size}, perhaps you need to increase it?"
191 end
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
192 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
193 end
194 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
195 end
196
a293278 @lifo Merge docrails
lifo authored
197 # Check-in a database connection back into the pool, indicating that you
198 # no longer need this connection.
199 #
200 # +conn+: an AbstractAdapter object, which was obtained by earlier by
201 # calling +checkout+ on this pool.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
202 def checkin(conn)
203 @connection_mutex.synchronize do
204 conn.run_callbacks :checkin
205 @checked_out.delete conn
206 @queue.signal
207 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
208 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
209
113cc4e @nicksieger Remove some synchronization that's probably overkill, assuming one do…
nicksieger authored
210 synchronize :clear_reloadable_connections!, :verify_active_connections!,
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
211 :connected?, :disconnect!, :with => :@connection_mutex
212
213 private
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
214 def new_connection
a3f12f5 @nicksieger Default connection allow_concurrency to false (for PostgreSQL)
nicksieger authored
215 ActiveRecord::Base.send(spec.adapter_method, spec.config)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
216 end
217
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
218 def current_connection_id #:nodoc:
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
219 Thread.current.object_id
220 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
221
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
222 # Remove stale threads from the cache.
223 def remove_stale_cached_threads!(cache, &block)
224 keys = Set.new(cache.keys)
225
226 Thread.list.each do |thread|
227 keys.delete(thread.object_id) if thread.alive?
228 end
229 keys.each do |key|
230 next unless cache.has_key?(key)
231 block.call(key, cache[key])
232 cache.delete(key)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
233 end
234 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
235
236 def checkout_new_connection
237 c = new_connection
238 @connections << c
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
239 checkout_and_verify(c)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
240 end
241
242 def checkout_existing_connection
d7d2d73 @nicksieger Fix typo: was using brackets instead of parens. Must need more sleep.
nicksieger authored
243 c = (@connections - @checked_out).first
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
244 checkout_and_verify(c)
245 end
246
247 def checkout_and_verify(c)
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
248 c.verify!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
249 c.run_callbacks :checkout
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
250 @checked_out << c
251 c
252 end
253 end
254
a293278 @lifo Merge docrails
lifo authored
255 # ConnectionHandler is a collection of ConnectionPool objects. It is used
256 # for keeping separate connection pools for ActiveRecord models that connect
257 # to different databases.
258 #
259 # For example, suppose that you have 5 models, with the following hierarchy:
260 #
261 # |
262 # +-- Book
263 # | |
264 # | +-- ScaryBook
265 # | +-- GoodBook
266 # +-- Author
267 # +-- BankAccount
268 #
269 # Suppose that Book is to connect to a separate database (i.e. one other
270 # than the default database). Then Book, ScaryBook and GoodBook will all use
271 # the same connection pool. Likewise, Author and BankAccount will use the
272 # same connection pool. However, the connection pool used by Author/BankAccount
273 # is not the same as the one used by Book/ScaryBook/GoodBook.
274 #
275 # Normally there is only a single ConnectionHandler instance, accessible via
276 # ActiveRecord::Base.connection_handler. ActiveRecord models use this to
277 # determine that connection pool that they should use.
ca6d717 @nicksieger Deprecate allow_concurrency and make it have no effect
nicksieger authored
278 class ConnectionHandler
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
279 def initialize(pools = {})
280 @connection_pools = pools
281 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
282
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
283 def connection_pools
284 @connection_pools ||= {}
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
285 end
286
287 def establish_connection(name, spec)
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
288 @connection_pools[name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
289 end
290
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
291 # Returns any connections in use by the current thread back to the pool,
292 # and also returns connections to the pool cached by threads that are no
293 # longer alive.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
294 def clear_active_connections!
529c271 @nicksieger Simplify dispatcher callbacks to eliminate unnecessary stale thread p…
nicksieger authored
295 @connection_pools.each_value {|pool| pool.release_connection }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
296 end
297
298 # Clears the cache which maps classes
299 def clear_reloadable_connections!
300 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
301 end
302
303 def clear_all_connections!
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
304 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
305 end
306
307 # Verify active connections.
308 def verify_active_connections! #:nodoc:
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
309 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
310 end
311
312 # Locate the connection of the nearest super class. This can be an
313 # active or defined connection: if it is the latter, it will be
314 # opened and set as the active connection for the class it was defined
315 # for (not necessarily the current class).
316 def retrieve_connection(klass) #:nodoc:
317 pool = retrieve_connection_pool(klass)
318 (pool && pool.connection) or raise ConnectionNotEstablished
319 end
320
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
321 # Returns true if a connection that's accessible to this class has
322 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
323 def connected?(klass)
0832bc6 @lifo Make sure ActiveRecord::Base.connected? doesn't raise an exception fo…
lifo authored
324 conn = retrieve_connection_pool(klass)
325 conn ? conn.connected? : false
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
326 end
327
328 # Remove the connection for this class. This will close the active
329 # connection and the defined connection (if they exist). The result
330 # can be used as an argument for establish_connection, for easily
331 # re-establishing the connection.
332 def remove_connection(klass)
333 pool = @connection_pools[klass.name]
334 @connection_pools.delete_if { |key, value| value == pool }
335 pool.disconnect! if pool
336 pool.spec.config if pool
337 end
338
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
339 def retrieve_connection_pool(klass)
3007545 @nicksieger Minor tweak to retrieve_connection_pool -- recurse instead of loop
nicksieger authored
340 pool = @connection_pools[klass.name]
341 return pool if pool
342 return nil if ActiveRecord::Base == klass
343 retrieve_connection_pool klass.superclass
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
344 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
345 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
346 end
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
347 end
Something went wrong with that request. Please try again.