Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 645 lines (566 sloc) 22.044 kb
d869298 @raggi Don't depend on rubygems loading thread (for Mutex)
raggi authored
1 require 'thread'
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
2 require 'thread_safe'
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
3 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
4 require 'set'
5
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
6 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
7 # Raised when a connection could not be obtained within the connection
5b7cfc5 @jrochkind ConnectionPool, unify exceptions, ConnectionTimeoutError
jrochkind authored
8 # acquisition timeout period: because max connections in pool
a30b8d3 @frodsan rename AR::Model::Tag to AR::Tag - fixes #7714
frodsan authored
9 # are in use.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
10 class ConnectionTimeoutError < ConnectionNotEstablished
11 end
12
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
13 module ConnectionAdapters
f17159b @fxn edit pass: the names of Rails components have a space, ie, "Active Recor...
fxn authored
14 # Connection pool base class for managing Active Record database
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
15 # connections.
16 #
a293278 @lifo Merge docrails
lifo authored
17 # == Introduction
18 #
19 # A connection pool synchronizes thread access to a limited number of
20 # database connections. The basic idea is that each thread checks out a
21 # database connection from the pool, uses that connection, and checks the
22 # connection back in. ConnectionPool is completely thread-safe, and will
23 # ensure that a connection cannot be used by two threads at the same time,
24 # as long as ConnectionPool's contract is correctly followed. It will also
25 # handle cases in which there are more threads than connections: if all
26 # connections have been checked out, and a thread tries to checkout a
27 # connection anyway, then ConnectionPool will wait until some other thread
28 # has checked in a connection.
29 #
30 # == Obtaining (checking out) a connection
31 #
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
32 # Connections can be obtained and used from a connection pool in several
33 # ways:
34 #
f17159b @fxn edit pass: the names of Rails components have a space, ie, "Active Recor...
fxn authored
35 # 1. Simply use ActiveRecord::Base.connection as with Active Record 2.1 and
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
36 # earlier (pre-connection-pooling). Eventually, when you're done with
37 # the connection(s) and wish it to be returned to the pool, you call
38 # ActiveRecord::Base.clear_active_connections!. This will be the
f17159b @fxn edit pass: the names of Rails components have a space, ie, "Active Recor...
fxn authored
39 # default behavior for Active Record when used in conjunction with
40 # Action Pack's request handling cycle.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
41 # 2. Manually check out a connection from the pool with
42 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
43 # returning this connection to the pool when finished by calling
44 # ActiveRecord::Base.connection_pool.checkin(connection).
45 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
46 # obtains a connection, yields it as the sole argument to the block,
47 # and returns it to the pool after the block completes.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
48 #
a293278 @lifo Merge docrails
lifo authored
49 # Connections in the pool are actually AbstractAdapter objects (or objects
50 # compatible with AbstractAdapter's interface).
51 #
52 # == Options
53 #
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
54 # There are several connection-pooling-related options that you can add to
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
55 # your database connection configuration:
56 #
57 # * +pool+: number indicating size of connection pool (default 5)
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
58 # * +checkout_timeout+: number of seconds to block and wait for a connection
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
59 # before giving up and raising a timeout error (default 5 seconds).
82b05fb @rafaelfranca Whitespaces :scissors:
rafaelfranca authored
60 # * +reaping_frequency+: frequency in seconds to periodically run the
61 # Reaper, which attempts to find and close dead connections, which can
62 # occur if a programmer forgets to close a connection at the end of a
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
63 # thread or a thread dies unexpectedly. (Default nil, which means don't
82b05fb @rafaelfranca Whitespaces :scissors:
rafaelfranca authored
64 # run the Reaper).
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
65 # * +dead_connection_timeout+: number of seconds from last checkout
66 # after which the Reaper will consider a connection reapable. (default
82b05fb @rafaelfranca Whitespaces :scissors:
rafaelfranca authored
67 # 5 seconds).
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
68 class ConnectionPool
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
69 # Threadsafe, fair, FIFO queue. Meant to be used by ConnectionPool
70 # with which it shares a Monitor. But could be a generic Queue.
71 #
72 # The Queue in stdlib's 'thread' could replace this class except
73 # stdlib's doesn't support waiting with a timeout.
74 class Queue
75 def initialize(lock = Monitor.new)
76 @lock = lock
77 @cond = @lock.new_cond
78 @num_waiting = 0
79 @queue = []
80 end
81
82 # Test if any threads are currently waiting on the queue.
83 def any_waiting?
84 synchronize do
85 @num_waiting > 0
86 end
87 end
88
89 # Return the number of threads currently waiting on this
90 # queue.
91 def num_waiting
92 synchronize do
93 @num_waiting
94 end
95 end
96
97 # Add +element+ to the queue. Never blocks.
98 def add(element)
99 synchronize do
100 @queue.push element
101 @cond.signal
102 end
103 end
104
105 # If +element+ is in the queue, remove and return it, or nil.
106 def delete(element)
107 synchronize do
108 @queue.delete(element)
109 end
110 end
111
112 # Remove all elements from the queue.
113 def clear
114 synchronize do
115 @queue.clear
116 end
117 end
118
119 # Remove the head of the queue.
120 #
121 # If +timeout+ is not given, remove and return the head the
122 # queue if the number of available elements is strictly
123 # greater than the number of threads currently waiting (that
124 # is, don't jump ahead in line). Otherwise, return nil.
125 #
126 # If +timeout+ is given, block if it there is no element
127 # available, waiting up to +timeout+ seconds for an element to
128 # become available.
129 #
130 # Raises:
131 # - ConnectionTimeoutError if +timeout+ is given and no element
132 # becomes available after +timeout+ seconds,
133 def poll(timeout = nil)
134 synchronize do
135 if timeout
136 no_wait_poll || wait_poll(timeout)
137 else
138 no_wait_poll
139 end
140 end
141 end
142
143 private
144
145 def synchronize(&block)
146 @lock.synchronize(&block)
147 end
148
149 # Test if the queue currently contains any elements.
150 def any?
151 !@queue.empty?
152 end
153
154 # A thread can remove an element from the queue without
155 # waiting if an only if the number of currently available
156 # connections is strictly greater than the number of waiting
157 # threads.
158 def can_remove_no_wait?
159 @queue.size > @num_waiting
160 end
161
162 # Removes and returns the head of the queue if possible, or nil.
163 def remove
164 @queue.shift
165 end
166
167 # Remove and return the head the queue if the number of
168 # available elements is strictly greater than the number of
169 # threads currently waiting. Otherwise, return nil.
170 def no_wait_poll
171 remove if can_remove_no_wait?
172 end
173
174 # Waits on the queue up to +timeout+ seconds, then removes and
175 # returns the head of the queue.
176 def wait_poll(timeout)
177 @num_waiting += 1
178
179 t0 = Time.now
180 elapsed = 0
181 loop do
182 @cond.wait(timeout - elapsed)
183
184 return remove if any?
185
186 elapsed = Time.now - t0
5b7cfc5 @jrochkind ConnectionPool, unify exceptions, ConnectionTimeoutError
jrochkind authored
187 if elapsed >= timeout
188 msg = 'could not obtain a database connection within %0.3f seconds (waited %0.3f seconds)' %
189 [timeout, elapsed]
190 raise ConnectionTimeoutError, msg
191 end
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
192 end
193 ensure
194 @num_waiting -= 1
195 end
196 end
197
41c24eb @tenderlove each connection pool has a reaper
tenderlove authored
198 # Every +frequency+ seconds, the reaper will call +reap+ on +pool+.
199 # A reaper instantiated with a nil frequency will never reap the
200 # connection pool.
641b43e @tenderlove updating the reaping frequency documentation
tenderlove authored
201 #
202 # Configure the frequency by setting "reaping_frequency" in your
82b05fb @rafaelfranca Whitespaces :scissors:
rafaelfranca authored
203 # database yaml file.
cde7692 @tenderlove introduce a timer class for reaping connections
tenderlove authored
204 class Reaper
205 attr_reader :pool, :frequency
206
207 def initialize(pool, frequency)
208 @pool = pool
209 @frequency = frequency
210 end
211
59f2696 @tenderlove rename start to run and use Thread.pass rather than sleeping to schedule...
tenderlove authored
212 def run
cde7692 @tenderlove introduce a timer class for reaping connections
tenderlove authored
213 return unless frequency
214 Thread.new(frequency, pool) { |t, p|
215 while true
216 sleep t
217 p.reap
218 end
219 }
220 end
221 end
222
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
223 include MonitorMixin
224
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
225 attr_accessor :automatic_reconnect, :checkout_timeout, :dead_connection_timeout
41c24eb @tenderlove each connection pool has a reaper
tenderlove authored
226 attr_reader :spec, :connections, :size, :reaper
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
227
a293278 @lifo Merge docrails
lifo authored
228 # Creates a new ConnectionPool object. +spec+ is a ConnectionSpecification
229 # object which describes database connection information (e.g. adapter,
230 # host name, username, password, etc), as well as the maximum size for
231 # this ConnectionPool.
232 #
233 # The default ConnectionPool maximum size is 5.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
234 def initialize(spec)
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
235 super()
236
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
237 @spec = spec
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
238
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
239 @checkout_timeout = spec.config[:checkout_timeout] || 5
a8eb13a @amatsuda Default dead_connection_timeout to 5
amatsuda authored
240 @dead_connection_timeout = spec.config[:dead_connection_timeout] || 5
41c24eb @tenderlove each connection pool has a reaper
tenderlove authored
241 @reaper = Reaper.new self, spec.config[:reaping_frequency]
59f2696 @tenderlove rename start to run and use Thread.pass rather than sleeping to schedule...
tenderlove authored
242 @reaper.run
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
243
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
244 # default max pool size to 5
245 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
dd77733 @jeremy Timeout the connection pool monitor on ruby 1.8 only
jeremy authored
246
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
247 # The cache of reserved connections mapped to threads
248 @reserved_connections = ThreadSafe::Cache.new(:initial_capacity => @size)
249
7db90aa @jonleighton Make it the responsibility of the connection to hold onto an ARel visito...
jonleighton authored
250 @connections = []
acccb72 @tenderlove column cache now lives on the connection pool
tenderlove authored
251 @automatic_reconnect = true
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
252
253 @available = Queue.new self
254 end
255
256 # Hack for tests to be able to add connections. Do not call outside of tests
257 def insert_connection_for_test!(c) #:nodoc:
258 synchronize do
259 @connections << c
260 @available.add c
261 end
c94651f @tenderlove almost fisted
tenderlove authored
262 end
263
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
264 # Retrieve the connection associated with the current thread, or call
265 # #checkout to obtain one if necessary.
266 #
267 # #connection can be called any number of times; the connection is
268 # held in a hash keyed by the thread id.
269 def connection
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
270 # this is correctly done double-checked locking
271 # (ThreadSafe::Cache's lookups have volatile semantics)
272 @reserved_connections[current_connection_id] || synchronize do
c2d416f @pmahoney Synchronize read and modification of @reserved_connections hash to avoid...
pmahoney authored
273 @reserved_connections[current_connection_id] ||= checkout
274 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
275 end
276
d523504 @tenderlove make active_connection? return true only if there is an open connection ...
tenderlove authored
277 # Is there an open connection that is being used for the current thread?
4211866 @tenderlove adding active_connection? to the connection pool
tenderlove authored
278 def active_connection?
c2d416f @pmahoney Synchronize read and modification of @reserved_connections hash to avoid...
pmahoney authored
279 synchronize do
280 @reserved_connections.fetch(current_connection_id) {
281 return false
282 }.in_use?
283 end
4211866 @tenderlove adding active_connection? to the connection pool
tenderlove authored
284 end
285
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
286 # Signal that the thread is finished with the current connection.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integration ...
nicksieger authored
287 # #release_connection releases the connection-thread association
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
288 # and returns the connection to the pool.
3344520 @tenderlove reduce the number of times current_connection_id is called in with_conne...
tenderlove authored
289 def release_connection(with_id = current_connection_id)
b8f7482 @tenderlove opening a connection will block if the pool is full
tenderlove authored
290 synchronize do
291 conn = @reserved_connections.delete(with_id)
292 checkin conn if conn
293 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
294 end
295
0034b78 @smartinez87 Remove extra white spaces on ActiveRecord docs.
smartinez87 authored
296 # If a connection already exists yield it to the block. If no connection
b451de0 @spastorino Deletes trailing whitespaces (over text files only find * -type f -exec ...
spastorino authored
297 # exists checkout a connection, yield it to the block, and checkin the
5501b99 @coderrr Ensure ActiveRecord::Base.connection_pool.with_connection creates a new ...
coderrr authored
298 # connection when finished.
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
299 def with_connection
3344520 @tenderlove reduce the number of times current_connection_id is called in with_conne...
tenderlove authored
300 connection_id = current_connection_id
ce3d8d6 @tenderlove Start implementing @reserved_connections in terms of connection leases.
tenderlove authored
301 fresh_connection = true unless active_connection?
5501b99 @coderrr Ensure ActiveRecord::Base.connection_pool.with_connection creates a new ...
coderrr authored
302 yield connection
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
303 ensure
3344520 @tenderlove reduce the number of times current_connection_id is called in with_conne...
tenderlove authored
304 release_connection(connection_id) if fresh_connection
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
305 end
306
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
307 # Returns true if a connection has already been opened.
308 def connected?
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
309 synchronize { @connections.any? }
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
310 end
311
a293278 @lifo Merge docrails
lifo authored
312 # Disconnects all connections in the pool, and clears the pool.
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve/re...
nicksieger authored
313 def disconnect!
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
314 synchronize do
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
315 @reserved_connections.clear
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
316 @connections.each do |conn|
317 checkin conn
318 conn.disconnect!
319 end
320 @connections = []
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
321 @available.clear
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
322 end
323 end
324
8f1b141 @smartinez87 Fixed punctuation errors.
smartinez87 authored
325 # Clears the cache which maps classes.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
326 def clear_reloadable_connections!
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
327 synchronize do
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
328 @reserved_connections.clear
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
329 @connections.each do |conn|
330 checkin conn
331 conn.disconnect! if conn.requires_reloading?
332 end
333 @connections.delete_if do |conn|
334 conn.requires_reloading?
335 end
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
336 @available.clear
337 @connections.each do |conn|
338 @available.add conn
339 end
62c4e4d @ebeigarts Fix connection reloading in development mode. [#4929 state:resolved]
ebeigarts authored
340 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
341 end
342
cceabe0 @tenderlove raise a pull full error when the connection pool is full and no connecti...
tenderlove authored
343 def clear_stale_cached_connections! # :nodoc:
27ebb3d @jrochkind deprecated clear_stale_active_connections! can call #reap instead of no-...
jrochkind authored
344 reap
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
345 end
27ebb3d @jrochkind deprecated clear_stale_active_connections! can call #reap instead of no-...
jrochkind authored
346 deprecate :clear_stale_cached_connections! => "Please use #reap instead"
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
347
a293278 @lifo Merge docrails
lifo authored
348 # Check-out a database connection from the pool, indicating that you want
349 # to use it. You should call #checkin when you no longer need this.
350 #
cceabe0 @tenderlove raise a pull full error when the connection pool is full and no connecti...
tenderlove authored
351 # This is done by either returning and leasing existing connection, or by
352 # creating a new connection and leasing it.
353 #
354 # If all connections are leased and the pool is at capacity (meaning the
355 # number of currently leased connections is greater than or equal to the
5b7cfc5 @jrochkind ConnectionPool, unify exceptions, ConnectionTimeoutError
jrochkind authored
356 # size limit set), an ActiveRecord::ConnectionTimeoutError exception will be raised.
a293278 @lifo Merge docrails
lifo authored
357 #
358 # Returns: an AbstractAdapter object.
359 #
360 # Raises:
5b7cfc5 @jrochkind ConnectionPool, unify exceptions, ConnectionTimeoutError
jrochkind authored
361 # - ConnectionTimeoutError: no connection can be obtained from the pool.
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
362 def checkout
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
363 synchronize do
364 conn = acquire_connection
365 conn.lease
366 checkout_and_verify(conn)
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
367 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
368 end
369
a293278 @lifo Merge docrails
lifo authored
370 # Check-in a database connection back into the pool, indicating that you
371 # no longer need this connection.
372 #
373 # +conn+: an AbstractAdapter object, which was obtained by earlier by
374 # calling +checkout+ on this pool.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
375 def checkin(conn)
c606fe2 @tenderlove push synchronization in to each method. Reduces method calls and makes
tenderlove authored
376 synchronize do
57bc25c @jfirebaugh Use run_callbacks; the generated _run_<name>_callbacks method is not a p...
jfirebaugh authored
377 conn.run_callbacks :checkin do
b72b477 @tenderlove Use connection lease to determine "checked_out" connections
tenderlove authored
378 conn.expire
471a394 @nicksieger Modify connection pool callbacks to be compatible w/ new style
nicksieger authored
379 end
0210c44 @tenderlove make sure connections returned after close are marked as in_use
tenderlove authored
380
381 release conn
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
382
383 @available.add conn
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
384 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
385 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
386
17ff88c @tenderlove connections can be removed from the pool
tenderlove authored
387 # Remove a connection from the connection pool. The connection will
388 # remain open and active but will no longer be managed by this pool.
389 def remove(conn)
390 synchronize do
391 @connections.delete conn
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
392 @available.delete conn
e060cf0 @tenderlove deal with removing connections associated with the current thread
tenderlove authored
393
394 # FIXME: we might want to store the key on the connection so that removing
395 # from the reserved hash will be a little easier.
0210c44 @tenderlove make sure connections returned after close are marked as in_use
tenderlove authored
396 release conn
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
397
398 @available.add checkout_new_connection if @available.any_waiting?
17ff88c @tenderlove connections can be removed from the pool
tenderlove authored
399 end
400 end
401
86729eb @tenderlove connections can be reaped via the `reap` method
tenderlove authored
402 # Removes dead connections from the pool. A dead connection can occur
403 # if a programmer forgets to close a connection at the end of a thread
404 # or a thread dies unexpectedly.
405 def reap
406 synchronize do
cb6f839 @jrochkind ConnectionPool wait_timeout no longer used for different types of timeou...
jrochkind authored
407 stale = Time.now - @dead_connection_timeout
86729eb @tenderlove connections can be reaped via the `reap` method
tenderlove authored
408 connections.dup.each do |conn|
b1ac881 @tenderlove connections are only removed if they are inactve
tenderlove authored
409 remove conn if conn.in_use? && stale > conn.last_use && !conn.active?
86729eb @tenderlove connections can be reaped via the `reap` method
tenderlove authored
410 end
411 end
412 end
413
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
414 private
7db90aa @jonleighton Make it the responsibility of the connection to hold onto an ARel visito...
jonleighton authored
415
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
416 # Acquire a connection by one of 1) immediately removing one
417 # from the queue of available connections, 2) creating a new
418 # connection if the pool is not at capacity, 3) waiting on the
419 # queue for a connection to become available.
420 #
421 # Raises:
a30b8d3 @frodsan rename AR::Model::Tag to AR::Tag - fixes #7714
frodsan authored
422 # - ConnectionTimeoutError if a connection could not be acquired
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
423 def acquire_connection
424 if conn = @available.poll
425 conn
426 elsif @connections.size < @size
427 checkout_new_connection
428 else
5b7cfc5 @jrochkind ConnectionPool, unify exceptions, ConnectionTimeoutError
jrochkind authored
429 @available.poll(@checkout_timeout)
02b2335 @pmahoney Make connection pool fair with respect to waiting threads.
pmahoney authored
430 end
431 end
432
0210c44 @tenderlove make sure connections returned after close are marked as in_use
tenderlove authored
433 def release(conn)
d56f5c8 @mark-rushakoff Remove unused assignments
mark-rushakoff authored
434 thread_id = if @reserved_connections[current_connection_id] == conn
435 current_connection_id
0210c44 @tenderlove make sure connections returned after close are marked as in_use
tenderlove authored
436 else
d56f5c8 @mark-rushakoff Remove unused assignments
mark-rushakoff authored
437 @reserved_connections.keys.find { |k|
0210c44 @tenderlove make sure connections returned after close are marked as in_use
tenderlove authored
438 @reserved_connections[k] == conn
439 }
440 end
441
442 @reserved_connections.delete thread_id if thread_id
443 end
444
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
445 def new_connection
9e4c41c @jonleighton Remove ActiveRecord::Model
jonleighton authored
446 Base.send(spec.adapter_method, spec.config)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
447 end
448
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
449 def current_connection_id #:nodoc:
9e4c41c @jonleighton Remove ActiveRecord::Model
jonleighton authored
450 Base.connection_id ||= Thread.current.object_id
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
451 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
452
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
453 def checkout_new_connection
acccb72 @tenderlove column cache now lives on the connection pool
tenderlove authored
454 raise ConnectionNotEstablished unless @automatic_reconnect
455
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
456 c = new_connection
29d2040 @tenderlove AbstractAdapter#close can be called to add the connection back to the
tenderlove authored
457 c.pool = self
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
458 @connections << c
b72b477 @tenderlove Use connection lease to determine "checked_out" connections
tenderlove authored
459 c
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
460 end
461
462 def checkout_and_verify(c)
471a394 @nicksieger Modify connection pool callbacks to be compatible w/ new style
nicksieger authored
463 c.run_callbacks :checkout do
464 c.verify!
465 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
466 c
467 end
468 end
469
a293278 @lifo Merge docrails
lifo authored
470 # ConnectionHandler is a collection of ConnectionPool objects. It is used
f17159b @fxn edit pass: the names of Rails components have a space, ie, "Active Recor...
fxn authored
471 # for keeping separate connection pools for Active Record models that connect
a293278 @lifo Merge docrails
lifo authored
472 # to different databases.
473 #
474 # For example, suppose that you have 5 models, with the following hierarchy:
475 #
476 # |
477 # +-- Book
478 # | |
479 # | +-- ScaryBook
480 # | +-- GoodBook
481 # +-- Author
482 # +-- BankAccount
483 #
484 # Suppose that Book is to connect to a separate database (i.e. one other
485 # than the default database). Then Book, ScaryBook and GoodBook will all use
486 # the same connection pool. Likewise, Author and BankAccount will use the
487 # same connection pool. However, the connection pool used by Author/BankAccount
488 # is not the same as the one used by Book/ScaryBook/GoodBook.
489 #
490 # Normally there is only a single ConnectionHandler instance, accessible via
f17159b @fxn edit pass: the names of Rails components have a space, ie, "Active Recor...
fxn authored
491 # ActiveRecord::Base.connection_handler. Active Record models use this to
2c96638 @alup Fix a typo
alup authored
492 # determine the connection pool that they should use.
ca6d717 @nicksieger Deprecate allow_concurrency and make it have no effect
nicksieger authored
493 class ConnectionHandler
ba1544d @jonleighton One hash is enough
jonleighton authored
494 def initialize
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
495 # These caches are keyed by klass.name, NOT klass. Keying them by klass
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
496 # alone would lead to memory leaks in development mode as all previous
497 # instances of the class would stay in memory.
45448a5 @thedarkone Replace some global Hash usages with the new thread safe cache.
thedarkone authored
498 @owner_to_pool = ThreadSafe::Cache.new(:initial_capacity => 2) do |h,k|
499 h[k] = ThreadSafe::Cache.new(:initial_capacity => 2)
500 end
501 @class_to_pool = ThreadSafe::Cache.new(:initial_capacity => 2) do |h,k|
502 h[k] = ThreadSafe::Cache.new
503 end
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
504 end
505
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
506 def connection_pool_list
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
507 owner_to_pool.values.compact
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
508 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
509
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
510 def connection_pools
511 ActiveSupport::Deprecation.warn(
512 "In the next release, this will return the same as #connection_pool_list. " \
513 "(An array of pools, rather than a hash mapping specs to pools.)"
514 )
515 Hash[connection_pool_list.map { |pool| [pool.spec, pool] }]
516 end
517
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
518 def establish_connection(owner, spec)
519 @class_to_pool.clear
bc43763 @dahakawang fix anonymous class issue
dahakawang authored
520 raise RuntimeError, "Anonymous class is not allowed." unless owner.name
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
521 owner_to_pool[owner.name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
522 end
523
25f9497 @tenderlove adding active_connections? to the connection pool for finding open conne...
tenderlove authored
524 # Returns true if there are any active connections among the connection
525 # pools that the ConnectionHandler is managing.
526 def active_connections?
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
527 connection_pool_list.any?(&:active_connection?)
25f9497 @tenderlove adding active_connections? to the connection pool for finding open conne...
tenderlove authored
528 end
529
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to the...
nicksieger authored
530 # Returns any connections in use by the current thread back to the pool,
531 # and also returns connections to the pool cached by threads that are no
532 # longer alive.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
533 def clear_active_connections!
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
534 connection_pool_list.each(&:release_connection)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
535 end
536
8f1b141 @smartinez87 Fixed punctuation errors.
smartinez87 authored
537 # Clears the cache which maps classes.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
538 def clear_reloadable_connections!
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
539 connection_pool_list.each(&:clear_reloadable_connections!)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
540 end
541
542 def clear_all_connections!
c3ca7ac @jonleighton Properly deprecate ConnectionHandler#connection_pools
jonleighton authored
543 connection_pool_list.each(&:disconnect!)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
544 end
545
546 # Locate the connection of the nearest super class. This can be an
547 # active or defined connection: if it is the latter, it will be
548 # opened and set as the active connection for the class it was defined
549 # for (not necessarily the current class).
550 def retrieve_connection(klass) #:nodoc:
551 pool = retrieve_connection_pool(klass)
552 (pool && pool.connection) or raise ConnectionNotEstablished
553 end
554
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
555 # Returns true if a connection that's accessible to this class has
556 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
557 def connected?(klass)
0832bc6 @lifo Make sure ActiveRecord::Base.connected? doesn't raise an exception for d...
lifo authored
558 conn = retrieve_connection_pool(klass)
a4458f5 @tenderlove removing useless ternary
tenderlove authored
559 conn && conn.connected?
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
560 end
561
562 # Remove the connection for this class. This will close the active
563 # connection and the defined connection (if they exist). The result
564 # can be used as an argument for establish_connection, for easily
565 # re-establishing the connection.
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
566 def remove_connection(owner)
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
567 if pool = owner_to_pool.delete(owner.name)
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
568 @class_to_pool.clear
ba1544d @jonleighton One hash is enough
jonleighton authored
569 pool.automatic_reconnect = false
570 pool.disconnect!
571 pool.spec.config
572 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHandle...
nicksieger authored
573 end
574
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
575 # Retrieving the connection pool happens a lot so we cache it in @class_to_pool.
576 # This makes retrieving the connection pool O(1) once the process is warm.
577 # When a connection is established or removed, we invalidate the cache.
578 #
579 # Ideally we would use #fetch here, as class_to_pool[klass] may sometimes be nil.
57b0ae8 @amatsuda Gist URLs are now namespaced
amatsuda authored
580 # However, benchmarking (https://gist.github.com/jonleighton/3552829) showed that
581 # #fetch is significantly slower than #[]. So in the nil case, no caching will
582 # take place, but that's ok since the nil case is not the common one that we wish
583 # to optimise for.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
584 def retrieve_connection_pool(klass)
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
585 class_to_pool[klass.name] ||= begin
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
586 until pool = pool_for(klass)
587 klass = klass.superclass
9e4c41c @jonleighton Remove ActiveRecord::Model
jonleighton authored
588 break unless klass <= Base
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
589 end
221571b @jonleighton Make connection pool retrieval faster
jonleighton authored
590
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
591 class_to_pool[klass.name] = pool
e030f26 @jonleighton Simplify AR configuration code.
jonleighton authored
592 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implemented,...
nicksieger authored
593 end
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
594
595 private
596
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
597 def owner_to_pool
598 @owner_to_pool[Process.pid]
599 end
600
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
601 def class_to_pool
decafdd @tenderlove use Process.pid rather than $$
tenderlove authored
602 @class_to_pool[Process.pid]
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
603 end
604
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
605 def pool_for(owner)
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
606 owner_to_pool.fetch(owner.name) {
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
607 if ancestor_pool = pool_from_any_process_for(owner)
4a274ed @jonleighton Refactor connection handler
jonleighton authored
608 # A connection was established in an ancestor process that must have
609 # subsequently forked. We can't reuse the connection, but we can copy
610 # the specification and establish a new connection with it.
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
611 establish_connection owner, ancestor_pool.spec
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
612 else
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
613 owner_to_pool[owner.name] = nil
aaff1a4 @tenderlove database connections are automatically established after forking.
tenderlove authored
614 end
615 }
616 end
4a274ed @jonleighton Refactor connection handler
jonleighton authored
617
e96558f @jonleighton Cache the connection pool for a given class
jonleighton authored
618 def pool_from_any_process_for(owner)
68e4442 @jonleighton Fix memory leak in development mode
jonleighton authored
619 owner_to_pool = @owner_to_pool.values.find { |v| v[owner.name] }
620 owner_to_pool && owner_to_pool[owner.name]
4a274ed @jonleighton Refactor connection handler
jonleighton authored
621 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
622 end
1b22071 @josh Ensure ActiveRecord session store's connections are checked in after eac...
josh authored
623
624 class ConnectionManagement
625 def initialize(app)
626 @app = app
627 end
628
629 def call(env)
c7b7c6a @tenderlove make sure that active connections are not cleared during test when an ex...
tenderlove authored
630 testing = env.key?('rack.test')
631
2b81240 @lest use Rack::BodyProxy in activerecord middlewares
lest authored
632 response = @app.call(env)
633 response[2] = ::Rack::BodyProxy.new(response[2]) do
634 ActiveRecord::Base.clear_active_connections! unless testing
635 end
e524609 @tenderlove proxy body responses so we close database connections after body is flus...
tenderlove authored
636
2b81240 @lest use Rack::BodyProxy in activerecord middlewares
lest authored
637 response
3b2a032 @tenderlove clearing active connections in the ConnectionManagement middleware if an...
tenderlove authored
638 rescue
c7b7c6a @tenderlove make sure that active connections are not cleared during test when an ex...
tenderlove authored
639 ActiveRecord::Base.clear_active_connections! unless testing
3b2a032 @tenderlove clearing active connections in the ConnectionManagement middleware if an...
tenderlove authored
640 raise
1b22071 @josh Ensure ActiveRecord session store's connections are checked in after eac...
josh authored
641 end
642 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
643 end
21eb18a @alk Fix race in ConnectionPool#checkout
alk authored
644 end
Something went wrong with that request. Please try again.