Skip to content
Newer
Older
100644 282 lines (249 sloc) 9.24 KB
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
1 require 'monitor'
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
2 require 'set'
3
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
4 module ActiveRecord
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
10 module ConnectionAdapters
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
17 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
18 # earlier (pre-connection-pooling). Eventually, when you're done with
19 # the connection(s) and wish it to be returned to the pool, you call
20 # ActiveRecord::Base.clear_active_connections!. This will be the
21 # default behavior for ActiveRecord when used in conjunction with
22 # ActionPack's request handling cycle.
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
23 # 2. Manually check out a connection from the pool with
24 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
25 # returning this connection to the pool when finished by calling
26 # ActiveRecord::Base.connection_pool.checkin(connection).
27 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
28 # obtains a connection, yields it as the sole argument to the block,
29 # and returns it to the pool after the block completes.
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
30 #
31 # There are two connection-pooling-related options that you can add to
32 # your database connection configuration:
33 #
34 # * +pool+: number indicating size of connection pool (default 5)
35 # * +wait_timeout+: number of seconds to block and wait for a connection
36 # before giving up and raising a timeout error (default 5 seconds).
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
37 class ConnectionPool
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
38 attr_reader :spec
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
39
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
40 def initialize(spec)
41 @spec = spec
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
42 # The cache of reserved connections mapped to threads
43 @reserved_connections = {}
cab76ce @nicksieger Add synchronization to connection pool also
nicksieger authored
44 # The mutex used to synchronize pool access
45 @connection_mutex = Monitor.new
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
46 @queue = @connection_mutex.new_cond
47 # default 5 second timeout
48 @timeout = spec.config[:wait_timeout] || 5
49 # default max pool size to 5
50 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
51 @connections = []
52 @checked_out = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
53 end
54
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
55 # Retrieve the connection associated with the current thread, or call
56 # #checkout to obtain one if necessary.
57 #
58 # #connection can be called any number of times; the connection is
59 # held in a hash keyed by the thread id.
60 def connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
61 if conn = @reserved_connections[current_connection_id]
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
62 conn
63 else
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
64 @reserved_connections[current_connection_id] = checkout
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
65 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
66 end
67
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
68 # Signal that the thread is finished with the current connection.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
69 # #release_connection releases the connection-thread association
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
70 # and returns the connection to the pool.
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
71 def release_connection
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
72 conn = @reserved_connections.delete(current_connection_id)
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
73 checkin conn if conn
74 end
75
76 # Reserve a connection, and yield it to a block. Ensure the connection is
77 # checked back in when finished.
78 def with_connection
79 conn = checkout
80 yield conn
81 ensure
82 checkin conn
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
83 end
84
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
85 # Returns true if a connection has already been opened.
86 def connected?
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
87 !@connections.empty?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
88 end
89
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
90 # Disconnect all connections in the pool.
91 def disconnect!
92 @reserved_connections.each do |name,conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
93 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
94 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
95 @reserved_connections = {}
96 @connections.each do |conn|
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
97 conn.disconnect!
98 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
99 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
100 end
101
50cd4bd @nicksieger Introduce synchronization around connection pool access
nicksieger authored
102 # Clears the cache which maps classes
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
103 def clear_reloadable_connections!
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
104 @reserved_connections.each do |name, conn|
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
105 checkin conn
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
106 end
107 @reserved_connections = {}
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
108 @connections.each do |conn|
109 conn.disconnect! if conn.requires_reloading?
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
110 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
111 @connections = []
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
112 end
113
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
114 # Verify active connections and remove and disconnect connections
115 # associated with stale threads.
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
116 def verify_active_connections! #:nodoc:
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
117 clear_stale_cached_connections!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
118 @connections.each do |connection|
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
119 connection.verify!
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
120 end
121 end
122
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
123 # Return any checked-out connections back to the pool by threads that
124 # are no longer alive.
125 def clear_stale_cached_connections!
126 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
127 checkin conn
128 end
129 end
130
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
131 # Check-out a database connection from the pool.
132 def checkout
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
133 # Checkout an available connection
134 @connection_mutex.synchronize do
21eb18a Fix race in ConnectionPool#checkout
Aliaksey Kandratsenka authored
135 loop do
136 conn = if @checked_out.size < @connections.size
137 checkout_existing_connection
138 elsif @connections.size < @size
139 checkout_new_connection
140 end
141 return conn if conn
142 # No connections available; wait for one
143 if @queue.wait(@timeout)
144 next
145 else
8343611 made ConnectionPool#checkout more robust by trying to loot dead threa…
Aliaksey Kandratsenka authored
146 # try looting dead threads
147 clear_stale_cached_connections!
148 if @size == @checked_out.size
149 raise ConnectionTimeoutError, "could not obtain a database connection within #{@timeout} seconds. The pool size is currently #{@size}, perhaps you need to increase it?"
150 end
21eb18a Fix race in ConnectionPool#checkout
Aliaksey Kandratsenka authored
151 end
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
152 end
153 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
154 end
155
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
156 # Check-in a database connection back into the pool.
157 def checkin(conn)
158 @connection_mutex.synchronize do
159 conn.run_callbacks :checkin
160 @checked_out.delete conn
161 @queue.signal
162 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
163 end
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
164
113cc4e @nicksieger Remove some synchronization that's probably overkill, assuming one do…
nicksieger authored
165 synchronize :clear_reloadable_connections!, :verify_active_connections!,
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
166 :connected?, :disconnect!, :with => :@connection_mutex
167
168 private
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
169 def new_connection
a3f12f5 @nicksieger Default connection allow_concurrency to false (for PostgreSQL)
nicksieger authored
170 ActiveRecord::Base.send(spec.adapter_method, spec.config)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
171 end
172
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
173 def current_connection_id #:nodoc:
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
174 Thread.current.object_id
175 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
176
029952e @nicksieger Extract a base class for connection pools, start to flesh out reserve…
nicksieger authored
177 # Remove stale threads from the cache.
178 def remove_stale_cached_threads!(cache, &block)
179 keys = Set.new(cache.keys)
180
181 Thread.list.each do |thread|
182 keys.delete(thread.object_id) if thread.alive?
183 end
184 keys.each do |key|
185 next unless cache.has_key?(key)
186 block.call(key, cache[key])
187 cache.delete(key)
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
188 end
189 end
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
190
191 def checkout_new_connection
192 c = new_connection
193 @connections << c
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
194 checkout_and_verify(c)
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
195 end
196
197 def checkout_existing_connection
d7d2d73 @nicksieger Fix typo: was using brackets instead of parens. Must need more sleep.
nicksieger authored
198 c = (@connections - @checked_out).first
a96b7d4 @nicksieger Add connection reset and verification upon each connection checkout
nicksieger authored
199 checkout_and_verify(c)
200 end
201
202 def checkout_and_verify(c)
7ba2872 @nicksieger Deprecate verification_timeout and verify before reset
nicksieger authored
203 c.verify!
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
204 c.run_callbacks :checkout
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
205 @checked_out << c
206 c
207 end
208 end
209
ca6d717 @nicksieger Deprecate allow_concurrency and make it have no effect
nicksieger authored
210 class ConnectionHandler
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
211 def initialize(pools = {})
212 @connection_pools = pools
213 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
214
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
215 def connection_pools
216 @connection_pools ||= {}
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
217 end
218
219 def establish_connection(name, spec)
8e5e02b @nicksieger Collapse connection pool class hierarchy; YAGNI.
nicksieger authored
220 @connection_pools[name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
221 end
222
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
223 # Returns any connections in use by the current thread back to the pool,
224 # and also returns connections to the pool cached by threads that are no
225 # longer alive.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
226 def clear_active_connections!
d07a6b1 @nicksieger Make clear_active_connections! also return stale connections back to …
nicksieger authored
227 @connection_pools.each_value do |pool|
228 pool.release_connection
229 pool.clear_stale_cached_connections!
230 end
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
231 end
232
233 # Clears the cache which maps classes
234 def clear_reloadable_connections!
235 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
236 end
237
238 def clear_all_connections!
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
239 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
240 end
241
242 # Verify active connections.
243 def verify_active_connections! #:nodoc:
817a07b @nicksieger More doco and class/method renames. Now have a strategy for integrati…
nicksieger authored
244 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
245 end
246
247 # Locate the connection of the nearest super class. This can be an
248 # active or defined connection: if it is the latter, it will be
249 # opened and set as the active connection for the class it was defined
250 # for (not necessarily the current class).
251 def retrieve_connection(klass) #:nodoc:
252 pool = retrieve_connection_pool(klass)
253 (pool && pool.connection) or raise ConnectionNotEstablished
254 end
255
82fcd9d @nicksieger Clean up the code, get rid of reserve/release, add some more docs
nicksieger authored
256 # Returns true if a connection that's accessible to this class has
257 # already been opened.
ff97e9d @nicksieger Connection handling methods extracted out into separate ConnectionHan…
nicksieger authored
258 def connected?(klass)
259 retrieve_connection_pool(klass).connected?
260 end
261
262 # Remove the connection for this class. This will close the active
263 # connection and the defined connection (if they exist). The result
264 # can be used as an argument for establish_connection, for easily
265 # re-establishing the connection.
266 def remove_connection(klass)
267 pool = @connection_pools[klass.name]
268 @connection_pools.delete_if { |key, value| value == pool }
269 pool.disconnect! if pool
270 pool.spec.config if pool
271 end
272
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
273 def retrieve_connection_pool(klass)
3007545 @nicksieger Minor tweak to retrieve_connection_pool -- recurse instead of loop
nicksieger authored
274 pool = @connection_pools[klass.name]
275 return pool if pool
276 return nil if ActiveRecord::Base == klass
277 retrieve_connection_pool klass.superclass
fe575dd @nicksieger Nearing the finish line. Initial fixed-size connection pool implement…
nicksieger authored
278 end
72d959d @nicksieger Split connection handler into single- and multiple-thread versions.
nicksieger authored
279 end
6edaa26 @nicksieger Initial conversion to connection pool
nicksieger authored
280 end
21eb18a Fix race in ConnectionPool#checkout
Aliaksey Kandratsenka authored
281 end
Something went wrong with that request. Please try again.