Skip to content
This repository
Newer
Older
100644 292 lines (257 sloc) 9.919 kb
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
1 require 'monitor'
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
2 require 'set'
3
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
4 module ActiveRecord
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
5 # Raised when a connection could not be obtained within the connection
6 # acquisition timeout period.
7 class ConnectionTimeoutError < ConnectionNotEstablished
8 end
9
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
10 module ConnectionAdapters
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
11 # Connection pool base class for managing ActiveRecord database
12 # connections.
13 #
14 # Connections can be obtained and used from a connection pool in several
15 # ways:
16 #
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
17 # 1. Simply use ActiveRecord::Base.connection as with ActiveRecord 2.1 and
18 # earlier (pre-connection-pooling). Eventually, when you're done with
19 # the connection(s) and wish it to be returned to the pool, you call
20 # ActiveRecord::Base.clear_active_connections!. This will be the
21 # default behavior for ActiveRecord when used in conjunction with
22 # ActionPack's request handling cycle.
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
23 # 2. Manually check out a connection from the pool with
24 # ActiveRecord::Base.connection_pool.checkout. You are responsible for
25 # returning this connection to the pool when finished by calling
26 # ActiveRecord::Base.connection_pool.checkin(connection).
27 # 3. Use ActiveRecord::Base.connection_pool.with_connection(&block), which
28 # obtains a connection, yields it as the sole argument to the block,
29 # and returns it to the pool after the block completes.
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
30 #
31 # There are two connection-pooling-related options that you can add to
32 # your database connection configuration:
33 #
34 # * +pool+: number indicating size of connection pool (default 5)
35 # * +wait_timeout+: number of seconds to block and wait for a connection
36 # before giving up and raising a timeout error (default 5 seconds).
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
37 class ConnectionPool
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
38 delegate :verification_timeout, :to => "::ActiveRecord::Base"
39 attr_reader :spec
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
40
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
41 def initialize(spec)
42 @spec = spec
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
43 # The cache of reserved connections mapped to threads
44 @reserved_connections = {}
cab76ce6 » nicksieger
2008-04-19 Add synchronization to connection pool also
45 # The mutex used to synchronize pool access
46 @connection_mutex = Monitor.new
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
47 @queue = @connection_mutex.new_cond
48 # default 5 second timeout
49 @timeout = spec.config[:wait_timeout] || 5
50 # default max pool size to 5
51 @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
52 @connections = []
53 @checked_out = []
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
54 end
55
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
56 # Retrieve the connection associated with the current thread, or call
57 # #checkout to obtain one if necessary.
58 #
59 # #connection can be called any number of times; the connection is
60 # held in a hash keyed by the thread id.
61 def connection
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
62 if conn = @reserved_connections[current_connection_id]
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
63 conn.verify!(verification_timeout)
64 conn
65 else
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
66 @reserved_connections[current_connection_id] = checkout
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
67 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
68 end
69
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
70 # Signal that the thread is finished with the current connection.
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
71 # #release_connection releases the connection-thread association
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
72 # and returns the connection to the pool.
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
73 def release_connection
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
74 conn = @reserved_connections.delete(current_connection_id)
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
75 checkin conn if conn
76 end
77
78 # Reserve a connection, and yield it to a block. Ensure the connection is
79 # checked back in when finished.
80 def with_connection
81 conn = checkout
82 yield conn
83 ensure
84 checkin conn
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
85 end
86
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
87 # Returns true if a connection has already been opened.
88 def connected?
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
89 !@connections.empty?
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
90 end
91
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
92 # Disconnect all connections in the pool.
93 def disconnect!
94 @reserved_connections.each do |name,conn|
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
95 checkin conn
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
96 end
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
97 @reserved_connections = {}
98 @connections.each do |conn|
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
99 conn.disconnect!
100 end
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
101 @connections = []
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
102 end
103
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
104 # Clears the cache which maps classes
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
105 def clear_reloadable_connections!
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
106 @reserved_connections.each do |name, conn|
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
107 checkin conn
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
108 end
109 @reserved_connections = {}
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
110 @connections.each do |conn|
111 conn.disconnect! if conn.requires_reloading?
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
112 end
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
113 @connections = []
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
114 end
115
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
116 # Verify active connections and remove and disconnect connections
117 # associated with stale threads.
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
118 def verify_active_connections! #:nodoc:
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
119 clear_stale_cached_connections!
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
120 @connections.each do |connection|
50cd4bdc » nicksieger
2008-04-19 Introduce synchronization around connection pool access
121 connection.verify!(verification_timeout)
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
122 end
123 end
124
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
125 # Return any checked-out connections back to the pool by threads that
126 # are no longer alive.
127 def clear_stale_cached_connections!
128 remove_stale_cached_threads!(@reserved_connections) do |name, conn|
129 checkin conn
130 end
131 end
132
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
133 # Check-out a database connection from the pool.
134 def checkout
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
135 # Checkout an available connection
136 conn = @connection_mutex.synchronize do
137 if @checked_out.size < @connections.size
138 checkout_existing_connection
139 elsif @connections.size < @size
140 checkout_new_connection
141 end
142 end
143 return conn if conn
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
144
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
145 # No connections available; wait for one
146 @connection_mutex.synchronize do
147 if @queue.wait(@timeout)
148 checkout_existing_connection
149 else
150 raise ConnectionTimeoutError, "could not obtain a database connection in a timely fashion"
151 end
152 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
153 end
154
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
155 # Check-in a database connection back into the pool.
156 def checkin(conn)
157 @connection_mutex.synchronize do
158 conn.run_callbacks :checkin
159 @checked_out.delete conn
160 @queue.signal
161 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
162 end
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
163
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
164 synchronize :connection, :release_connection,
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
165 :clear_reloadable_connections!, :verify_active_connections!,
166 :connected?, :disconnect!, :with => :@connection_mutex
167
168 private
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
169 def new_connection
ca6d7175 » nicksieger
2008-08-22 Deprecate allow_concurrency and make it have no effect
170 config = spec.config.reverse_merge(:allow_concurrency => true)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
171 ActiveRecord::Base.send(spec.adapter_method, config)
172 end
173
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
174 def current_connection_id #:nodoc:
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
175 Thread.current.object_id
176 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
177
029952ed » nicksieger
2008-08-04 Extract a base class for connection pools, start to flesh out reserve…
178 # Remove stale threads from the cache.
179 def remove_stale_cached_threads!(cache, &block)
180 keys = Set.new(cache.keys)
181
182 Thread.list.each do |thread|
183 keys.delete(thread.object_id) if thread.alive?
184 end
185 keys.each do |key|
186 next unless cache.has_key?(key)
187 block.call(key, cache[key])
188 cache.delete(key)
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
189 end
190 end
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
191
192 def checkout_new_connection
193 c = new_connection
194 @connections << c
a96b7d4c » nicksieger
2008-08-22 Add connection reset and verification upon each connection checkout
195 checkout_and_verify(c)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
196 end
197
198 def checkout_existing_connection
d7d2d73d » nicksieger
2008-08-08 Fix typo: was using brackets instead of parens. Must need more sleep.
199 c = (@connections - @checked_out).first
a96b7d4c » nicksieger
2008-08-22 Add connection reset and verification upon each connection checkout
200 checkout_and_verify(c)
201 end
202
203 def checkout_and_verify(c)
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
204 c.run_callbacks :checkout
a96b7d4c » nicksieger
2008-08-22 Add connection reset and verification upon each connection checkout
205 c.verify!(verification_timeout)
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
206 @checked_out << c
207 c
208 end
209 end
210
ca6d7175 » nicksieger
2008-08-22 Deprecate allow_concurrency and make it have no effect
211 class ConnectionHandler
212 attr_reader :connection_pools_lock
213
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
214 def initialize(pools = {})
215 @connection_pools = pools
ca6d7175 » nicksieger
2008-08-22 Deprecate allow_concurrency and make it have no effect
216 @connection_pools_lock = Monitor.new
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
217 end
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
218
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
219 def connection_pools
220 @connection_pools ||= {}
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
221 end
222
223 def establish_connection(name, spec)
8e5e02bd » nicksieger
2008-08-22 Collapse connection pool class hierarchy; YAGNI.
224 @connection_pools[name] = ConnectionAdapters::ConnectionPool.new(spec)
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
225 end
226
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
227 # Returns any connections in use by the current thread back to the pool,
228 # and also returns connections to the pool cached by threads that are no
229 # longer alive.
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
230 def clear_active_connections!
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
231 @connection_pools.each_value do |pool|
232 pool.release_connection
233 pool.clear_stale_cached_connections!
234 end
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
235 end
236
237 # Clears the cache which maps classes
238 def clear_reloadable_connections!
239 @connection_pools.each_value {|pool| pool.clear_reloadable_connections! }
240 end
241
242 def clear_all_connections!
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
243 @connection_pools.each_value {|pool| pool.disconnect! }
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
244 end
245
246 # Verify active connections.
247 def verify_active_connections! #:nodoc:
817a07b4 » nicksieger
2008-08-07 More doco and class/method renames. Now have a strategy for integrati…
248 @connection_pools.each_value {|pool| pool.verify_active_connections! }
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
249 end
250
251 # Locate the connection of the nearest super class. This can be an
252 # active or defined connection: if it is the latter, it will be
253 # opened and set as the active connection for the class it was defined
254 # for (not necessarily the current class).
255 def retrieve_connection(klass) #:nodoc:
256 pool = retrieve_connection_pool(klass)
257 (pool && pool.connection) or raise ConnectionNotEstablished
258 end
259
82fcd9d8 » nicksieger
2008-08-06 Clean up the code, get rid of reserve/release, add some more docs
260 # Returns true if a connection that's accessible to this class has
261 # already been opened.
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
262 def connected?(klass)
263 retrieve_connection_pool(klass).connected?
264 end
265
266 # Remove the connection for this class. This will close the active
267 # connection and the defined connection (if they exist). The result
268 # can be used as an argument for establish_connection, for easily
269 # re-establishing the connection.
270 def remove_connection(klass)
271 pool = @connection_pools[klass.name]
272 @connection_pools.delete_if { |key, value| value == pool }
273 pool.disconnect! if pool
274 pool.spec.config if pool
275 end
276
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
277 def retrieve_connection_pool(klass)
278 loop do
279 pool = @connection_pools[klass.name]
280 return pool if pool
281 return nil if ActiveRecord::Base == klass
282 klass = klass.superclass
ff97e9d0 » nicksieger
2008-06-07 Connection handling methods extracted out into separate ConnectionHan…
283 end
fe575dd4 » nicksieger
2008-08-07 Nearing the finish line. Initial fixed-size connection pool implement…
284 end
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
285
286 # Apply monitor to all public methods that access the pool.
d07a6b1a » nicksieger
2008-08-22 Make clear_active_connections! also return stale connections back to …
287 synchronize :establish_connection, :retrieve_connection, :connected?, :remove_connection,
288 :clear_active_connections!, :clear_reloadable_connections!, :clear_all_connections!,
289 :verify_active_connections!, :with => :connection_pools_lock
72d959d9 » nicksieger
2008-06-07 Split connection handler into single- and multiple-thread versions.
290 end
6edaa267 » nicksieger
2008-04-19 Initial conversion to connection pool
291 end
292 end
Something went wrong with that request. Please try again.