Skip to content
This repository

ConnectionPool accepts spec key 'checkout_timeout' (Backport) #7684

Merged
merged 1 commit into from over 1 year ago

3 participants

Jonathan Rochkind Steve Klabnik Rafael Mendonça França
Jonathan Rochkind

Backport of applicable parts of #6441 cb6f839, in a backwards compatible way.

Previously, ConnectionPool uses connection spec key 'wait_timeout' (default 5 seconds) to decide how long a thread can block waiting for a connection to be avail before raising.

However, mysql2 adapter uses this same 'wait_timeout' key for an entirely different value -- setting the MySQL server's own 'wait timeout', how long the server will allow a connection to be idle before closing it. Default is several hours.

Extreme differences in default shows you often do not want these two values to be the same. But if you don't want them to be default, there is no supported API to change one without changing the other to be the same, since they both use 'wait_timeout'.

So this change maeks ConnectionPool use checkout_timeout in the spec, prefering it over wait_timeout. But wait_timeout IS (unlike in master) still used if there without checkout_timeout. If you want mysql2's wait_timeout to be differnet than ConnectionPool's timeout, you can (and have to) set them both -- mysql2 will use the wait_timeout, ConnectionPool will use the checkout_timeout. But prior to this patch there was no way to do that.

The actual code change is very straightforward. But I wasn't sure how to write the test. The test is a bit wonky, if someone has a suggestion of how to organize the test better, please do share. @rafaelfranca , any interest/feedback?

Steve Klabnik
Collaborator

This no longer merges cleanly, and will need a rebase.

Jonathan Rochkind

had some trouble figuring out how to get this properly rebased, but I think I've done it right. thanks @steveklabnik

Steve Klabnik
Collaborator

You have! :D

Jonathan Rochkind

so, um, what would it take to get this merged into 3-2-stable (or feedback given on why it's not suitable for such)? Anything I can do?

Steve Klabnik
Collaborator

Needs someone from Rails Core to review. I'm not allowed to use my merge privledges.

I'd bet either @tenderlove or @jonleighton would be good people here.

Rafael Mendonça França

It seems fine to me. I just want to know if @tenderlove have something against this change.

Rafael Mendonça França

We will need a CHANGELOG entry

Jonathan Rochkind
Rafael Mendonça França

I'll merge it as soon you add a CHANGELOG.

Jonathan Rochkind ConnectionPool accepts spec key 'checkout_timeout'
Backport of #6441 cb6f839 . Old 'wait_timeout' is still supported,
but conflicts with mysql2 using that spec key for different thing.
'checkout_timeout' can now be used taking precedence for ConnectionPool
over 'wait_timeout'.
3908706
Jonathan Rochkind

done. and yeah,I can rebase again if I need to just let me know.

Rafael Mendonça França rafaelfranca merged commit 8800aae into from September 25, 2012
Rafael Mendonça França rafaelfranca closed this September 25, 2012
Rafael Mendonça França

Thanks

Jens Braeuer jbraeuer referenced this pull request in aws/opsworks-cookbooks February 02, 2014
Closed

Include pool size in database.yml.erb #99

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 1 unique commit by 1 author.

Sep 24, 2012
Jonathan Rochkind ConnectionPool accepts spec key 'checkout_timeout'
Backport of #6441 cb6f839 . Old 'wait_timeout' is still supported,
but conflicts with mysql2 using that spec key for different thing.
'checkout_timeout' can now be used taking precedence for ConnectionPool
over 'wait_timeout'.
3908706
This page is out of date. Refresh to see the latest.
6  activerecord/CHANGELOG.md
Source Rendered
... ...
@@ -1,5 +1,11 @@
1 1
 ## Rails 3.2.9 (unreleased)
2 2
 
  3
+*   ConnectionPool recognizes checkout_timeout spec key as taking
  4
+    precedence over legacy wait_timeout spec key, can be used to avoid
  5
+    conflict with mysql2 use of wait_timeout.  Closes #7684.
  6
+
  7
+    *jrochkind*
  8
+
3 9
 *   Rename field_changed? to _field_changed? so that users can create a field named field
4 10
 
5 11
     *Akira Matsuda*, backported by *Steve Klabnik*
12  activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
@@ -54,8 +54,11 @@ module ConnectionAdapters
54 54
     # your database connection configuration:
55 55
     #
56 56
     # * +pool+: number indicating size of connection pool (default 5)
57  
-    # * +wait_timeout+: number of seconds to block and wait for a connection
58  
-    #   before giving up and raising a timeout error (default 5 seconds).
  57
+    # * +checkout _timeout+: number of seconds to block and wait for a 
  58
+    #   connection before giving up and raising a timeout error 
  59
+    #   (default 5 seconds). ('wait_timeout' supported for backwards
  60
+    #   compatibility, but conflicts with key used for different purpose
  61
+    #   by mysql2 adapter). 
59 62
     class ConnectionPool
60 63
       include MonitorMixin
61 64
 
@@ -77,7 +80,10 @@ def initialize(spec)
77 80
         @reserved_connections = {}
78 81
 
79 82
         @queue = new_cond
80  
-        @timeout = spec.config[:wait_timeout] || 5
  83
+        # 'wait_timeout', the backward-compatible key, conflicts with spec key 
  84
+        # used by mysql2 for something entirely different, checkout_timeout
  85
+        # preferred to avoid conflict and allow independent values. 
  86
+        @timeout = spec.config[:checkout_timeout] || spec.config[:wait_timeout] || 5
81 87
 
82 88
         # default max pool size to 5
83 89
         @size = (spec.config[:pool] && spec.config[:pool].to_i) || 5
22  activerecord/test/cases/connection_pool_test.rb
@@ -151,6 +151,28 @@ def test_automatic_reconnect=
151 151
       def test_pool_sets_connection_visitor
152 152
         assert @pool.connection.visitor.is_a?(Arel::Visitors::ToSql)
153 153
       end
  154
+      
  155
+      def test_timeout_spec_keys
  156
+        # 'wait_timeout' is supported for backwards compat, 
  157
+        # 'checkout_timeout' is preferred to avoid conflicting
  158
+        #  with mysql2 adapters key of name 'wait_timeout' but
  159
+        #  different meaning.
  160
+        config = ActiveRecord::Base.connection_pool.spec.config.merge(:wait_timeout => nil, :connection_timeout => nil)
  161
+        method = ActiveRecord::Base.connection_pool.spec.adapter_method        
  162
+        
  163
+        pool = ConnectionPool.new ActiveRecord::Base::ConnectionSpecification.new(config.merge(:wait_timeout => 1), method)
  164
+        assert_equal 1, pool.instance_variable_get(:@timeout)
  165
+        pool.disconnect!
  166
+        
  167
+        pool = ConnectionPool.new ActiveRecord::Base::ConnectionSpecification.new(config.merge(:checkout_timeout => 1), method)
  168
+        assert_equal 1, pool.instance_variable_get(:@timeout)
  169
+        pool.disconnect!
  170
+        
  171
+        pool = ConnectionPool.new ActiveRecord::Base::ConnectionSpecification.new(config.merge(:wait_timeout => 6000, :checkout_timeout => 1), method)
  172
+        assert_equal 1, pool.instance_variable_get(:@timeout)
  173
+        pool.disconnect!
  174
+      end
  175
+      
154 176
     end
155 177
   end
156 178
 end
4  activerecord/test/cases/pooled_connections_test.rb
@@ -17,7 +17,7 @@ def teardown
17 17
   end
18 18
 
19 19
   def checkout_connections
20  
-    ActiveRecord::Base.establish_connection(@connection.merge({:pool => 2, :wait_timeout => 0.3}))
  20
+    ActiveRecord::Base.establish_connection(@connection.merge({:pool => 2, :checkout_timeout => 0.3}))
21 21
     @connections = []
22 22
     @timed_out = 0
23 23
 
@@ -42,7 +42,7 @@ def test_pooled_connection_checkout
42 42
   end
43 43
 
44 44
   def checkout_checkin_connections(pool_size, threads)
45  
-    ActiveRecord::Base.establish_connection(@connection.merge({:pool => pool_size, :wait_timeout => 0.5}))
  45
+    ActiveRecord::Base.establish_connection(@connection.merge({:pool => pool_size, :checkout_timeout => 0.5}))
46 46
     @connection_count = 0
47 47
     @timed_out = 0
48 48
     threads.times do
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.