Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

QueryCache/ConnectionPool thread fix #1670

Closed
wants to merge 6 commits into from

6 participants

@mjtko

Under thin, QueryCache is leaking connections.

This is caused by the fact that the thread that runs the QueryCache::BodyProxy#close method is not the same as the thread that executes QueryCache#call. A connection is checked out within QueryCache#call; this connection becomes orphaned, while a new connection is checked out within #close when theclear_query_cache call is made.

This patch prevents the connection checked out within QueryCache#call from being orphaned by creating an identifier that is stored within a thread local and used within ConnectionPool to uniquely identify the connection. This identifier is passed to BodyProxy at construction time and pushed into the thread that ultimately becomes responsible for the clear_query_cache call.

The connection is subsequently checked back in as expected at the end of the request lifecycle, plugging the leak.

@jhollinger

Thanks for patching this. Hopefully someday this will be merged into 3.1.something and I can run multi-threaded apps under Thin again. It's killing me.

@spastorino
Owner

Hey @mjtko can you rebase your patch?

@mjtko
@jhollinger

I don't know, I'm sure there's worse code in Rails :-)

I suppose the question is "has Thin been doing things wrong all along, and it just now became obvious?", in which case Thin should be patched, or "has Rails 3.1 accidentally broken a sane expectation held by Thin?", in which case Rails should be patched. Honestly I don't have enough knowledge of either code base to hazard a guess.

@mjtko

@spastorino I'll rebase it tomorrow (Europe), though, personally, I think this should wait until after 3.1.1 so, even though I'll rebase it against 3-1-stable, I'd wait until after 3.1.1 is out of the door before pulling. Up to you to decide of course! :)

@jhollinger This has always been the case when running Rails in threadsafe mode under Thin (AFAICS) and, while from my POV Thin is doing the right thing, Rails has been making an assumption about the threading model within a rack server - I would expect Rainbows! with the event machine concurrency model to suffer the same problem.

In short, my belief is that active record should be patched to support this use case.

@spastorino
Owner

@mjtko don't worry this is not going in 3.1.1. Btw please rebase it on top of master, we will merge in 3-1-stable. Thanks.

mjtko added some commits
@mjtko mjtko use thread locals and an instance variable within QueryCache#BodyProx…
…y to maintain appropriate linkage with AR database connection across threads
648eb48
@mjtko mjtko Merge branch 'connection-pool-thread-fixes' of github.com:mjtko/rails…
… into connection-pool-thread-fixes

Conflicts:
	activerecord/lib/active_record/query_cache.rb
4d530c3
@mjtko

Argh, I've totally messed this branch up trying to rebase it! Rather than trying to unpick the mess I'll open a new pull request against master. :)

@mjtko mjtko closed this
@mjtko

@spastorino Opened against master as #3243. Sorry for the confusion. :-)

@artemave

+1

very annoying issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 13, 2011
  1. @mjtko

    use thread locals and an instance variable within QueryCache#BodyProx…

    mjtko authored
    …y to maintain appropriate linkage with AR database connection across threads
Commits on Oct 6, 2011
  1. @fxn

    revises the precompilation section of the asset pipeline guide (web s…

    fxn authored
    …erver configuration for gzipped assets pending)
  2. @fxn
  3. @vijaydev
Commits on Oct 7, 2011
  1. @mjtko

    use thread locals and an instance variable within QueryCache#BodyProx…

    mjtko authored
    …y to maintain appropriate linkage with AR database connection across threads
  2. @mjtko

    Merge branch 'connection-pool-thread-fixes' of github.com:mjtko/rails…

    mjtko authored
    … into connection-pool-thread-fixes
    
    Conflicts:
    	activerecord/lib/active_record/query_cache.rb
This page is out of date. Refresh to see the latest.
View
2  activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
@@ -314,7 +314,7 @@ def new_connection
end
def current_connection_id #:nodoc:
- Thread.current.object_id
+ ActiveRecord::Base.connection_id ||= Thread.current.object_id
end
def checkout_new_connection
View
8 activerecord/lib/active_record/connection_adapters/abstract/connection_specification.rb
@@ -89,6 +89,14 @@ def connection
retrieve_connection
end
+ def connection_id
+ Thread.current['ActiveRecord::Base.connection_id']
+ end
+
+ def connection_id=(connection_id)
+ Thread.current['ActiveRecord::Base.connection_id'] = connection_id
+ end
+
# Returns the configuration of the associated connection as a hash:
#
# ActiveRecord::Base.connection_config
View
6 activerecord/lib/active_record/query_cache.rb
@@ -28,9 +28,10 @@ def initialize(app)
end
class BodyProxy # :nodoc:
- def initialize(original_cache_value, target)
+ def initialize(original_cache_value, target, connection_id)
@original_cache_value = original_cache_value
@target = target
+ @connection_id = connection_id
end
def method_missing(method_sym, *arguments, &block)
@@ -48,6 +49,7 @@ def each(&block)
def close
@target.close if @target.respond_to?(:close)
ensure
+ ActiveRecord::Base.connection_id = @connection_id
ActiveRecord::Base.connection.clear_query_cache
unless @original_cache_value
ActiveRecord::Base.connection.disable_query_cache!
@@ -60,7 +62,7 @@ def call(env)
ActiveRecord::Base.connection.enable_query_cache!
status, headers, body = @app.call(env)
- [status, headers, BodyProxy.new(old, body)]
+ [status, headers, BodyProxy.new(old, body, ActiveRecord::Base.connection_id)]
rescue Exception => e
ActiveRecord::Base.connection.clear_query_cache
unless old
View
6 railties/guides/source/asset_pipeline.textile
@@ -398,7 +398,7 @@ This can be changed with the +config.assets.manifest+ option. A fully specified
config.assets.manifest = '/path/to/some/other/location'
</erb>
-NOTE: If there are missing precompiled files in production you will get an <tt>AssetNoPrecompiledError</tt> exception indicating the name of the missing file(s).
+NOTE: If there are missing precompiled files in production you will get an <tt>Sprockets::Helpers::RailsHelper::AssetPaths::AssetNotPrecompiledError</tt> exception indicating the name of the missing file(s).
h5. Server Configuration
@@ -436,9 +436,9 @@ location ~ ^/assets/ {
}
</plain>
-When files are precompiled, Sprockets also creates a "Gzip":http://en.wikipedia.org/wiki/Gzip (.gz) version of your assets. This avoids the server having to do this for any requests; it can simply read the compressed files from disk. You must configure your server to use gzip compression and serve the compressed assets that will be stored in the +public/assets+ folder. The following configuration options can be used:
+When files are precompiled, Sprockets also creates a "gzipped":http://en.wikipedia.org/wiki/Gzip (.gz) version of your assets. Web servers are typically configured to use a moderate compression ratio as a compromise, but since precompilation happens once Sprockets uses the maximum compression ratio, thus reducing the size of the data transfer to the minimum. One the other hand, web servers can be configured to serve compressed content directly from disk, rather than deflating non-compressed files themselves.
-For Apache:
+A possible configuration for Apache could be:
<plain>
<LocationMatch "^/assets/.*$">
Something went wrong with that request. Please try again.