Mysql2::Error: This connection is still waiting for a result, try again once you have the result #99
Comments
|
I'm also having this problem. |
|
This also happens when using Timeout: |
|
Same here, I had to press ctrl-c as things were becoming way too slow (we were doing some load testing). This is a reliability issue, as sooner or later things can get interrupted under load, etc. An app that needs to be available reliably cannot afford to have something like this lurking. Let us know how we can help fix this. The repro above with Timeout looks like a great start! I am on RoR 3.0.3 with rack 1.2.1, ruby ruby-1.8.7.302-2.fc14.i686 and mysql2 0.2.6 |
|
I have a similar problem. After this error occurred all following queries fail with "Exception ActiveRecord::StatementInvalid -> Mysql2::Error: This connection is still waiting for a result, try again once you have the result" RoR 3.0.3 in a Stalker background process, ruby 1.9.2; mysql2 0.2.6 |
|
I've run into this problem in production. I'm using Timeout, so a query could have been interrupted. Rails 3.0.3, Ruby 1.8.7, mysql2 0.2.6 |
|
Using the Mysql2::EM::Client.new class and running into this too. Ruby 1.9.2, Rails 2.3.9, EventMachine 0.12.10. |
|
Same. Rails3/1.9.2. |
|
Same time. Rails 3 (threadsafe), 1.9.2 |
|
Same. Rails3.0.6/1.9.2. |
|
Same here ; Rails 3.0.6, Ruby 1.8.7 |
|
When used in Rails with ActiveRecord, it's possible to recover from this exception. # This is just to abort a query
Timeout.timeout(1) { MyModel.connection.execute("select sleep(5);"); }
# This is the recovering hack
begin
# Any query on a model
MyModel.first
rescue ActiveRecord::StatementInvalid => ex
# Since all Mysql2 errors are wrapped into ActiveRecord::StatementInvalid
# we have to retry only for this kind of error
if ex.message["Mysql2::Error: This connection is still waiting for a result"]
@retries ||= 0
# Retry only 3 times
puts @retries
if @retries < 3
@retries += 1
MyModel.connection.reconnect!
retry
else
raise
end
else
raise
end
else
@retries = 0
end |
|
Here's a monkey-patch to auto-reconnect when this occurs: module ActiveRecord
module ConnectionAdapters
class Mysql2Adapter
def query_with_reconnect(sql)
@connection.query(sql)
rescue Mysql2::Error => exception
if exception.message.starts_with? "This connection is still waiting for a result"
reconnect!
retry
end
raise
end
# Executes the SQL statement in the context of this connection.
def execute(sql, name = nil)
# make sure we carry over any changes to ActiveRecord::Base.default_timezone that have been
# made since we established the connection
@connection.query_options[:database_timezone] = ActiveRecord::Base.default_timezone
if name == :skip_logging
query_with_reconnect(sql)
else
log(sql, name) { query_with_reconnect(sql) }
end
rescue ActiveRecord::StatementInvalid => exception
if exception.message.split(":").first =~ /Packets out of order/
raise ActiveRecord::StatementInvalid, "'Packets out of order' error was received from the database. Please update your mysql bindings (gem install mysql) and read http://dev.mysql.com/doc/mysql/en/password-hashing.html for more information. If you're on Windows, use the Instant Rails installer to get the updated mysql bindings."
else
raise
end
end
end
end
end |
|
Please have a look at #168. Does this solve the issue you guys are facing? Looking for your feedback. |
|
@ramsingla: I think you've addressed a different issue with the same error message. I'm using mysql2_adapter.rb not em_mysql2_adapter.rb so any EM related solution doesn't apply. |
|
This is also an issue if you set read_timeout in database.yml, as the connection is closed but still waiting for a result. This is with the non-EM client. |
|
With mysql2 0.2.7, I'm also running into this problem today when upgrade to ruby 1.9.2. Never encountered before in ruby 1.8.7 |
|
the hotfix made by @ghazel fixed the problem for us. Thanks and hope this bug gets solved .. we run into that error with lots of data in a table (sessions), it just popped up after some time without a problem. |
|
I suspect people using Unicorn do not notice this as they kill the entire worker process which causes a new one to be spawned. The new worker reconnects when starting up. We use a timeout in the application to kill long running requests, and it causes queries to die in this state. |
|
I'm thinking rb_mysql_client_query should be wrapped in an rb_rescue and the callback should reset the connection. |
|
I just pushed a fix for this in the 0.2.x and master branches, would you guys mind giving it a try? I'll push another release if it fixes it for you guys. |
|
This causes another test to fail. I've amended my test to use :reconnect => true, you can pull it in as a test for this. |
|
Ah good catch, lemme get a fix pushed for that |
|
Sent a pull request for each branch, you can just pull one and merge/cherry-pick. |
|
Ok just pushed another patch, try again? |
|
Yay:
|
|
@ghzel - summary of the changes to fix this issue: The problem: if a signal interrupted the The fix: Wrap the This should fix the issue, but I'd like to hear back from at least another person that it has before closing the issue. |
|
I will ship this into production on 3 well trafficked applications. |
|
This problem is alive and well on mysql2 master unfortunately: |
|
putting AfterStep("@with_mysql2_failures") do
sleep 1
end |
|
@rilian, no kidding. |
|
new versions pushed, would you guys mind giving them a try? Make sure you've removed all other versions of mysql2 from your system first - just so we're certain you're using the new version |
|
@brianmario, worked for me! Cool. Maybe you've just decreased the probability of the bug occurrence, but I'll give it a try. |
|
@brianmario still no luck, using AfterStep workaround mentioned by @rilian does the trick for me for "Mysql2::Error: This connection is still waiting for a result" |
|
Also for me still no luck with the latest version... The error only occurs for longer scenarios though. |
|
Unable to run my capybara request specs in rails app because of this bug... It happens whenever I create some test which invokes 'submit' action in some form element on page - this causes a db save action to be performed, and then everything just hangs... |
|
Just so this doesn't get forgotten ... .it still happens in my test suite. Particularly when switching to capybara-webkit, given the tests run a lot faster in that setting. |
|
Adding my 2c here, because i'm experiencing this issue when running my test suites.
Tested every kind of workaround but still no luck... |
Only tests in my app. |
|
I'm seeing this in production. This is not a rails app, I'm using active record and rufu-scheduler to schedule jobs and put them on a beanstalk queue for other workers. So I have threads a plenty. Interestingly I've only really noticed it since adding ar-octopus into the mix. But I suppose that could be causing other threads to die. Just upgrading from 0.3.10 to 0.3.11 now to see if that makes a difference |
|
Just wanted to highlight why most people are having issues with capybara-webkit and on Javascript tests only. If you look at the capybara readme there's a suggestion to have ActiveRecord use the same thread and connection. The reason for this is so you can share the same data in your database across multiple tests, Cucumber patches this in a similar fashion too. Still working on a solution but hoping this will help to debug if it wasn't obvious at first. class ActiveRecord::Base
mattr_accessor :shared_connection
@@shared_connection = nil
def self.connection
@@shared_connection || retrieve_connection
end
end
ActiveRecord::Base.shared_connection = ActiveRecord::Base.connection
|
|
Please see my explanation of the problem and the solution I came up with: https://gist.github.com/3049152 Could someone else confirm the fix works for them? |
|
I got following error using you code https://gist.github.com/3051701, maybe I just set it wrong. I put you after.rb code in features/support/env.rb which is loaded before any test being run. I thought it would monkey patch AR. Using Active record 3.2.6 |
|
@rogercampos thx for the hint, I put it in config/environments/test.rb config block but I got ActiveRecord::ConnectionNotEstablished (ActiveRecord::ConnectionNotEstablished) error. Where did you put this after.rb code excatly? I also tried to put it outside of config block, but same result. I tried to put it into initializer where AR should be already initialized but then I receieved well known Mysql2::Error |
|
Same as for ares here... where exactly should we put that code? |
|
I get this same issue with a web scrapping script that I wrote that use threads. So in my case it is unrelated to testing or activerecord. When the exception happens, I am handling it by waiting 1 sec and trying again. So far it works fine this way. |
|
@mperham should this be a PR against rails then? |
|
@ares, @vitabotta, I'm using RSpec and put the code into my spec_helper.rb file in the RSpec config block. I added the connection_pool gem to my test group. This seems to have fixed my problems in my capybara/selenium specs. They seem to be related to me using delayed_job with Delayed::Worker.delay_jobs = false when in test mode, causing the delayed code to apparently be executed in parallel in another thread. My guess is that this trips up mysql2 with the above error. The code from @mperham (thank you!) appears to fix it by controlling access to the DB connections. So far so good but I need to test this some more. |
|
Closing this. The linked issues and various merged fixes to connection pool implementations resolve this issue. |
and others
Easy to reproduce. From a console, run a query and Ctrl-C it while it's running. Then try to submit a second query:
The client object remains broken forever - long after the initial query would have returned.
The text was updated successfully, but these errors were encountered: