Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random failures using prepared statements #956

Closed
catphish opened this issue Apr 3, 2018 · 4 comments · Fixed by #958
Closed

Random failures using prepared statements #956

catphish opened this issue Apr 3, 2018 · 4 comments · Fixed by #958
Milestone

Comments

@catphish
Copy link

catphish commented Apr 3, 2018

While investigating some random errors in my MySQL connections, I have discovered that using prepared statements as described in the mysql2 documentation will almost invariably result in random errors, most notably Commands out of sync.

Example code:

gem 'mysql2', '0.5.0'
require 'mysql2'

mysql_spec = {
  host: ENV['HOSTNAME'],
  username: ENV['USERNAME'],
  password: ENV['PASSWORD'],
  database: ENV['DATABASE']
}

connection_1 =  Mysql2::Client.new(mysql_spec)

1000.times do |n|
  statement_1 = connection_1.prepare("SELECT * FROM test WHERE test_id = ?")
  result_1    = statement_1.execute(1).to_a
end

The above code has been tested across various MySQL / Maria installations using both mysql 0.4.10 and 0.5.0, and usually fails before completing 1000 iterations:

/home/charlie/.rvm/gems/ruby-2.4.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:7:in `_execute': Commands out of sync; you can't run this command now (Mysql2::Error)
	from /home/charlie/.rvm/gems/ruby-2.4.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:7:in `block in execute'
	from /home/charlie/.rvm/gems/ruby-2.4.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:6:in `handle_interrupt'
	from /home/charlie/.rvm/gems/ruby-2.4.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:6:in `execute'
	from mysqltest2.rb:15:in `block in <main>'
	from mysqltest2.rb:13:in `times'
	from mysqltest2.rb:13:in `<main>'

By trial and error, I discovered 2 ways to resolve this. Firstly the problem doesn't occur when GC is disabled.

GC.disable

Alternatively, this can be resolved by changing the code to explicitly close the statement after each execute has completed.

1000.times do |n|
  statement_1 = connection_1.prepare("SELECT * FROM test WHERE test_id = ?")
  result_1    = statement_1.execute(1).to_a
  statement_1.close
end

I would appreciate any advice on whether this is a bug, or whether I am simply using the API incorrectly. If it is the latter, could this be a documentation problem? Thanks!

@kamipo
Copy link
Contributor

kamipo commented Apr 3, 2018

This is a same issue with #694 which was fixed by #705 in 0.4.4.

@adamcooke
Copy link

adamcooke commented Apr 3, 2018

This is reproducible using 0.5.0 so perhaps a regression has appeared somewhere?

/Users/adam/.rbenv/versions/2.3.7/lib/ruby/gems/2.3.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:7:in `_execute': Commands out of sync; you can't run this command now (Mysql2::Error)
	from /Users/adam/.rbenv/versions/2.3.7/lib/ruby/gems/2.3.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:7:in `block in execute'
	from /Users/adam/.rbenv/versions/2.3.7/lib/ruby/gems/2.3.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:6:in `handle_interrupt'
	from /Users/adam/.rbenv/versions/2.3.7/lib/ruby/gems/2.3.0/gems/mysql2-0.5.0/lib/mysql2/statement.rb:6:in `execute'
	from mysql.rb:14:in `block in <main>'

Can't reproduce on 0.4.3 or 0.4.4 but can on 0.4.2.

@kamipo
Copy link
Contributor

kamipo commented Apr 3, 2018

ah... will investigate this tomorrow.

@sodabrew
Copy link
Collaborator

sodabrew commented Apr 3, 2018

Thank you! Yes, sounds like another GC race :(

kamipo added a commit to kamipo/mysql2 that referenced this issue Apr 4, 2018
This fixes a regression caused by brianmario#912 due to calling `rb_funcall`
between `mysql_stmt_execute` and `mysql_stmt_store_result`, it will
cause `mysql_stmt_close` to be called in wrong order.

Fixes brianmario#956.
sodabrew added a commit to sodabrew/mysql2 that referenced this issue Apr 4, 2018
Thanks to @kamipo for diagnosing the problem and drafting the first PR.

This fixes a regression caused by brianmario#912 due to calling `rb_funcall`
between `mysql_stmt_execute` and `mysql_stmt_store_result`, it will
cause `mysql_stmt_close` to be called in wrong order.

In further testing, `rb_hash_aref` can also call `rb_funcall` if the
query_options hash has a `default_proc` set, so adding a stronger
comment to remind future maintainers.

Fixes brianmario#956.
sodabrew added a commit to sodabrew/mysql2 that referenced this issue Apr 4, 2018
Statement execute must return a cursor because Ruby is highly garbage
collected. If the connection is in results-streaming-mode for Statement A,
and in the middle Statement B gets garbage collected, a message will be
sent to the server notifying it to release Statement B, resulting in one
of these errors:
  Commands out of sync; you can't run this command now
  Row retrieval was canceled by mysql_stmt_close

By telling the server to give us a cursor to the result set, other
commands can be sent on the wire during results streaming.

Fixes brianmario#956
sodabrew added a commit to sodabrew/mysql2 that referenced this issue Apr 7, 2018
This commit fixes two error scenarios. The first is to avoid GC runs between
`mysql_stmt_execute` and `mysql_stmt_store_result`. This fixes a regression
caused by brianmario#912 due to calling `rb_funcall` between `mysql_stmt_execute` and
`mysql_stmt_store_result`. The error in this case is:
  Commands out of sync; you can't run this command now

Thanks to @kamipo for diagnosing the problem and drafting the first PR.

The second problem is that in streaming mode, rows are returned to Ruby space
one at a time, so garbage collection will naturally occur at any time. By
requesting a cursor into the result set, other MySQL commands can be sent on
the wire between row fetches. The error in this case is:
  Row retrieval was canceled by mysql_stmt_close

Fixes brianmario#956, updates brianmario#957.
sodabrew added a commit to sodabrew/mysql2 that referenced this issue Apr 7, 2018
This commit fixes two error scenarios. The first is to avoid GC runs between
`mysql_stmt_execute` and `mysql_stmt_store_result`. This fixes a regression
caused by brianmario#912 due to calling `rb_funcall` between `mysql_stmt_execute` and
`mysql_stmt_store_result`. The error in this case is:
  Commands out of sync; you can't run this command now

Thanks to @kamipo for diagnosing the problem and drafting the first PR.

The second problem is that in streaming mode, rows are returned to Ruby space
one at a time, so garbage collection will naturally occur at any time. By
requesting a cursor into the result set, other MySQL commands can be sent on
the wire between row fetches. The error in this case is:
  Row retrieval was canceled by mysql_stmt_close

Fixes brianmario#956, updates brianmario#957.
sodabrew added a commit to sodabrew/mysql2 that referenced this issue Apr 7, 2018
This commit fixes two error scenarios. The first is to avoid GC runs between
`mysql_stmt_execute` and `mysql_stmt_store_result`. This fixes a regression
caused by brianmario#912 due to calling `rb_funcall` between `mysql_stmt_execute` and
`mysql_stmt_store_result`. The error in this case is:
  Commands out of sync; you can't run this command now

Thanks to @kamipo for diagnosing the problem and drafting the first PR.

The second problem is that in streaming mode, rows are returned to Ruby space
one at a time, so garbage collection will naturally occur at any time. By
requesting a cursor into the result set, other MySQL commands can be sent on
the wire between row fetches. The error in this case is:
  Row retrieval was canceled by mysql_stmt_close

Fixes brianmario#956, updates brianmario#957.
sodabrew added a commit that referenced this issue Apr 7, 2018
This commit fixes two error scenarios. The first is to avoid GC runs between
`mysql_stmt_execute` and `mysql_stmt_store_result`. This fixes a regression
caused by #912 due to calling `rb_funcall` between `mysql_stmt_execute` and
`mysql_stmt_store_result`. The error in this case is:
  Commands out of sync; you can't run this command now

Thanks to @kamipo for diagnosing the problem and drafting the first PR.

The second problem is that in streaming mode, rows are returned to Ruby space
one at a time, so garbage collection will naturally occur at any time. By
requesting a cursor into the result set, other MySQL commands can be sent on
the wire between row fetches. The error in this case is:
  Row retrieval was canceled by mysql_stmt_close

Fixes #956, updates #957.
@sodabrew sodabrew added this to the 0.5.1 milestone Apr 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants