Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prepared statements support for `Mysql2Adapter` #23461

Merged
merged 1 commit into from Apr 24, 2016

Conversation

@kamipo
Copy link
Member

kamipo commented Feb 3, 2016

Reopen #22415.

Currently mysql2 adapter does not support prepared statements. And mysql adapter has been removed. This means that Active Record does not support prepared statements for MySQL now.

We need to support prepared statements for MySQL. But mysql2 gem still have GC issue in prepared statements. brianmario/mysql2#694

I made default to @prepared_statements = false in Mysql2Adapter for does not affect default behavior. I think it is still safety.

@rails-bot
Copy link

rails-bot commented Feb 3, 2016

r? @chancancode

(@rails-bot has picked a reviewer for you, use r? to override)

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch 2 times, most recently Feb 3, 2016
@sgrif
sgrif reviewed Feb 4, 2016
View changes
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb Outdated
@@ -126,6 +141,10 @@ def supports_datetime_with_precision?
version >= '5.6.4'
end

def supports_json?

This comment has been minimized.

@sgrif

sgrif Feb 4, 2016 Contributor

This seems unrelated to prepared statement support

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch 2 times, most recently Feb 4, 2016
@kamipo
Copy link
Member Author

kamipo commented Feb 4, 2016

Extracted unrelated change to #23469, thanks!
r? @sgrif

@rails-bot rails-bot assigned sgrif and unassigned chancancode Feb 4, 2016
@sgrif
Copy link
Contributor

sgrif commented Feb 5, 2016

This will need to handle nodes which we consider to be unpreparable, similar to what we do in PG and SQLite. We can probably just move that logic up to the abstract adapter now.

@kamipo
Copy link
Member Author

kamipo commented Feb 6, 2016

Do you means #23515?

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch 3 times, most recently Feb 8, 2016
@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch 2 times, most recently Feb 17, 2016
@sodabrew
Copy link
Contributor

sodabrew commented Feb 24, 2016

I've just pushed mysql2 0.4.3 with important prepared statement fixes from @kamipo!

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch 3 times, most recently Feb 28, 2016
@jeremy
Copy link
Member

jeremy commented Apr 19, 2016

Rebase time 😊

@kamipo
kamipo reviewed Apr 19, 2016
View changes
activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb Outdated
end
else
exec_stmt_and_free(sql, name, binds, cache_stmt: prepare) do |_, result|
ActiveRecord::Result.new(result.fields, result.to_a) if result

This comment has been minimized.

@kamipo

kamipo Apr 19, 2016 Author Member

In mysql2 0.4.3, result.fields causes segv when num of rows is zero.
This issue was fixed on master by brianmario/mysql2#741.
@sodabrew Could you release mysql2 0.4.4?

This comment has been minimized.

@sodabrew

sodabrew Apr 19, 2016 Contributor

Done! mysql2 0.4.4 is now on Rubygems.org.

@sodabrew
Copy link
Contributor

sodabrew commented Apr 19, 2016

I've just posted mysql2 0.4.4 with additional prepared statement fixes from @kamipo!

@kamipo
Copy link
Member Author

kamipo commented Apr 19, 2016

Thanks @sodabrew !! 😍

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch Apr 19, 2016
@kamipo
Copy link
Member Author

kamipo commented Apr 19, 2016

Rebased on master!

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch Apr 19, 2016
@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch Apr 20, 2016
@jeremy jeremy added this to the 5.1.0 milestone Apr 20, 2016
def initialize(connection, logger, connection_options, config)
super
@prepared_statements = false
@prepared_statements = false unless config.key?(:prepared_statements)

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

Should we use the value of the prepared_statements config here like we do in the other adapters?

This comment has been minimized.

@kamipo

kamipo Apr 20, 2016 Author Member

This is for prepared_statements disabled by default (keep previous default).
Should we make to enabled by default?

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

We should respect it. If users ask to enable it, it should be enabled, to disable it, it should be disabled. This implementation while keeping the default is enabling if you do prepared_statements: false in your config.

This comment has been minimized.

@kamipo

kamipo Apr 20, 2016 Author Member

This implementation is:

if config.key?(:prepared_statements)
  # @prepared_statements is configured in `super`
  super
else
  # keep the previous default (disabled) when `:prepared_statements` is not specified
  @prepared_statements = false
end

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

Oh. I missed that change that moved AbstractAdapter to configure prepared_statements 👍

# as values.
def select_one(arel, name = nil, binds = [])
arel, binds = binds_from_relation(arel, binds)
@connection.query_options.merge!(as: :hash)

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

What this does?

This comment has been minimized.

@kamipo

kamipo Apr 20, 2016 Author Member

stmt.execute calls result.each in the internal.
https://github.com/brianmario/mysql2/blob/0.4.4/ext/mysql2/statement.c#L384
Need to set query_options before stmt.execute.

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

But how returning a hash instead of array in the result would be affected by the result.each call?

This comment has been minimized.

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

I see now. Thanks for the explanation!

@jeremy
Copy link
Member

jeremy commented Apr 20, 2016

Changing to 5.0 since prepared statements are disabled by default.

@jeremy jeremy modified the milestones: 5.0.0, 5.1.0 Apr 20, 2016
@rafaelfranca
rafaelfranca reviewed Apr 20, 2016
View changes
activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb Outdated
# Executes the SQL statement in the context of this connection.
def execute(sql, name = nil)
log(sql, name) { @connection.query(sql) }
end

# Mysql2Adapter doesn't have to free a result after using it, but we use this method
# to write stuff in an abstract way without concerning ourselves about whether it
# needs to be explicitly freed or not.
def execute_and_free(sql, name = nil) # :nodoc:

This comment has been minimized.

@rafaelfranca

rafaelfranca Apr 20, 2016 Member

We can remove this method here in a different PR. This abstract class can be removed too since we only have one concrete class.

@kamipo kamipo force-pushed the kamipo:prepared_statements_for_mysql2_adapter branch to 3f6574e Apr 21, 2016

gem 'mysql2', '>= 0.3.18', '< 0.5'
gem 'mysql2', '~> 0.4.4'

This comment has been minimized.

@jeremy

jeremy Apr 24, 2016 Member

Do we need to bump mysql2 minimum version? 0.4.x is only required if prepared_statements: true.

This comment has been minimized.

@kamipo

kamipo Apr 24, 2016 Author Member

Unfortunately, mysql2 0.4.3 was broken without prepared statements.
I would like to skip this version.

@jeremy jeremy merged commit 3f6574e into rails:master Apr 24, 2016
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jeremy added a commit that referenced this pull request Apr 24, 2016
…adapter

Add prepared statements support for `Mysql2Adapter`
@kamipo kamipo deleted the kamipo:prepared_statements_for_mysql2_adapter branch Apr 24, 2016
vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 24, 2016
kaspth added a commit that referenced this pull request Apr 24, 2016
Add #23461 to release notes
@sodabrew
Copy link
Contributor

sodabrew commented Apr 24, 2016

Hurray!

vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 24, 2016
- Rename max to statement_limit
- Remove magic number 1000 from everywhere
- Defined StatementPool::DEFAULT_STATEMENT_LIMIT and started using it everywhere
jeremy added a commit that referenced this pull request Apr 24, 2016
- Rename max to statement_limit
- Remove magic number 1000 from everywhere
- Defined StatementPool::DEFAULT_STATEMENT_LIMIT and started using it everywhere

Signed-off-by: Jeremy Daer <jeremydaer@gmail.com>
kamipo added a commit that referenced this pull request Mar 5, 2019
Since #23461, all adapters supports prepared statements, so that clears
the prepared statements cache is no longer database specific.

Actually, I struggled to identify the cause of random CI failure in
#23461, that was missing `@statements.clear` in `clear_cache!`.

This extracts `clear_cache!` to ensure the common concerns in the
abstract adapter.
kamipo added a commit to kamipo/rails that referenced this pull request Jul 17, 2020
Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

* Making boundable queries

While working on rails#23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

rails#26117
7d53993
rails#39219

Now, `where` with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
`predicate_builder` directly.

kufu/activerecord-bitemporal#62

* Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using `User.arel_table[:name]` is not preferable for
`unscope` and `merge`).

Example:

```ruby
id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)
```

* Automatic other table references detection

It works only for the hash syntax.

ee7f666
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants
You can’t perform that action at this time.