Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prepared statements support for Mysql2Adapter #23461

Merged
merged 1 commit into from Apr 24, 2016

Conversation

kamipo
Copy link
Member

@kamipo kamipo commented Feb 3, 2016

Reopen #22415.

Currently mysql2 adapter does not support prepared statements. And mysql adapter has been removed. This means that Active Record does not support prepared statements for MySQL now.

We need to support prepared statements for MySQL. But mysql2 gem still have GC issue in prepared statements. brianmario/mysql2#694

I made default to @prepared_statements = false in Mysql2Adapter for does not affect default behavior. I think it is still safety.

@rails-bot
Copy link

r? @chancancode

(@rails-bot has picked a reviewer for you, use r? to override)

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch 2 times, most recently from 63b18e6 to 22858f6 Compare Feb 3, 2016
@@ -126,6 +141,10 @@ def supports_datetime_with_precision?
version >= '5.6.4'
end

def supports_json?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems unrelated to prepared statement support

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch 2 times, most recently from 07e428d to c962972 Compare Feb 4, 2016
@kamipo
Copy link
Member Author

kamipo commented Feb 4, 2016

Extracted unrelated change to #23469, thanks!
r? @sgrif

@rails-bot rails-bot assigned sgrif and unassigned chancancode Feb 4, 2016
@sgrif
Copy link
Contributor

sgrif commented Feb 5, 2016

This will need to handle nodes which we consider to be unpreparable, similar to what we do in PG and SQLite. We can probably just move that logic up to the abstract adapter now.

@kamipo
Copy link
Member Author

kamipo commented Feb 6, 2016

Do you means #23515?

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch 3 times, most recently from a370e7c to c021804 Compare Feb 13, 2016
@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch 2 times, most recently from 8980117 to d7f30c2 Compare Feb 18, 2016
@sodabrew
Copy link
Contributor

I've just pushed mysql2 0.4.3 with important prepared statement fixes from @kamipo!

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch 3 times, most recently from 428f612 to 86b2576 Compare Feb 28, 2016
@jeremy
Copy link
Member

jeremy commented Apr 19, 2016

Rebase time 😊

end
else
exec_stmt_and_free(sql, name, binds, cache_stmt: prepare) do |_, result|
ActiveRecord::Result.new(result.fields, result.to_a) if result
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In mysql2 0.4.3, result.fields causes segv when num of rows is zero.
This issue was fixed on master by brianmario/mysql2#741.
@sodabrew Could you release mysql2 0.4.4?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! mysql2 0.4.4 is now on Rubygems.org.

@sodabrew
Copy link
Contributor

I've just posted mysql2 0.4.4 with additional prepared statement fixes from @kamipo!

@kamipo
Copy link
Member Author

kamipo commented Apr 19, 2016

Thanks @sodabrew !! 😍

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch from 86b2576 to c2c5708 Compare Apr 19, 2016
@kamipo
Copy link
Member Author

kamipo commented Apr 19, 2016

Rebased on master!

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch from c2c5708 to bc7fe67 Compare Apr 19, 2016
# as values.
def select_one(arel, name = nil, binds = [])
arel, binds = binds_from_relation(arel, binds)
@connection.query_options.merge!(as: :hash)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What this does?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stmt.execute calls result.each in the internal.
https://github.com/brianmario/mysql2/blob/0.4.4/ext/mysql2/statement.c#L384
Need to set query_options before stmt.execute.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But how returning a hash instead of array in the result would be affected by the result.each call?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now. Thanks for the explanation!

@jeremy
Copy link
Member

jeremy commented Apr 20, 2016

Changing to 5.0 since prepared statements are disabled by default.

@jeremy jeremy modified the milestones: 5.0.0, 5.1.0 Apr 20, 2016
# Executes the SQL statement in the context of this connection.
def execute(sql, name = nil)
log(sql, name) { @connection.query(sql) }
end

# Mysql2Adapter doesn't have to free a result after using it, but we use this method
# to write stuff in an abstract way without concerning ourselves about whether it
# needs to be explicitly freed or not.
def execute_and_free(sql, name = nil) # :nodoc:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this method here in a different PR. This abstract class can be removed too since we only have one concrete class.

@kamipo kamipo force-pushed the prepared_statements_for_mysql2_adapter branch from 14dd46f to 3f6574e Compare Apr 21, 2016

gem 'mysql2', '>= 0.3.18', '< 0.5'
gem 'mysql2', '~> 0.4.4'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to bump mysql2 minimum version? 0.4.x is only required if prepared_statements: true.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, mysql2 0.4.3 was broken without prepared statements.
I would like to skip this version.

@jeremy jeremy merged commit 3f6574e into rails:master Apr 24, 2016
jeremy added a commit that referenced this pull request Apr 24, 2016
…adapter

Add prepared statements support for `Mysql2Adapter`
@kamipo kamipo deleted the prepared_statements_for_mysql2_adapter branch Apr 24, 2016
vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 24, 2016
kaspth added a commit that referenced this pull request Apr 24, 2016
@sodabrew
Copy link
Contributor

Hurray!

vipulnsward added a commit to vipulnsward/rails that referenced this pull request Apr 24, 2016
- Rename max to statement_limit
- Remove magic number 1000 from everywhere
- Defined StatementPool::DEFAULT_STATEMENT_LIMIT and started using it everywhere
jeremy pushed a commit that referenced this pull request Apr 24, 2016
- Rename max to statement_limit
- Remove magic number 1000 from everywhere
- Defined StatementPool::DEFAULT_STATEMENT_LIMIT and started using it everywhere

Signed-off-by: Jeremy Daer <jeremydaer@gmail.com>
kamipo added a commit that referenced this pull request Mar 5, 2019
Since #23461, all adapters supports prepared statements, so that clears
the prepared statements cache is no longer database specific.

Actually, I struggled to identify the cause of random CI failure in
#23461, that was missing `@statements.clear` in `clear_cache!`.

This extracts `clear_cache!` to ensure the common concerns in the
abstract adapter.
kamipo added a commit to kamipo/rails that referenced this pull request Jul 17, 2020
Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

* Making boundable queries

While working on rails#23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

rails#26117
7d53993
rails#39219

Now, `where` with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
`predicate_builder` directly.

kufu/activerecord-bitemporal#62

* Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using `User.arel_table[:name]` is not preferable for
`unscope` and `merge`).

Example:

```ruby
id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)
```

* Automatic other table references detection

It works only for the hash syntax.

ee7f666
kamipo added a commit to kamipo/rails that referenced this pull request Jan 12, 2021
Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

* Making boundable queries

While working on rails#23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

rails#26117
7d53993
rails#39219

Now, `where` with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
`predicate_builder` directly.

kufu/activerecord-bitemporal#62

* Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using `User.arel_table[:name]` is not preferable for
`unscope` and `merge`).

Example:

```ruby
id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)
```

* Automatic other table references detection

It works only for the hash syntax.

ee7f666
kamipo added a commit to kamipo/rails that referenced this pull request Jan 14, 2021
Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

* Making boundable queries

While working on rails#23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

rails#26117
7d53993
rails#39219

Now, `where` with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
`predicate_builder` directly.

kufu/activerecord-bitemporal#62

* Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using `User.arel_table[:name]` is not preferable for
`unscope` and `merge`).

Example:

```ruby
id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)
```

* Automatic other table references detection

It works only for the hash syntax.

ee7f666
kamipo added a commit to kamipo/rails that referenced this pull request Mar 11, 2021
Revert "Revert "Merge pull request rails#39613 from kamipo/where_with_custom_operator""

This reverts commit da02291.

```ruby
posts = Post.order(:id)

posts.where("id >": 9).pluck(:id)  # => [10, 11]
posts.where("id >=": 9).pluck(:id) # => [9, 10, 11]
posts.where("id <": 3).pluck(:id)  # => [1, 2]
posts.where("id <=": 3).pluck(:id) # => [1, 2, 3]
```

From type casting and table/column name resolution's point of view,
`where("created_at >=": time)` is better alternative than `where("created_at >= ?", time)`.

```ruby
class Post < ActiveRecord::Base
  attribute :created_at, :datetime, precision: 3
end

time = Time.now.utc # => 2020-06-24 10:11:12.123456 UTC

Post.create!(created_at: time) # => #<Post id: 1, created_at: "2020-06-24 10:11:12.123000">

# SELECT `posts`.* FROM `posts` WHERE (created_at >= '2020-06-24 10:11:12.123456')
Post.where("created_at >= ?", time) # => []

# SELECT `posts`.* FROM `posts` WHERE `posts`.`created_at` >= '2020-06-24 10:11:12.123000'
Post.where("created_at >=": time) # => [#<Post id: 1, created_at: "2020-06-24 10:11:12.123000">]
```

As a main contributor of the predicate builder area, I'd recommend to
people use the hash syntax, the hash syntax also have other useful
effects (making boundable queries, unscopeable queries, hash-like
relation merging friendly, automatic other table references detection).

* Making boundable queries

While working on rails#23461, I realized that Active Record doesn't generate
boundable queries perfectly, so I've been improving generated queries to
be boundable for a long time.

e.g.

rails#26117
7d53993
rails#39219

Now, `where` with the hash syntax will generate boundable queries
perfectly.

I also want to generate boundable queries with a comparison operator in
a third party gem, but currently there is no other way than calling
`predicate_builder` directly.

kufu/activerecord-bitemporal#62

* Unscopeable queries, Hash-like relation merging friendly

Unscopeable, and Hash-like merging friendly queries are relying on where
clause is an array of attr with value, and attr name is normalized as a
string (i.e. using `User.arel_table[:name]` is not preferable for
`unscope` and `merge`).

Example:

```ruby
id = User.arel_table[:id]

users = User.where(id.gt(1).and(id.lteq(10)))

# no-op due to `id.gt(1).and(id.lteq(10))` is not an attr with value
users.unscope(:id)
```

* Automatic other table references detection

It works only for the hash syntax.

ee7f666
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants