Fix upsert warning for MySQL
#51274
Merged
Fix upsert warning for MySQL
#51274
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fatkodima
force-pushed
the
fix-mysql-upsert-warning
branch
from
89c1430
to
0ad99dd
Compare
|
That aliasing doesn't work on MySQL 5? |
|
Seems like not :/ |
casperisfine
reviewed
Mar 7, 2024
| sql = +"INSERT #{insert.into} #{insert.values_list}" | ||
| new_on_duplicate_syntax = (!mariadb? && database_version >= "8.0.0") | ||
|
|
||
| if new_on_duplicate_syntax |
There was a problem hiding this comment.
I think I'd prefer a single top level if new_on_duplicate_syntax even if that means some level of duplication.
I even wouldn't be against just using two sub-methods, one for the old syntax one for the old one.
There was a problem hiding this comment.
I think I'd prefer a single top level if
new_on_duplicate_syntaxeven if that means some level of duplication.
This was my original thought. Updated with this version.
fatkodima
force-pushed
the
fix-mysql-upsert-warning
branch
from
0ad99dd
to
0e7826e
Compare
casperisfine
reviewed
Mar 7, 2024
| skip unless supports_insert_on_duplicate_update? | ||
|
|
||
| begin | ||
| with_db_warnings_action(:raise) do |
There was a problem hiding this comment.
Not worth doing here, more for a followup, but I wonder if it would make sense to run CI/tests with DB warnings treated as errors by default. I think it might have caught this issue way sooner.
Also we probably should backport that fix.
|
cc @byroot: do backports Monday. |
rafaelfranca
pushed a commit
that referenced
this pull request
Mar 7, 2024
|
Already did |
Merged
yahonda
added a commit
to yahonda/rails
that referenced
this pull request
Mar 8, 2024
…RT INTO ... ON DUPLICATE KEY UPDATE statement This commit addresses the following errors against MySQL 8.0.18 or lower version of MySQL 8.0. - Steps to reproduce ```ruby git clone https://github.com/rails/rails cd rails git clone https://github.com/rails/buildkite-config .buildkite/ RUBY_IMAGE=ruby:3.3 docker-compose -f .buildkite/docker-compose.yml build base && CI=1 MYSQL_IMAGE=mysql:8.0.18 docker-compose -f .buildkite/docker-compose.yml run mysqldb runner activerecord 'rake db:mysql:rebuild test:mysql2' ``` - Actual behavior ```ruby ... snip ... Error: InsertAllTest#test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden: ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AS `ships_values` ON DUPLICATE KEY UPDATE updated_at=(CASE WHEN (`ships`.`name`<' at line 1 /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `_query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `block in query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `handle_interrupt' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `query' lib/active_record/connection_adapters/mysql2/database_statements.rb:104:in `block (2 levels) in raw_execute' lib/active_record/connection_adapters/abstract_adapter.rb:997:in `block in with_raw_connection' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `block in synchronize' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `synchronize' lib/active_record/connection_adapters/abstract_adapter.rb:969:in `with_raw_connection' lib/active_record/connection_adapters/mysql2/database_statements.rb:102:in `block in raw_execute' /rails/activesupport/lib/active_support/notifications/instrumenter.rb:58:in `instrument' lib/active_record/connection_adapters/abstract_adapter.rb:1112:in `log' lib/active_record/connection_adapters/mysql2/database_statements.rb:101:in `raw_execute' lib/active_record/connection_adapters/abstract_mysql_adapter.rb:237:in `execute_and_free' lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query' lib/active_record/connection_adapters/abstract/database_statements.rb:171:in `exec_insert_all' lib/active_record/connection_adapters/abstract/query_cache.rb:26:in `exec_insert_all' lib/active_record/insert_all.rb:55:in `execute' lib/active_record/insert_all.rb:13:in `block in execute' lib/active_record/connection_adapters/abstract/connection_pool.rb:384:in `with_connection' lib/active_record/connection_handling.rb:270:in `with_connection' lib/active_record/insert_all.rb:12:in `execute' lib/active_record/persistence.rb:363:in `upsert_all' test/cases/insert_all_test.rb:561:in `block in test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' test/cases/insert_all_test.rb:809:in `with_record_timestamps' test/cases/insert_all_test.rb:560:in `test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' bin/rails test /rails/activerecord/test/cases/insert_all_test.rb:557 E ... snip ... 8856 runs, 25842 assertions, 1 failures, 52 errors, 41 skips ``` Follow up rails#51274 Refer to these release notes, WL and commits for MySQL 8.0.19 and 8.0.20. - MySQL 8.0.19 supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-19.html > MySQL now supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement > for the row to be inserted and its columns. Consider a statement such as this one: https://dev.mysql.com/worklog/task/?id=6312 mysql/mysql-server@c39355e - MySQL 8.0.20 deprecates the old `VALUES()` syntax in INSERT ... ON DUPLICATE KEY UPDATE statements https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-20.html > The use of VALUES() to access new row values in INSERT ... ON DUPLICATE KEY UPDATE statements > is now deprecated, and is subject to removal in a future MySQL release. > Instead, you should use aliases for the new row and its columns as implemented in MySQL 8.0.19 and later. https://dev.mysql.com/worklog/task/?id=13325 mysql/mysql-server@6f3b9df
4 tasks
| elsif insert.update_duplicates? | ||
| sql << " ON DUPLICATE KEY UPDATE " | ||
| if insert.raw_update_sql? | ||
| sql << insert.raw_update_sql |
There was a problem hiding this comment.
Note: we have been ignoring the MySQL warning
Currently, we have a couple of places that provide a raw_update_sql pointing to the VALUES(...), which, with this change, created a mixed query:
INSERT table (...) VALUES (...) as table_alias ON DUPLICATE KEY UPDATE x=VALUES(x)
The query didn't work as expected, and the expected behavior broke. We may need some way to keep using the previous behavior for a while. (I'm working on a PR unless someone beat me)
cc: @matthewd
There was a problem hiding this comment.
Yes. If the raw_update_sql is provided, we should not try to alias, because we don't even know the alias.
There was a problem hiding this comment.
Created this PR to address this issue.
maxprokopiev
added a commit
to maxprokopiev/rails
that referenced
this pull request
Mar 19, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
4 tasks
maxprokopiev
added a commit
to maxprokopiev/rails
that referenced
this pull request
Mar 19, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
maxprokopiev
added a commit
to maxprokopiev/rails
that referenced
this pull request
Mar 19, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
maxprokopiev
added a commit
to maxprokopiev/rails
that referenced
this pull request
Mar 19, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
viralpraxis
pushed a commit
to viralpraxis/rails
that referenced
this pull request
Mar 24, 2024
…RT INTO ... ON DUPLICATE KEY UPDATE statement This commit addresses the following errors against MySQL 8.0.18 or lower version of MySQL 8.0. - Steps to reproduce ```ruby git clone https://github.com/rails/rails cd rails git clone https://github.com/rails/buildkite-config .buildkite/ RUBY_IMAGE=ruby:3.3 docker-compose -f .buildkite/docker-compose.yml build base && CI=1 MYSQL_IMAGE=mysql:8.0.18 docker-compose -f .buildkite/docker-compose.yml run mysqldb runner activerecord 'rake db:mysql:rebuild test:mysql2' ``` - Actual behavior ```ruby ... snip ... Error: InsertAllTest#test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden: ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AS `ships_values` ON DUPLICATE KEY UPDATE updated_at=(CASE WHEN (`ships`.`name`<' at line 1 /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `_query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `block in query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `handle_interrupt' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `query' lib/active_record/connection_adapters/mysql2/database_statements.rb:104:in `block (2 levels) in raw_execute' lib/active_record/connection_adapters/abstract_adapter.rb:997:in `block in with_raw_connection' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `block in synchronize' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `synchronize' lib/active_record/connection_adapters/abstract_adapter.rb:969:in `with_raw_connection' lib/active_record/connection_adapters/mysql2/database_statements.rb:102:in `block in raw_execute' /rails/activesupport/lib/active_support/notifications/instrumenter.rb:58:in `instrument' lib/active_record/connection_adapters/abstract_adapter.rb:1112:in `log' lib/active_record/connection_adapters/mysql2/database_statements.rb:101:in `raw_execute' lib/active_record/connection_adapters/abstract_mysql_adapter.rb:237:in `execute_and_free' lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query' lib/active_record/connection_adapters/abstract/database_statements.rb:171:in `exec_insert_all' lib/active_record/connection_adapters/abstract/query_cache.rb:26:in `exec_insert_all' lib/active_record/insert_all.rb:55:in `execute' lib/active_record/insert_all.rb:13:in `block in execute' lib/active_record/connection_adapters/abstract/connection_pool.rb:384:in `with_connection' lib/active_record/connection_handling.rb:270:in `with_connection' lib/active_record/insert_all.rb:12:in `execute' lib/active_record/persistence.rb:363:in `upsert_all' test/cases/insert_all_test.rb:561:in `block in test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' test/cases/insert_all_test.rb:809:in `with_record_timestamps' test/cases/insert_all_test.rb:560:in `test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' bin/rails test /rails/activerecord/test/cases/insert_all_test.rb:557 E ... snip ... 8856 runs, 25842 assertions, 1 failures, 52 errors, 41 skips ``` Follow up rails#51274 Refer to these release notes, WL and commits for MySQL 8.0.19 and 8.0.20. - MySQL 8.0.19 supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-19.html > MySQL now supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement > for the row to be inserted and its columns. Consider a statement such as this one: https://dev.mysql.com/worklog/task/?id=6312 mysql/mysql-server@c39355e - MySQL 8.0.20 deprecates the old `VALUES()` syntax in INSERT ... ON DUPLICATE KEY UPDATE statements https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-20.html > The use of VALUES() to access new row values in INSERT ... ON DUPLICATE KEY UPDATE statements > is now deprecated, and is subject to removal in a future MySQL release. > Instead, you should use aliases for the new row and its columns as implemented in MySQL 8.0.19 and later. https://dev.mysql.com/worklog/task/?id=13325 mysql/mysql-server@6f3b9df
viralpraxis
pushed a commit
to viralpraxis/rails
that referenced
this pull request
Mar 24, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
fractaledmind
pushed a commit
to fractaledmind/rails
that referenced
this pull request
May 13, 2024
…RT INTO ... ON DUPLICATE KEY UPDATE statement This commit addresses the following errors against MySQL 8.0.18 or lower version of MySQL 8.0. - Steps to reproduce ```ruby git clone https://github.com/rails/rails cd rails git clone https://github.com/rails/buildkite-config .buildkite/ RUBY_IMAGE=ruby:3.3 docker-compose -f .buildkite/docker-compose.yml build base && CI=1 MYSQL_IMAGE=mysql:8.0.18 docker-compose -f .buildkite/docker-compose.yml run mysqldb runner activerecord 'rake db:mysql:rebuild test:mysql2' ``` - Actual behavior ```ruby ... snip ... Error: InsertAllTest#test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden: ActiveRecord::StatementInvalid: Mysql2::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AS `ships_values` ON DUPLICATE KEY UPDATE updated_at=(CASE WHEN (`ships`.`name`<' at line 1 /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `_query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:151:in `block in query' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `handle_interrupt' /usr/local/bundle/gems/mysql2-0.5.6/lib/mysql2/client.rb:150:in `query' lib/active_record/connection_adapters/mysql2/database_statements.rb:104:in `block (2 levels) in raw_execute' lib/active_record/connection_adapters/abstract_adapter.rb:997:in `block in with_raw_connection' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:23:in `block in synchronize' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `handle_interrupt' /rails/activesupport/lib/active_support/concurrency/load_interlock_aware_monitor.rb:19:in `synchronize' lib/active_record/connection_adapters/abstract_adapter.rb:969:in `with_raw_connection' lib/active_record/connection_adapters/mysql2/database_statements.rb:102:in `block in raw_execute' /rails/activesupport/lib/active_support/notifications/instrumenter.rb:58:in `instrument' lib/active_record/connection_adapters/abstract_adapter.rb:1112:in `log' lib/active_record/connection_adapters/mysql2/database_statements.rb:101:in `raw_execute' lib/active_record/connection_adapters/abstract_mysql_adapter.rb:237:in `execute_and_free' lib/active_record/connection_adapters/mysql2/database_statements.rb:23:in `internal_exec_query' lib/active_record/connection_adapters/abstract/database_statements.rb:171:in `exec_insert_all' lib/active_record/connection_adapters/abstract/query_cache.rb:26:in `exec_insert_all' lib/active_record/insert_all.rb:55:in `execute' lib/active_record/insert_all.rb:13:in `block in execute' lib/active_record/connection_adapters/abstract/connection_pool.rb:384:in `with_connection' lib/active_record/connection_handling.rb:270:in `with_connection' lib/active_record/insert_all.rb:12:in `execute' lib/active_record/persistence.rb:363:in `upsert_all' test/cases/insert_all_test.rb:561:in `block in test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' test/cases/insert_all_test.rb:809:in `with_record_timestamps' test/cases/insert_all_test.rb:560:in `test_upsert_all_implicitly_sets_timestamps_on_create_when_model_record_timestamps_is_false_but_overridden' bin/rails test /rails/activerecord/test/cases/insert_all_test.rb:557 E ... snip ... 8856 runs, 25842 assertions, 1 failures, 52 errors, 41 skips ``` Follow up rails#51274 Refer to these release notes, WL and commits for MySQL 8.0.19 and 8.0.20. - MySQL 8.0.19 supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-19.html > MySQL now supports aliases in the VALUES and SET clauses of INSERT INTO ... ON DUPLICATE KEY UPDATE statement > for the row to be inserted and its columns. Consider a statement such as this one: https://dev.mysql.com/worklog/task/?id=6312 mysql/mysql-server@c39355e - MySQL 8.0.20 deprecates the old `VALUES()` syntax in INSERT ... ON DUPLICATE KEY UPDATE statements https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-20.html > The use of VALUES() to access new row values in INSERT ... ON DUPLICATE KEY UPDATE statements > is now deprecated, and is subject to removal in a future MySQL release. > Instead, you should use aliases for the new row and its columns as implemented in MySQL 8.0.19 and later. https://dev.mysql.com/worklog/task/?id=13325 mysql/mysql-server@6f3b9df
fractaledmind
pushed a commit
to fractaledmind/rails
that referenced
this pull request
May 13, 2024
The change to upserts (rails#51274) is causing an issue with Vitess, since it doesn't support the `row_alias` syntax added in MySQL 8.0.19. There is an ongoing work (vitessio/vitess#15510) to add that support and it is likely to be included into Vitess v20, but in the meantime it would be nice to have an ability to control that behavior in the existing apps.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #51268.
The code looks scary we these
ifs, but I am not sure we will make it much better extracting pieces into some other places.