Permalink
Browse files

INSERT INTO schema_migrations in 1 SQL

We found that inserting all 600 schema_migrations for our mid-sized app takes about a minute on a cloud based CI environment.

I assume that the original code did not use multi-row-insert because SQLite3 was not supporting the syntax back then,
but it's been supported since 3.7.11: http://www.sqlite.org/releaselog/3_7_11.html
  • Loading branch information...
amatsuda committed Jan 27, 2016
1 parent 3844854 commit 42dd2336b31a8d98776d039a2b9fd7f834156a78
@@ -1,3 +1,8 @@
* Improve schema_migrations insertion performance by inserting all versions
in one INSERT SQL.
*Akira Matsuda*, *Naoto Koshikawa*
* Using `references` or `belongs_to` in migrations will always add index
for the referenced column by default, without adding `index: true` option
to generated migration file. Users can opt out of this by passing
@@ -957,9 +957,9 @@ def foreign_key_options(from_table, to_table, options) # :nodoc:
def dump_schema_information #:nodoc:
sm_table = ActiveRecord::Migrator.schema_migrations_table_name
ActiveRecord::SchemaMigration.order('version').map { |sm|
"INSERT INTO #{sm_table} (version) VALUES ('#{sm.version}');"
}.join "\n\n"
sql = "INSERT INTO #{sm_table} (version) VALUES "
sql << ActiveRecord::SchemaMigration.order('version').pluck(:version).map {|v| "('#{v}')" }.join(', ')
sql << ";\n\n"
end
# Should not be called normally, but this operation is non-destructive.
@@ -987,14 +987,12 @@ def assume_migrated_upto_version(version, migrations_paths)
execute "INSERT INTO #{sm_table} (version) VALUES ('#{version}')"
end
inserted = Set.new
(versions - migrated).each do |v|
if inserted.include?(v)
raise "Duplicate migration #{v}. Please renumber your migrations to resolve the conflict."
elsif v < version
execute "INSERT INTO #{sm_table} (version) VALUES ('#{v}')"
inserted << v
inserting = (versions - migrated).select {|v| v < version}
if inserting.any?
if (duplicate = inserting.detect {|v| inserting.count(v) > 1})
raise "Duplicate migration #{duplicate}. Please renumber your migrations to resolve the conflict."
end
execute "INSERT INTO #{sm_table} (version) VALUES #{inserting.map {|v| '(#{v})'}.join(', ') }"
end
end

2 comments on commit 42dd233

@jeremy

This comment has been minimized.

Member

jeremy replied Apr 21, 2016

This has caused some issues for people on older SQLite.

It's a tough bug to diagnose because it isn't obvious that SQLite doesn't support multi-insert. The SQL certainly looks fine, but it complains there's an error!

Proposal to bump Rails 5 minimum SQLite version to 3.8+ in response: https://groups.google.com/d/msg/rubyonrails-core/Mm4ZN_g1RXk/2ZHvuxKwCAAJ

Maybe it'd be simpler to do a multi-insert only if the adapter supports it? Then the adapter is free to version-check or feature-check the database to see whether it's possible.

@vipulnsward

This comment has been minimized.

Member

vipulnsward replied Apr 22, 2016

It makes sense to allow both given the number of bug reports we have had and that Ubuntu 12.04 LTS uses unsupported version. I will send a PR for this.

Please sign in to comment.