Skip to content

Commit

Permalink
Use execute_batch2 rather than execute_batch to fix performance r…
Browse files Browse the repository at this point in the history
…egression for fixture loading

d8d6bd5 makes fixture loading to bulk statements by using
`execute_batch` for sqlite3 adapter. But `execute_batch` is slower and
it caused the performance regression for fixture loading.

In sqlite3 1.4.0, it have new batch method `execute_batch2`. I've
confirmed `execute_batch2` is extremely faster than `execute_batch`.
So I think it is worth to upgrade sqlite3 to 1.4.0 to use that method.

Before:

```
% ARCONN=sqlite3 bundle exec ruby -w -Itest test/cases/associations/eager_test.rb -n test_eager_loading_too_may_ids
Using sqlite3
Run options: -n test_eager_loading_too_may_ids --seed 35790

# Running:

.

Finished in 202.437406s, 0.0049 runs/s, 0.0049 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
ARCONN=sqlite3 bundle exec ruby -w -Itest  -n test_eager_loading_too_may_ids  142.57s user 60.83s system 98% cpu 3:27.08 total
```

After:

```
% ARCONN=sqlite3 bundle exec ruby -w -Itest test/cases/associations/eager_test.rb -n test_eager_loading_too_may_ids
Using sqlite3
Run options: -n test_eager_loading_too_may_ids --seed 16649

# Running:

.

Finished in 8.471032s, 0.1180 runs/s, 0.1180 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
ARCONN=sqlite3 bundle exec ruby -w -Itest  -n test_eager_loading_too_may_ids  10.71s user 1.36s system 95% cpu 12.672 total
```
  • Loading branch information
kamipo committed Apr 3, 2019
1 parent d39b2b6 commit 0908184
Show file tree
Hide file tree
Showing 7 changed files with 9 additions and 10 deletions.
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ platforms :ruby, :mswin, :mswin64, :mingw, :x64_mingw do
gem "racc", ">=1.4.6", require: false

# Active Record.
gem "sqlite3", "~> 1.3", ">= 1.3.6"
gem "sqlite3", "~> 1.4"

group :db do
gem "pg", ">= 0.18.0"
Expand Down
7 changes: 3 additions & 4 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,7 @@ GEM
image_processing (1.7.1)
mini_magick (~> 4.0)
ruby-vips (>= 2.0.13, < 3)
jar-dependencies (0.4.0)
jaro_winkler (1.5.2)
jaro_winkler (1.5.2-java)
jdbc-mysql (5.1.46)
Expand Down Expand Up @@ -468,9 +469,7 @@ GEM
actionpack (>= 4.0)
activesupport (>= 4.0)
sprockets (>= 3.0.0)
sqlite3 (1.3.13)
sqlite3 (1.3.13-x64-mingw32)
sqlite3 (1.3.13-x86-mingw32)
sqlite3 (1.4.0)
stackprof (0.2.12)
sucker_punch (2.1.1)
concurrent-ruby (~> 1.0)
Expand Down Expand Up @@ -582,7 +581,7 @@ DEPENDENCIES
sidekiq
sneakers
sprockets-export
sqlite3 (~> 1.3, >= 1.3.6)
sqlite3 (~> 1.4)
stackprof
sucker_punch
turbolinks (~> 5)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def execute_batch(sql, name = nil)

log(sql, name) do
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
@connection.execute_batch(sql)
@connection.execute_batch2(sql)
end
end
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
require "active_record/connection_adapters/sqlite3/schema_dumper"
require "active_record/connection_adapters/sqlite3/schema_statements"

gem "sqlite3", "~> 1.3", ">= 1.3.6"
gem "sqlite3", "~> 1.4"
require "sqlite3"

module ActiveRecord
Expand Down
2 changes: 1 addition & 1 deletion railties/lib/rails/generators/database.rb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def gem_for_database(database = options[:database])
case database
when "mysql" then ["mysql2", [">= 0.4.4"]]
when "postgresql" then ["pg", [">= 0.18", "< 2.0"]]
when "sqlite3" then ["sqlite3", ["~> 1.3", ">= 1.3.6"]]
when "sqlite3" then ["sqlite3", ["~> 1.4"]]
when "oracle" then ["activerecord-oracle_enhanced-adapter", nil]
when "frontbase" then ["ruby-frontbase", nil]
when "sqlserver" then ["activerecord-sqlserver-adapter", nil]
Expand Down
2 changes: 1 addition & 1 deletion railties/test/generators/app_generator_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ def test_config_database_is_added_by_default
if defined?(JRUBY_VERSION)
assert_gem "activerecord-jdbcsqlite3-adapter"
else
assert_gem "sqlite3", "'~> 1.3', '>= 1.3.6'"
assert_gem "sqlite3", "'~> 1.4'"
end
end

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ class ChangeGeneratorTest < Rails::Generators::TestCase

assert_file("Gemfile") do |content|
assert_match "# Use sqlite3 as the database for Active Record", content
assert_match "gem 'sqlite3', '~> 1.3', '>= 1.3.6'", content
assert_match "gem 'sqlite3', '~> 1.4'", content
end
end

Expand Down

0 comments on commit 0908184

Please sign in to comment.