Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Parallelize tests only when overhead is justified
Parallelizing tests has a cost in terms of database setup and fixture loading. This change makes Rails disable parallelization when the number of tests is below a configurable threshold. When running tests in parallel each process gets its own database instance. On each execution, each process will update each database schema (if needed) and load all the fixtures. This can be very expensive for non trivial datasets. As an example, for HEY, when running a single file with 18 tests, running tests in parallel in my box adds an overhead of 13 seconds versus not parallelizing them. Of course parallelizing is totally worthy when there are many tests to run, but not when running just a few tests. The threshold is configurable via config.active_support.test_parallelization_minimum_number_of_tests, which is 30 50 by default. This also adds some tracing to know how tests are being executed: When in parallel: ``` Running 2829 tests in parallel in 8 processes ``` When not in parallel: ``` Running 15 tests in a single process (parallelization threshold is 30) ```
- Loading branch information
1 parent
d364cfb
commit ecc5afe
Showing
8 changed files
with
140 additions
and
15 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
72 changes: 72 additions & 0 deletions
72
activesupport/lib/active_support/testing/parallelize_executor.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# frozen_string_literal: true | ||
|
||
module ActiveSupport | ||
module Testing | ||
class ParallelizeExecutor # :nodoc: | ||
attr_reader :size, :parallelize_with, :parallel_executor | ||
|
||
def initialize(size:, with:) | ||
@size = size | ||
@parallelize_with = with | ||
@parallel_executor = build_parallel_executor | ||
end | ||
|
||
def start | ||
parallelize if should_parallelize? | ||
show_execution_info | ||
|
||
parallel_executor.start if parallelized? | ||
end | ||
|
||
def <<(work) | ||
parallel_executor << work if parallelized? | ||
end | ||
|
||
def shutdown | ||
parallel_executor.shutdown if parallelized? | ||
end | ||
|
||
private | ||
def build_parallel_executor | ||
case parallelize_with | ||
when :processes | ||
Testing::Parallelization.new(size) | ||
when :threads | ||
ActiveSupport::TestCase.lock_threads = false if defined?(ActiveSupport::TestCase.lock_threads) | ||
Minitest::Parallel::Executor.new(size) | ||
else | ||
raise ArgumentError, "#{parallelize_with} is not a supported parallelization executor." | ||
end | ||
end | ||
|
||
def parallelize | ||
@parallelized = true | ||
Minitest::Test.parallelize_me! | ||
end | ||
|
||
def parallelized? | ||
@parallelized | ||
end | ||
|
||
def should_parallelize? | ||
ENV["PARALLEL_WORKERS"] || tests_count > ActiveSupport.test_parallelization_minimum_number_of_tests | ||
end | ||
|
||
def tests_count | ||
@tests_count ||= Minitest::Runnable.runnables.sum { |runnable| runnable.runnable_methods.size } | ||
end | ||
|
||
def show_execution_info | ||
puts execution_info | ||
end | ||
|
||
def execution_info | ||
if should_parallelize? | ||
"Running #{tests_count} tests in parallel using #{parallel_executor.size} #{parallelize_with}" | ||
else | ||
"Running #{tests_count} tests in a single process (parallelization threshold is #{ActiveSupport.test_parallelization_minimum_number_of_tests})" | ||
end | ||
end | ||
end | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters