Support parallelization #235

tikkss · 2024-01-08T00:26:14Z

I'm a contributor of red-data-tools/red-datasets. The tests for red-datasets use test-unit. Thanks!

With the increasing size of test suite, the execution time for tests are slow down. I'll give it a try to improve the execution time below:

First, improve the slowest test.
Next, improve the next slowest test.
Repeat it until exists slow tests.

Nevertheless the execution time for tests are slow, I want to parallelization support of test-unit.

I would like to discuss the design of parallelization support of test-unit in this issue.

kou · 2024-01-15T00:47:36Z

Requirements:

Supported backends: Thread, spawn (not fork) and Ractor
Use pull style (workers pull a next test from controller) not push style (controller pushes tests to workers)
Fast as much as possible
- e.g.: Avoid calling startup/shutdown (test case level fixture) as much as possible by running tests for a test case in the same worker
Don't depend on external libraries including default/bundled gems such as drb
Implement step-by-step
- e.g.: Implement Thread backend and then implement spawn backend
...

tikkss · 2024-01-27T22:59:17Z

Thanks for your requirements!

Supported backends: Thread, spawn (not fork) and Ractor

Let me summarize my understanding.

Thread:

Easier to implement than spawn and Ractor
Need to exchange test execution results between Thread's
Creating a Thread involves overhead, but is faster than fork and spawn
Although a typical Thread can execute parallel, Ruby's Global VM Lock (GVL) constraints allow only a single execution at a time
- However, this restriction doesn't apply during I/O operations, such as file downloads in red-data-tools/red-datasets
Ruby's GVL switches automatically

fork:

Only on UNIX-based platforms
Copying a process involves overhead, making it slower than Thread but faster than spawn
Copying is fast due to the Copy On Write (COW) approach, where memory is allocated only when needed
Managing $LOAD_PATH is complex
Mixing fork and Thread can be dangerous

spawn:

Also works on Windows
Creating a process involves overhead, making it slower than Thread and fork
Easier to Implement than fork

Key Points:

As a testing framework, it's important to run on various platforms
Keeping workers active without breaks is important
We plan to start the implementation with Thread and subsequently add the implementation of spawn (not fork) and Ractor

I will write about Ractor later.

tikkss · 2024-02-07T21:33:14Z

Ractor:

Ruby's unique parallelization feature
Execute parallel multiple tasks within a single process
- Safe and highly parallelizable Thread
Share immutable data between Ractor
- Can’t share mutable data between Ractor, but can be passed by copying
Also works on Windows
Ractor is an experimental feature, so we implement it last
- It's difficult to determine if it's running or not. Is it because of Ractor, or is it our fault?

Use pull style (workers pull the next test from the controller) not push style (controller pushes tests to workers)

Let me summarize my understanding.

We want to finish the testing as quickly as possible. So we need to assign tasks promptly to workers who seem to be idle.

Here, we designate the entities generating tests as producers and executing tests as consumers. There are two ways of assigning tests. It's push style and pull style. We use pull style because:

push style:

Producer pushes tests to consumers
Pushing tests evenly is difficult because the execution time of tests varies depending on the test cases
Pushing all of the tests at once. Results in significant variance
Pushing tests one by one. The producer needs to know consumers are idle, so the producer polls consumers when consumers are idle or not
- Difficult to poll in parallel

pull style:

Consumers pull the next tests from the producers
Pulling tests one by one for the consumers to pull from the producer, so the variance is minimal
Is it possible to allocate while maintaining consistency when pulled in parallel?

Fast as much as possible
e.g.: Avoid calling startup/shutdown (test case level fixture) as much as possible by running tests for a test case in the same worker

Let me summarize my understanding.

test-unit calling startup/shutdown for the each TestCase. Such as preparing a database.
When startup/shutdown is called by different consumers, it leads to a lot of waste, so we want to call the startup/shutdown on the same consumer as much as possible.

tikkss · 2024-02-18T11:49:35Z

Don't depend on external libraries including default/bundled gems such as drb

Let me summarize my understanding.

We want the testing framework to work in various environments.

Using external libraries can make implementation more convenient, but it also increases dependencies, making it challenging to work in various environments.

test-unit is a testing framework, so its priority is to work in various environments.

So it doesn't depend on external libraries including default/bundled gems.

Implement step-by-step
e.g.: Implement Thread backend and then implement spawn backend

Internal workings of the test-unit following:

Test::Unit::TestCase wraps up a collection of test methods
Test::Unit::TestSuite wraps up a collection of TestCases and other TestSuites
At least one TestSuite is always required when executing tests

Let's implement the Thread backend!

We temporarily named the producer ParallelRunner and the consumer ThreadWorker.

tikkss · 2024-03-07T21:24:41Z

First, we implement a standalone script that communicates with Thread base.
After, we implement it with test-unit.

Ruby's Thread has the following:

Thread.new { expr } creates a new Thread and expr is run in parallel
Ruby interpreter invokes with the first Thread (called main Thread)
If main Thread terminated, Ruby interpreter sends all running threads to terminate execution
Thread#join waits until non-main thread is finished
Variables created outside of Thread.new { expr } can lead to unintended results when referenced within expr
- Using Thread.new(*arg) { |*arg| expr } ensures that variables are not shared between threads, thus preventing race conditions

First, we implement Consumer only without Producer.
After, we implement Consumer with Producer.

tikkss · 2024-03-15T23:43:03Z

First, we implement Consumer only without Producer.
After, we implement Consumer with Producer.

In general, the number of tasks is often greater than the number of consumers.
If consumers have work, they work, otherwise they end. So it becomes a loop.

Ruby's Thread automatically switches execution to another thread. It occurs at the following points:

Kernel.#sleep
I/O such as network communication, read/write to sockets

We can also explicitly switch execution to another thread using the following:

Thread.pass

When multiple threads execute destructive operations on the same object, it occurs race conditions.
To resolve it, we need to implement exclusive control to prevent at the same time destructive operations.
Thread::Queue is a queue that provides mutual exclusion.

Thread::Queue#pop blocks execution if the queue is empty.
If we don't notify to consumers when the work is finished, a deadlock will occur.

We implemented Consumer with Producer.

We implement it with test-unit
How do we address the workload imbalance?

tikkss · 2024-03-27T21:44:48Z

We implement it with test-unit

How do we address the workload imbalance?

We created 100 tests for testing below.

# sample-test.rb
require 'test-unit'

class SampleTest < Test::Unit::TestCase
  100.times do |i|
    define_method("test_#{i}") do
      p i
    end
  end
end

We checked the version of the local test-unit.

# lib/test/unit/version.rb
module Test
  module Unit
    VERSION = "3.6.3"
  end
end

To use the local test-unit, we added lib to Kernel$$LOAD_PATH below.

# sample-test.rb
require 'test-unit'
require "test/unit/version"
p Test::Unit::VERSION

class SampleTest < Test::Unit::TestCase
  100.times do |i|
    define_method("test_#{i}") do
      p i
    end
  end
end

$ ruby sample-test.rb
"3.6.1"

$ ruby -I lib sample-test.rb
"3.6.3"

We added Kernel.#caller to lib/test/unit/testcase.rb#run to check the backtrace and try running only one test below.

# lib/test/unit/testcase.rb
module Test
  module Unit
    class TestCase
      def run(result)
        puts caller
        begin
        ...
      end
    end
  end
end

$ ruby -I lib sample-test.rb -n test_1
~/src/github.com/test-unit/test-unit/lib/test/unit/testsuite.rb:124:in `run_test'
~/src/github.com/test-unit/test-unit/lib/test/unit/testsuite.rb:53:in `run'
...

We found multiple tests are being executed sequentially.

# lib/test/unit/testsuite.rb
module Test
  module Unit
    class TestSuite
      def run(result, &progress_block)
        ...
        while test = @tests.shift
          @n_tests += test.size
          run_test(test, result, &progress_block)
          @passed = false unless test.passed?
        end
        ...
      end
    end
  end
end

We implemented Consumer with Producer below.

# lib/test/unit/testsuite.rb
module Test
  module Unit
    class TestSuite
      def run(result, &progress_block)
        @start_time = Time.now
        yield(STARTED, name)
        yield(STARTED_OBJECT, self)
        run_startup(result)
        n_consumers = 5
        tests = Thread::Queue.new
        producer = Thread.new do
          @tests.each do |test|
            tests << test
          end
          n_consumers.times do
            tests << nil
          end
        end

        consumers = n_consumers.times.collect do
          Thread.new do
            loop do
              test = tests.pop
              break if test.nil?

              @n_tests += test.size
              run_test(test, result, &progress_block)
              @passed = false unless test.passed?
            end
          end
        end

        producer.join
        consumers.each(&:join)

        # while test = @tests.shift
        #   @n_tests += test.size
        #   # run_test(test, result, &progress_block)
        #   @passed = false unless test.passed?
        # end
      ensure
        begin
          run_shutdown(result)
        ensure
          @elapsed_time = Time.now - @start_time
          yield(FINISHED, name)
          yield(FINISHED_OBJECT, self)
        end
      end
    end
  end
end

Switching the backend with an option
Make the details run in parallel

for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss · 2024-04-02T21:47:56Z

Next:

Switching the backend with an option

Make the details run in parallel

We refactored the code to execute multiple tests in the TestSuite for future.
We making it easier to customize later by extracting it to #run_tests.

We've started abstracting the #run from TestSuite to TestSuiteRunner

The following methods has been migrated from TestSuite to TestSuiteRunner.

#initialize
#run
#run_startup

Migrate the folloing methods from TestSuite to TestSuiteRunner
- #run_tests
- #run_test
- #run_shutdown
- #handle_exception
Invoke TestSuiteRunner from TestSuite
Switching the backend with an option
Implement TestSuiteThreadRunner

Before: `Array#shift` **removes** and returns leading elements. So, after removing, it's rolled up the number of tests to `@n_tests`. After: `Array#each` iterates over array elements, **without removing** them. Using `Array#each` instead of `Array#shift`, means there's no need to rolled up the number of tests to `@n_tests`. for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Before: `Array#shift` **removes** and returns leading elements. So, after removing, it's rolled up the number of tests to `@n_tests`. After: `Array#each` iterates over array elements, **without removing** them. Using `Array#each` instead of `Array#shift`, means there's no need to rolled up the number of tests to `@n_tests`. for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss · 2024-04-06T22:28:20Z

Next:

Migrate the following methods from TestSuite to TestSuiteRunner
- #run_tests

The instance variables in TestSuiteRunner#run_tests are as following:

@n_tests
@passed

These instance variables can't refer from TestSuiteRunner because they were originally in TestSuite.

@n_tests was removed as it was unnecessary
@passed may be necessary

@passed may be necessary
- Find a way to make @passed writable from outside the TestSuite
- Or discover a more elegant approach

Task list:

Migrate the following methods from TestSuite to TestSuiteRunner
- #run_test
- #run_shutdown
- #handle_exception
Invoke TestSuiteRunner from TestSuite
Switching the backend with an option
Implement TestSuiteThreadRunner

Use `Array#each` instead of `Array#shift` in test-unitGH-240. Before test-unitGH-240 change: `Array#shift` **removes** and returns leading elements. So after removing, it cached `TestCase#passed?` or `TestSuite#passed?` results to `@passed`. After this change: `Array#each` iterates over array elements, **without removing** them. Using `Array#each` instead of `Array#shift`, means there's no need to cached `TestCase#passed?` or `TestSuite#passed?` results to `@passed`. Note: Since `@passed = true` was set in `TestSuite#initialize`, it should default to true when there are no tests. `Array#all?` returns true for an empty array below. The behavior remains unchanged, so there's no issue. ```ruby [].all? { true } # => true ``` for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Use `Array#each` instead of `Array#shift` in GH-240. Before GH-240 change: `Array#shift` **removes** and returns leading elements. So after removing, it cached `TestCase#passed?` or `TestSuite#passed?` results to `@passed`. After this change: `Array#each` iterates over array elements, **without removing** them. Using `Array#each` instead of `Array#shift`, means there's no need to cached `TestCase#passed?` or `TestSuite#passed?` results to `@passed`. Note: Since `@passed = true` was set in `TestSuite#initialize`, it should default to true when there are no tests. `Array#all?` returns true for an empty array below. The behavior remains unchanged, so there's no issue. ```ruby [].all? { true } # => true ``` for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Migrate the following methods from `TestSuite` to `TestSuiteRunner`. - `#run` - `#run_startup` - `#run_tests` - `#run_test` - `#run_shutdown` - `#handle_exception` Then invoke `TestSuiteRunner` from `TestSuite`. for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Migrate the following methods from `TestSuite` to `TestSuiteRunner`. - `#run` - `#run_startup` - `#run_tests` - `#run_test` - `#run_shutdown` - `#handle_exception` Then invoke `TestSuiteRunner` from `TestSuite`. for future parallelization support. Part of GH-235. --------- Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss · 2024-04-24T21:30:50Z

Next:

@passed may be necessary

Find a way to make @passed writable from outside the TestSuite

Or discover a more elegant approach

We discovered a more elegant approach GH-242, then @passed was removed as it was unnecessary.

Task list:

Migrate the following methods from TestSuite to TestSuiteRunner
- #run_test
- #run_shutdown
- #handle_exception

Invoke TestSuiteRunner from TestSuite

We implemented GH-243.

Switching the backend with an option

Backend is the TestSuiteRunner.
Currently, it operates sequentially, but we want to switch it to a Thread based or other parallel runner.

Currently, TestSuiteRunner is hardcoded to invoke within TestSuite#run.
It cannot be modified externally, so we need to implement a mechanism to enable external modification.

We aim to enable switching via command-line options.

We guess two main approaches:

Stop invoking the runner internally and instead accept it from the outside
Allow injection of the runner from the outside

We considered the following approaches:

Pass the Runner as a TestSuite#run argument
- Seems broken as the interface might change
```
# `TestSuiteRunner#run_test`
`test.run(result) do |event_name, *args|`
```
- Here, test is TestCase or TestSuite
- If it's TestCase, use this argument. If it's TestSuite, use that argument...
- Seems does not make sense
Pass the Runner as a TestSuite#initialize argument
- Switch the backend per TestSuite
- When grouping TestCase, TestSuite are nested
  - Executing a single TestSuite, multiple TestSuite needs to be run
- Seems does not make sense

Use this Runner for a system global

e.g.:

# `TestSuiteRunner.run`
class << self
  def run(test_suite, result, &progress_block)
  end
end

Switching the backend with an option

Task list:

Implement TestSuiteThreadRunner

We want to switch the backend (`TestSuiteRunner`) with an option. Currently, it operates sequentially, but we want to switch it to a `Thread` based or other parallel runner. `TestSuiteRunner` is hardcoded to invoke within `TestSuite#run`. It cannot be modified externally, so we need to implement a mechanism to enable external modification. We guess two main approaches: * Stop invoking the runner inside and instead accept it from the outside * Allow injection of the runner from the outside We considered the following approaches: 1. Pass the Runner as a `TestSuite#run` argument Seems broken as the interface might change ```ruby # `TestSuiteRunner#run_test` test.run(result) do |event_name, *args| ``` Here, `test` is a `TestCase` or `TestSuite` object. If it's `TestCase`, use this argument. If it's `TestSuite`, use that argument... Seems does not make sense 2. Pass the Runner as a `TestSuite#initialize` argument Switch the backend per `TestSuite`. When grouping `TestCase`, `TestSuite` are nested. Executing a single `TestSuite`, multiple `TestSuite` needs to be run. Seems does not make sense. 3. Use `TestSuiteRunner` for a system global It's uncommon to switch test runners per test suite, so it's better to handle it globally. e.g.: ```ruby # `TestSuiteRunner.run` class << self def run(test_suite, result, &progress_block) end end ``` Seems to make sense. We've decided to implement approach 3. We've inserted an abstracted layer. After, by replacing `TestSuiteRunner.run` from the outside. for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

We want to switch the backend (`TestSuiteRunner`) with an option. Currently, it operates sequentially, but we want to switch it to a `Thread` based or other parallel runner. `TestSuiteRunner` is hardcoded to invoke within `TestSuite#run`. It cannot be modified externally, so we need to implement a mechanism to enable external modification. We guess two main approaches: * Stop invoking the runner inside and instead accept it from the outside * Allow injection of the runner from the outside We considered the following approaches: 1. Pass the Runner as a `TestSuite#run` argument Seems broken as the interface might change ```ruby # `TestSuiteRunner#run_test` test.run(result) do |event_name, *args| ``` Here, `test` is a `TestCase` or `TestSuite` object. If it's `TestCase`, use this argument. If it's `TestSuite`, use that argument... Seems does not make sense 2. Pass the Runner as a `TestSuite#initialize` argument Switch the backend per `TestSuite`. When grouping `TestCase`, `TestSuite` are nested. Executing a single `TestSuite`, multiple `TestSuite` needs to be run. Seems does not make sense. 3. Use `TestSuiteRunner` for a system global It's uncommon to switch test runners per test suite, so it's better to handle it globally. e.g.: ```ruby # `TestSuiteRunner.run` class << self def run(test_suite, result, &progress_block) end end ``` Seems to make sense. We've decided to implement approach 3. We've inserted an abstracted layer. After, by replacing `TestSuiteRunner.run` from the outside. for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

`TestSuite` only invokes `TestSuiteRunner.run` (not `.run` of parallel runners such as `Thread` base). As a result, parallel runners don't need to store the default runner, so they use instance variables rather than class variables. for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

`TestSuite` only invokes `TestSuiteRunner.run` (not `.run` of parallel runners such as `Thread` base). As a result, parallel runners don't need to store the default runner, so they use instance variables rather than class variables. for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss · 2024-05-02T23:58:26Z

Next:

Switching the backend with an option

Use this Runner for a system global

We implemented GH-246.
We've inserted an abstracted layer.
After, by replacing TestSuiteRunner.run from the outside.

We implemented GH-247.
We can switch the backend globally below.

# e.g.: switch thread based runner (not available yet)
Test::Unit::TestSuiteRunner.default = Test::Unit::TestSuiteThreadRunner

# e.g.: switch sequential runner (default)
Test::Unit::TestSuiteRunner.default = Test::Unit::TestSuiteRunner

Switching the backend with an option

Task list:

Implement TestSuiteThreadRunner

Add support for switching the backend such as `Thread`. Please note that the `Thread` based runner is not yet available (raises `NameError`). Examples: * `ruby -I lib test/run-test.rb --parallel`: `Thread` * `ruby -I lib test/run-test.rb --parallel=thread`: `Thread` * `ruby -I lib test/run-test.rb --no-parallel`: Sequential * `ruby -I lib test/run-test.rb` (no --parallel option): Sequential Note: We considered the following other options. 1. `--runner` option: * Already exists but is used to switch the UI execution * UI execution and internal parallelism are independent * Seems does not make sense 2. `--executer` option * Does not exist * `TestSuiteRunner` has run methods (not execute) * Seems does not make sense for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

Add support for switching the backend such as `Thread`. Please note that the `Thread` based runner is not yet available (raises `NameError`). Examples: * `ruby -I lib test/run-test.rb --parallel`: `Thread` * `ruby -I lib test/run-test.rb --parallel=thread`: `Thread` * `ruby -I lib test/run-test.rb --no-parallel`: Sequential * `ruby -I lib test/run-test.rb` (no --parallel option): Sequential Note: We considered the following other options. 1. `--runner` option: * Already exists but is used to switch the UI execution * UI execution and internal parallelism are independent * Seems does not make sense 2. `--executer` option * Does not exist * `TestSuiteRunner` has run methods (not execute) * Seems does not make sense for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss · 2024-05-14T02:13:34Z

Next:

Switching the backend with an option

We implemented GH-250.

Add support for switching the backend such as Thread. Please note that
the Thread based runner is not yet available (raises NameError).

Examples:

ruby -I lib test/run-test.rb --parallel: Thread
ruby -I lib test/run-test.rb --parallel=thread: Thread
ruby -I lib test/run-test.rb --no-parallel: Sequential
ruby -I lib test/run-test.rb (no --parallel option): Sequential

Implement TestSuiteThreadRunner

tikkss · 2024-05-29T21:36:48Z

Next:

Implement TestSuiteThreadRunner

First, we implemented a sequential TestSuiteThreadRunner based on TestSuiteRunner. Then, we confirmed the following command that all tests passed.

$ ruby -I lib test/run-test.rb --parallel
-----------------------------------------------------------------
471 tests, 1600 assertions, 0 failures, 0 errors, 0 pendings, 6 omissions, 0 notifications 100% passed
-----------------------------------------------------------------
514.18 tests/s, 1755.41 assertions/s

Next, we implemented a parallel TestSuiteThreadRunner. Then, we confirmed the following command that some tests failed.

$ ruby -I lib test/run-test.rb --parallel
-----------------------------------------------------------------
471 tests, 1600 assertions, 6 failures, 14 errors, 0 pendings, 6 omissions, 0 notifications 96.9892% passed
-----------------------------------------------------------------
577.98 tests/s, 1963.40 assertions/s

We fixed them one by one. Adding the --stop-on-failure argument stops the execution at the first failure.

$ ruby -I lib test/run-test.rb --parallel --stop-on-failure
~/src/github.com/test-unit/test-unit/lib/test/unit/testresult.rb:99:in `throw': uncaught throw #<Object:0x000000010da08918> (UncaughtThrowError)

        throw @stop_tag
              ^^^^^^^^^
        from /Users/zzz/src/github.com/test-unit/test-unit/lib/test/unit/testresult.rb:99:in `stop'
        from /Users/zzz/src/github.com/test-unit/test-unit/lib/test/unit/autorunner.rb:597:in `block in attach_to_mediator'

The related parts are excerpted below.

# lib/test/unit/testresult.rb
      attr_accessor :stop_tag # lib/test/unit/testresult.rb:41

      # Constructs a new, empty TestResult.
      def initialize
        @run_count, @pass_count, @assertion_count = 0, 0, 0
        @summary_generators = []
        @problem_checkers = []
        @faults = []
        @stop_tag = nil # lib/test/unit/testresult.rb:49
        initialize_containers
      end

      def stop
        throw @stop_tag # lib/test/unit/testresult.rb:99
      end

We found sections setting @stop_tag using the following command.

$ grep -r 'stop_tag =' lib/
lib//test/unit/ui/testrunnermediator.rb:              result.stop_tag = stop_tag
lib//test/unit/testresult.rb:        @stop_tag = nil

The related parts are excerpted below.

# lib//test/unit/ui/testrunnermediator.rb
            catch do |stop_tag|
              result.stop_tag = stop_tag # lib//test/unit/ui/testrunnermediator.rb:40
              with_listener(result) do
                notify_listeners(RESET, @suite.size)
                notify_listeners(STARTED, result)

                run_suite(result)
              end
            end

Kernel.#catch and Kernel.#throw allow for global exit.

Why does test-unit use this?:

If everything works as expected, it runs to the end and returns with all tests passed
If it fails midway, it stops execution there. Running subsequent tests is meaningless
Used to return to a specific point, no matter how deep we are

We guess two approaches:

Parallelize TestRunnerMediator level
Create TestSuiteThreadRunner level sub-results
- Create a sub-result for each consumer
- Aggregate the sub-results into the main result after consumers.each(&:join)
- The tags of TestResult cannot be used as-is in the sub-results
  - Because if it returns to a higher process than run_tests, it can't be aggregated
- Sub-catch to temporarily stop the process
- After aggregating the sub-results, re-throw at the original result location.

Fix UncaughtThrowError
Switching the number of parallelism with an option

tikkss added a commit to tikkss/test-unit that referenced this issue Mar 29, 2024

testsuite: Extract run_tests method

4de2a9f

for future parallelization support. Part of test-unitGH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss mentioned this issue Mar 29, 2024

testsuite: Extract run_tests method #239

Merged

kou added a commit that referenced this issue Apr 2, 2024

testsuite: Extract run_tests method (#239)

f464770

for future parallelization support. Part of GH-235. Co-authored-by: Sutou Kouhei <kou@clear-code.com>

tikkss mentioned this issue Apr 6, 2024

testsuite: Remove unneeded instance variable #240

Merged

tikkss mentioned this issue Apr 12, 2024

testsuite: Remove unneeded cache #242

Merged

tikkss mentioned this issue Apr 13, 2024

Abstract #run from TestSuite to TestSuiteRunner #243

Merged

tikkss mentioned this issue Apr 27, 2024

Use TestSuiteRunner for a system global #246

Merged

tikkss mentioned this issue Apr 29, 2024

Make backend possible to switch globally #247

Merged

tikkss mentioned this issue May 12, 2024

Add --[no-]parallel option #250

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support parallelization #235

Support parallelization #235

tikkss commented Jan 8, 2024

kou commented Jan 15, 2024

tikkss commented Jan 27, 2024

tikkss commented Feb 7, 2024

tikkss commented Feb 18, 2024

tikkss commented Mar 7, 2024

tikkss commented Mar 15, 2024

tikkss commented Mar 27, 2024

tikkss commented Apr 2, 2024

tikkss commented Apr 6, 2024

tikkss commented Apr 24, 2024

tikkss commented May 2, 2024

tikkss commented May 14, 2024

tikkss commented May 29, 2024

Support parallelization #235

Support parallelization #235

Comments

tikkss commented Jan 8, 2024

kou commented Jan 15, 2024

tikkss commented Jan 27, 2024

tikkss commented Feb 7, 2024

tikkss commented Feb 18, 2024

tikkss commented Mar 7, 2024

tikkss commented Mar 15, 2024

tikkss commented Mar 27, 2024

tikkss commented Apr 2, 2024

tikkss commented Apr 6, 2024

tikkss commented Apr 24, 2024

tikkss commented May 2, 2024

tikkss commented May 14, 2024

tikkss commented May 29, 2024