test builder

[ono-max]

Screen.Recording.2021-06-21.at.3.42.17.PM.mov

What's this?

This is a test generator to make it easier to write tests.

Generate Tests

There is a test generator in debug.rb project to make it easier to write tests.

Quickstart

This section shows you how to create test file by test generator. For more advanced informations on creating tests, please take a look at gentest options. (You can also check by $bin/gentest -h)

1. Create a target file for debuggee.

Let's say, we created target.rb which is located in top level directory of debugger.

module Foo
  class Bar
    def self.a
      "hello"
    end
  end
  Bar.a
  bar = Bar.new
end

2. Run gentest as shown in the example below.

$ bin/gentest target.rb

3. Debugger will be executed. You can type any debug commands.

$ bin/gentest target.rb
[1, 9] in ~/workspace/debug/target.rb
=>    1| module Foo
      2|   class Bar
      3|     def self.a
      4|       "hello"
      5|     end
      6|   end
      7|   Bar.a
      8|   bar = Bar.new
      9| end
=>#0	<main> at ~/workspace/debug/target.rb:1
INTERNAL_INFO: {"location":"~/workspace/debug/target.rb:1","line":1}

(rdbg)s
 s
[1, 9] in ~/workspace/debug/target.rb
      1| module Foo
=>    2|   class Bar
      3|     def self.a
      4|       "hello"
      5|     end
      6|   end
      7|   Bar.a
      8|   bar = Bar.new
      9| end
=>#0	<module:Foo> at ~/workspace/debug/target.rb:2
  #1	<main> at ~/workspace/debug/target.rb:1
INTERNAL_INFO: {"location":"~/workspace/debug/target.rb:2","line":2}

(rdbg)n
 n
[1, 9] in ~/workspace/debug/target.rb
      1| module Foo
      2|   class Bar
=>    3|     def self.a
      4|       "hello"
      5|     end
      6|   end
      7|   Bar.a
      8|   bar = Bar.new
      9| end
=>#0	<class:Bar> at ~/workspace/debug/target.rb:3
  #1	<module:Foo> at ~/workspace/debug/target.rb:2
  #2	<main> at ~/workspace/debug/target.rb:1
INTERNAL_INFO: {"location":"~/workspace/debug/target.rb:3","line":3}

(rdbg)b 7
 b 7
#0  BP - Line  /Users/naotto/workspace/debug/target.rb:7 (line)
INTERNAL_INFO: {"location":"~/workspace/debug/target.rb:3","line":3}

(rdbg)c
 c
[2, 9] in ~/workspace/debug/target.rb
      2|   class Bar
      3|     def self.a
      4|       "hello"
      5|     end
      6|   end
=>    7|   Bar.a
      8|   bar = Bar.new
      9| end
=>#0	<module:Foo> at ~/workspace/debug/target.rb:7
  #1	<main> at ~/workspace/debug/target.rb:1

Stop by #0  BP - Line  /Users/naotto/workspace/debug/target.rb:7 (line)
INTERNAL_INFO: {"location":"~/workspace/debug/target.rb:7","line":7}

(rdbg)q!
 q!

4. The test file will be created as test/debug/foo_test.rb.

If the file already exists, only method will be added to it.

# frozen_string_literal: true

require_relative '../support/test_case'

module DEBUGGER__
  class FooTest < TestCase
    def program
      <<~RUBY
        1| module Foo
        1|   class Bar
        2|     def self.a
        3|       "hello"
        4|     end
        5|   end
        6|   Bar.a
        7|   bar = Bar.new
        8| end
      RUBY
    end
    
    def test_foo
      debug_code(program) do
        type 's'
        assert_line_num 2
        assert_line_text([
          /[1, 9] in .*/,
          /      1| module Foo/,
          /=>    2|   class Bar/,
          /      3|     def self.a/,
          /      4|       "hello"/,
          /      5|     end/,
          /      6|   end/,
          /      7|   Bar.a/,
          /      8|   bar = Bar.new/,
          /      9| end/,
          /=>#0	<module:Foo> at .*/,
          /  #1	<main> at .*/
        ])
        type 'n'
        assert_line_num 3
        assert_line_text([
          /[1, 9] in .*/,
          /      1| module Foo/,
          /      2|   class Bar/,
          /=>    3|     def self.a/,
          /      4|       "hello"/,
          /      5|     end/,
          /      6|   end/,
          /      7|   Bar.a/,
          /      8|   bar = Bar.new/,
          /      9| end/,
          /=>#0	<class:Bar> at .*/,
          /  #1	<module:Foo> at .*/,
          /  #2	<main> at .*/
        ])
        type 'b 7'
        assert_line_text(//)
        type 'c'
        assert_line_num 7
        assert_line_text([
          /[2, 9] in .*/,
          /      2|   class Bar/,
          /      3|     def self.a/,
          /      4|       "hello"/,
          /      5|     end/,
          /      6|   end/,
          /=>    7|   Bar.a/,
          /      8|   bar = Bar.new/,
          /      9| end/,
          /=>#0	<module:Foo> at .*/,
          /  #1	<main> at .*/,
          //,
          /Stop by #0  BP - Line  .*/
        ])
        type 'q!'
      end
    end
  end
end

gentest options

You can get more information about gentest here.

The default method name is test_foo and the class name is FooTest. The file name will be [Lowercase letters with "Test" removed from the class name]_test.rb.

# run without any options(test method name will be `test_foo`, class name will be `FooTest`, file name will be `foo_test.rb`)
$ bin/gentest target.rb
# specify the class name(test method name will be `test_foo`, class name will be `StepTest`, file name will be `step_test.rb`)
$ bin/gentest target.rb -c StepTest
# specify the method name(test method name will be `test_step`, class name will be `FooTest`, file name will be `foo_test.rb`)
$ bin/gentest target.rb -m test_step
# specify class name and method name(test method name will be `test_step`, class name will be `StepTest`, file name will be `step_test.rb`.)
$ bin/gentest target.rb -c StepTest -m test_step

[ko1]

I commented to ono-san via slack.

[st0012]

I know I'm not the target audience of this feature, but I want to provide some opinions on it.

While I appreciate the effort to make writing tests easier, I wonder if its maintenance cost would exceed the benefit it brings. The current test interface is already easy-to-use and copy&paste tests works pretty well, so the benefit is not significant to me.

If we breakdown the pros and cons of this feature:

Pros

• Writing tests may become easier to new devs.

Cons

• The code generation logic is hard to maintain.
  • Whenever we change the test framework, we also need to take care of this feature too.
  • Such complex feature should also have its end-to-end tests. This will be additional maintenance work too.
• Devs need to learn the command line interface.
• It adds another moving part that can cause tests to fail. When the tests it generates failed, the user wouldn't know if the test failed because of the test framework or because of the generation error.

(Personally, I wouldn't use this feature at all. Copying and paste existing tests feel easier to me. But I should also mention that I'm not a fan of Rails' code generator either. So you can argue that it's just my taste and I'd agree 😂)

[ko1]

It adds another moving part that can cause tests to fail. When the tests it generates failed, the user wouldn't know if the test failed because of the test framework or because of the generation error.

This tool is only test skeleton generator so the test author should maintain the test.