Core::Runner.run runs two specs with line number (args), the second line number setting is ignored

[zhimin]

Hi,

I try to use Core::Runner.run to execute some tests within my application.

require 'rspec'

err, out = StringIO.new, StringIO.new

args = ["add_spec.rb", "--line_number", "4"]
RSpec::Core::Runner.run(args, err, out)

args = ["minus_spec.rb", "--line_number", "10"]
RSpec::Core::Runner.run(args, err, out)

out.rewind; puts out.read

It executed first test OK, then ignore the line_number setting for the second one (still using the first setting), return 'All examples were filtered out'.

Add 1 + 2
Run options: include {:line_numbers=>[4]}
.

Finished in 0.00046 seconds
1 example, 0 failures
Run options: include {:line_numbers=>[4]}

All examples were filtered out

Is there anywhere to make Core::Runner.run always start fresh? Thanks.

Here are two simple specs (add_spec.rb, minus_spec.rb):

describe "Add" do

  it "can add single digit" do
     puts "Add 1 + 2"
    (1+2).should == 3
  end   

  it "can add double digit" do
     puts "Add 11 + 22"
     (11+22).should == 33
  end   
end

describe "Minus" do


   # No tests on line 4

  it "can minus single digit" do
    puts "3 - 2"
   (3-2).should == 1
  end   

  it "can minus double digit" do
     puts "22 - 11"
     (22-11).should == 11
  end
end     

Regards,
Zhimin

[dchelimsky]

RSpec is designed primarily to be used from the command line. Invoking the runner programmatically is supported, but we've never had a use case where someone was trying to run it twice in the same process. The issue is that there are a few global objects that are getting confused by trying to configure them twice.

Can I ask why you don't just run these two together?

spec add_spec.rb:4 minus_spec.rb:10

[zhimin]

David,

Thanks for quick response.

Our application is a functional testing IDE, RSpec is the core framework used in test scripts. One of most used feature is to run the test case where the cursor is, like below:

In other words, we pass line numbers to RSpec to run difference test cases. This works fine with RSpec 1.1.12. Now we are trying to upgrade to RSpec 2.

[dchelimsky]

@zhimin should work fine if you're only running time. Do you actually have a use case in which you run twice in the same process?

[zhimin]

@dchelimsky Yes, we have have a requirement to run difference behaviours in the same process.

I experienced "All examples were filtered out" in my application when upgrade to RSpec 2, when I ran second behaviour in another test file after executing once with --line_number argument, which exhibited in the code exampled I posted here.

I tried RSpec.reset. Also I could run another behaviour at different line number OK as long as it is the same test script file (the first one being executed).

[dchelimsky]

@zhimin after this fix you should be able to use RSpec.reset between each run. Please verify.

[zhimin]

@dchelimsky I applied the fix to RSpec lib, but the issue still remains. Here is my test case: https://gist.github.com/2722735

[dchelimsky]

Interesting - the output now shows that it is trying to filter on line 10 the 2nd time (which it wasn't before), but apparently it's not quite working yet (even though I thought I saw it work before). I'll follow up when I have a better fix :)

[dchelimsky]

@zhimin attempt #2. This should work - and you actually don't need to call RSpec.reset because it is already called for you at the end of RSpec::Core::Runner.run.

Please confirm.

[zhimin]

@dchelimsky Yes! It worked. Thanks heaps.

[dchelimsky]

Cool - I'll try to get this released this weekend either as 2.11 or at the very least as a patch release 2.10.1.