Core::Runner.run runs two specs with line number (args), the second line number setting is ignored

[zhimin]

Hi,

I try to use Core::Runner.run to execute some tests within my application.

require 'rspec'

err, out = StringIO.new, StringIO.new

args = ["add_spec.rb", "--line_number", "4"]
RSpec::Core::Runner.run(args, err, out)

args = ["minus_spec.rb", "--line_number", "10"]
RSpec::Core::Runner.run(args, err, out)

out.rewind; puts out.read

It executed first test OK, then ignore the line_number setting for the second one (still using the first setting), return 'All examples were filtered out'.

Add 1 + 2
Run options: include {:line_numbers=>[4]}
.

Finished in 0.00046 seconds
1 example, 0 failures
Run options: include {:line_numbers=>[4]}

All examples were filtered out

Is there anywhere to make Core::Runner.run always start fresh? Thanks.

Here are two simple specs (add_spec.rb, minus_spec.rb):

describe "Add" do

  it "can add single digit" do
     puts "Add 1 + 2"
    (1+2).should == 3
  end   

  it "can add double digit" do
     puts "Add 11 + 22"
     (11+22).should == 33
  end   
end

describe "Minus" do


   # No tests on line 4

  it "can minus single digit" do
    puts "3 - 2"
   (3-2).should == 1
  end   

  it "can minus double digit" do
     puts "22 - 11"
     (22-11).should == 11
  end
end     

Regards,
Zhimin

[dchelimsky]

RSpec is designed primarily to be used from the command line. Invoking the runner programmatically is supported, but we've never had a use case where someone was trying to run it twice in the same process. The issue is that there are a few global objects that are getting confused by trying to configure them twice.

Can I ask why you don't just run these two together?

spec add_spec.rb:4 minus_spec.rb:10

[zhimin]

David,

Thanks for quick response.

Our application is a functional testing IDE, RSpec is the core framework used in test scripts. One of most used feature is to run the test case where the cursor is, like below:

In other words, we pass line numbers to RSpec to run difference test cases. This works fine with RSpec 1.1.12. Now we are trying to upgrade to RSpec 2.

[dchelimsky]

@zhimin should work fine if you're only running time. Do you actually have a use case in which you run twice in the same process?

[zhimin]

@dchelimsky Yes, we have have a requirement to run difference behaviours in the same process.

I experienced "All examples were filtered out" in my application when upgrade to RSpec 2, when I ran second behaviour in another test file after executing once with --line_number argument, which exhibited in the code exampled I posted here.

I tried RSpec.reset. Also I could run another behaviour at different line number OK as long as it is the same test script file (the first one being executed).

[dchelimsky]

@zhimin after this fix you should be able to use RSpec.reset between each run. Please verify.

[zhimin]

@dchelimsky I applied the fix to RSpec lib, but the issue still remains. Here is my test case: https://gist.github.com/2722735

[dchelimsky]

Interesting - the output now shows that it is trying to filter on line 10 the 2nd time (which it wasn't before), but apparently it's not quite working yet (even though I thought I saw it work before). I'll follow up when I have a better fix :)

[dchelimsky]

@zhimin attempt #2. This should work - and you actually don't need to call RSpec.reset because it is already called for you at the end of RSpec::Core::Runner.run.

Please confirm.

[zhimin]

@dchelimsky Yes! It worked. Thanks heaps.

[dchelimsky]

Cool - I'll try to get this released this weekend either as 2.11 or at the very least as a patch release 2.10.1.

[dchelimsky]

@zhimin I haven't announced it yet, but I just released 2.10.1 with this fix. Announcement coming later today or tomorrow, but you are free to take advantage of it right away.

[zhimin]

@dchelimsky I updated to 2.10.1, works fine, thanks!

[sandro]

This change affects Specjour, which invokes the runner multiple times within the same process.

The issue occurs when using rspec-rails features that are included into the configuration object at require-time. Take rspec/rails/mocks for instance. Specjour loads the rails app by requiring spec_helper, which requires rspec/rails/mocks, which includes itself into the configuration object. Specjour then runs the next test off the stack.

The first test runs fine. When the configuration object is set to nil however, the next test that runs lazy-loads a new configuration object that is missing the mocks framework.

Specjour already does some monkey-patching, so I could monkey-patch my way out of this issue, but I figured I'd offer Specjour's perspective. Can we keep the configuration object around?

[dchelimsky]

Hey @sandro - it sounds like what you really need is for reset not to happen automatically (not just keep the Configuration around). Is that correct?

@zhimin if that is the case, are you OK with RSpec forcing reset on the user (not end-user - i.e. Testwise, in your case)?

[zhimin]

@dchelimsky Yeah, that's OK.

[dchelimsky]

OK. I've reopened this issue and will update the code to require the user to use RSpec.reset - otherwise the configuration stays put.

[sandro]

Sounds like a good solution. Thanks!

Sent from my iPhone

On Jun 9, 2012, at 5:22 AM, David Chelimskyreply@reply.github.com wrote:

Hey @sandro - it sounds like what you really need is for reset not to happen automatically (not just keep the Configuration around). Is that correct?

@zhimin if that is the case, are you OK with RSpec forcing reset on the user (not end-user - i.e. Testwise, in your case)?

---

Reply to this email directly or view it on GitHub:
#621 (comment)

[wallace]

Based on discussion in #703, this issue should be tagged as rspec-3.

[myronmarston]

There's a new PR and discussion for this in #1443. Closing.