Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Core::Runner.run runs two specs with line number (args), the second line number setting is ignored #621

Closed
zhimin opened this Issue · 19 comments

5 participants

@zhimin

Hi,

I try to use Core::Runner.run to execute some tests within my application.

require 'rspec'

err, out = StringIO.new, StringIO.new

args = ["add_spec.rb", "--line_number", "4"]
RSpec::Core::Runner.run(args, err, out)

args = ["minus_spec.rb", "--line_number", "10"]
RSpec::Core::Runner.run(args, err, out)

out.rewind; puts out.read

It executed first test OK, then ignore the line_number setting for the second one (still using the first setting), return 'All examples were filtered out'.

Add 1 + 2
Run options: include {:line_numbers=>[4]}
.

Finished in 0.00046 seconds
1 example, 0 failures
Run options: include {:line_numbers=>[4]}

All examples were filtered out

Is there anywhere to make Core::Runner.run always start fresh? Thanks.

Here are two simple specs (add_spec.rb, minus_spec.rb):

describe "Add" do

  it "can add single digit" do
     puts "Add 1 + 2"
    (1+2).should == 3
  end   

  it "can add double digit" do
     puts "Add 11 + 22"
     (11+22).should == 33
  end   
end
describe "Minus" do


   # No tests on line 4

  it "can minus single digit" do
    puts "3 - 2"
   (3-2).should == 1
  end   

  it "can minus double digit" do
     puts "22 - 11"
     (22-11).should == 11
  end
end     

Regards,
Zhimin

@dchelimsky
Owner

RSpec is designed primarily to be used from the command line. Invoking the runner programmatically is supported, but we've never had a use case where someone was trying to run it twice in the same process. The issue is that there are a few global objects that are getting confused by trying to configure them twice.

Can I ask why you don't just run these two together?

spec add_spec.rb:4 minus_spec.rb:10
@zhimin

David,

Thanks for quick response.

Our application is a functional testing IDE, RSpec is the core framework used in test scripts. One of most used feature is to run the test case where the cursor is, like below:

Run test at line

In other words, we pass line numbers to RSpec to run difference test cases. This works fine with RSpec 1.1.12. Now we are trying to upgrade to RSpec 2.

@dchelimsky
Owner

@zhimin should work fine if you're only running time. Do you actually have a use case in which you run twice in the same process?

@zhimin

@dchelimsky Yes, we have have a requirement to run difference behaviours in the same process.

I experienced "All examples were filtered out" in my application when upgrade to RSpec 2, when I ran second behaviour in another test file after executing once with --line_number argument, which exhibited in the code exampled I posted here.

I tried RSpec.reset. Also I could run another behaviour at different line number OK as long as it is the same test script file (the first one being executed).

@dchelimsky
Owner

@zhimin after this fix you should be able to use RSpec.reset between each run. Please verify.

@zhimin

@dchelimsky I applied the fix to RSpec lib, but the issue still remains. Here is my test case: https://gist.github.com/2722735

@dchelimsky
Owner

Interesting - the output now shows that it is trying to filter on line 10 the 2nd time (which it wasn't before), but apparently it's not quite working yet (even though I thought I saw it work before). I'll follow up when I have a better fix :)

@dchelimsky dchelimsky reopened this
@dchelimsky
Owner

@zhimin attempt #2. This should work - and you actually don't need to call RSpec.reset because it is already called for you at the end of RSpec::Core::Runner.run.

Please confirm.

@dchelimsky dchelimsky referenced this issue from a commit
@dchelimsky dchelimsky Changelog for #621 c9b8681
@zhimin

@dchelimsky Yes! It worked. Thanks heaps.

@dchelimsky
Owner

Cool - I'll try to get this released this weekend either as 2.11 or at the very least as a patch release 2.10.1.

@dchelimsky dchelimsky referenced this issue from a commit
@dchelimsky dchelimsky Changelog for #621 fffc4c5
@dchelimsky
Owner

@zhimin I haven't announced it yet, but I just released 2.10.1 with this fix. Announcement coming later today or tomorrow, but you are free to take advantage of it right away.

@zhimin

@dchelimsky I updated to 2.10.1, works fine, thanks!

@sandro

This change affects Specjour, which invokes the runner multiple times within the same process.

The issue occurs when using rspec-rails features that are included into the configuration object at require-time. Take rspec/rails/mocks for instance. Specjour loads the rails app by requiring spec_helper, which requires rspec/rails/mocks, which includes itself into the configuration object. Specjour then runs the next test off the stack.

The first test runs fine. When the configuration object is set to nil however, the next test that runs lazy-loads a new configuration object that is missing the mocks framework.

Specjour already does some monkey-patching, so I could monkey-patch my way out of this issue, but I figured I'd offer Specjour's perspective. Can we keep the configuration object around?

@dchelimsky
Owner

Hey @sandro - it sounds like what you really need is for reset not to happen automatically (not just keep the Configuration around). Is that correct?

@zhimin if that is the case, are you OK with RSpec forcing reset on the user (not end-user - i.e. Testwise, in your case)?

@zhimin

@dchelimsky Yeah, that's OK.

@dchelimsky dchelimsky reopened this
@dchelimsky
Owner

OK. I've reopened this issue and will update the code to require the user to use RSpec.reset - otherwise the configuration stays put.

@sandro
@wallace

Based on discussion in #703, this issue should be tagged as rspec-3.

@myronmarston

There's a new PR and discussion for this in #1443. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.