New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document config options #723
Merged
myronmarston
merged 10 commits into
rspec:master
from
charlietanksley:document-config-options
Jan 1, 2013
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
b15a03b
add feature to document failure_exit_code option
charlietanksley a3ce764
add feature to document profile_examples option
charlietanksley ce715c7
add feature to document run_all_when_everything_filtered
charlietanksley f0c93ed
add feature to document pattern option
charlietanksley 0e3b20d
add feature to document treat_symbols_as_metadata_keys_with_true_values
charlietanksley a52dad6
use aruba exit code matcher
charlietanksley 352bd79
add feature to document drb and drb_port options
charlietanksley e10fe0a
add feature to document show_failures_in_pending_blocks
charlietanksley 31edba4
add feature to document output_stream
charlietanksley 4385437
link to documentation for order and seed options
charlietanksley File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
Feature: drb and drb port | ||
|
||
Using the drb option and the drb_port option, users can run examples over DRb and set the port for that server. | ||
|
||
Scenario: DRb is off by default | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "DRb status" do | ||
it "is false" do | ||
RSpec.configuration.drb?.should be_false | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the examples should all pass | ||
|
||
Scenario: when DRb is turned on | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
RSpec.configure do |c| | ||
c.drb = true | ||
end | ||
|
||
describe "DRb status" do | ||
it "is true" do | ||
RSpec.configuration.drb?.should be_true | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the examples should all pass | ||
|
||
Scenario: the default DRb port | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
RSpec.configure do |c| | ||
c.drb = true | ||
end | ||
|
||
describe "DRb port" do | ||
it "is not set (is nil) by default" do | ||
RSpec.configuration.drb_port.should be_nil | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the examples should all pass | ||
|
||
Scenario: setting the DRb port | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
RSpec.configure do |c| | ||
c.drb = true | ||
c.drb_port = 42 | ||
end | ||
|
||
describe "DRb port" do | ||
it "is what I set it to" do | ||
RSpec.configuration.drb_port.should == 42 | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the examples should all pass | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
Feature: failure exit code | ||
|
||
Use the feature_exit_code option to set a custom exit code when RSpec fails. | ||
|
||
RSpec.configure { |c| c.failure_exit_code = 42 } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure { |c| c.failure_exit_code = 42 } | ||
""" | ||
|
||
Scenario: a failing spec with the default exit code | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "something" do | ||
it "fails" do | ||
fail | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the exit status should be 1 | ||
|
||
Scenario: a failing spec with a custom exit code | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require 'spec_helper' | ||
describe "something" do | ||
it "fails" do | ||
fail | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the exit status should be 42 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
Feature: set the order and/or seed | ||
|
||
You can set the order to run specs and specify a seed if you are running the specs using the 'random' ordering. See <a href='../command_line/order.feature'>the documentation on the --order command line option</a> for more on this. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
Feature: output_stream | ||
|
||
Define a custom output stream (default `$stdout`). Aliases: `:output`, `:out`. | ||
|
||
RSpec.configure { |c| c.output_stream = File.open('saved_output', 'w') } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure {|c| c.output_stream = File.open('saved_output', 'w') } | ||
""" | ||
|
||
Scenario: redirecting output | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require 'spec_helper' | ||
describe "an example" do | ||
it "passes" do | ||
true | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the file "saved_output" should contain "1 example, 0 failures" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
Feature: pattern | ||
|
||
Use the pattern option to tell RSpec to look for specs in files that match a pattern other than "**/*_spec.rb". | ||
|
||
Background: | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "two specs here" do | ||
it "passes" do | ||
end | ||
|
||
it "passes too" do | ||
end | ||
end | ||
""" | ||
And a file named "spec/example_test.rb" with: | ||
"""ruby | ||
describe "only one spec" do | ||
it "passes" do | ||
end | ||
end | ||
""" | ||
|
||
Scenario: by default, RSpec runs files that match "**/*_spec.rb" | ||
When I run `rspec` | ||
Then the output should contain "2 examples, 0 failures" | ||
|
||
Scenario: the --pattern flag makes RSpec run files matching the specified pattern and ignore the default pattern | ||
When I run `rspec -P "**/*_test.rb"` | ||
Then the output should contain "1 example, 0 failures" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
Feature: profile examples | ||
|
||
Use the profile_examples option to tell RSpec to report the times | ||
for the 10 slowest examples. | ||
|
||
RSpec.configure { |c| c.profile_examples = true } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure { |c| c.profile_examples = true } | ||
""" | ||
|
||
Scenario: profile_examples defaults to false | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "something" do | ||
it "passes" do | ||
end | ||
|
||
it "also passes" do | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should not contain "Top 2 slowest examples" | ||
|
||
Scenario: profile_examples reports on the slowest features | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require 'spec_helper' | ||
describe "something" do | ||
it "passes" do | ||
end | ||
|
||
it "also passes" do | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should contain "Top 2 slowest examples" |
60 changes: 60 additions & 0 deletions
60
features/configuration/run_all_when_everything_filtered.feature
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
Feature: run all when everything filtered | ||
|
||
Use the run_all_when_everything_filtered option to tell RSpec to run | ||
all the specs in the case where you try to run a filtered list of | ||
specs but no specs match that filter. | ||
|
||
RSpec.configure { |c| c.run_all_when_everything_filtered = true } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure {|c| c.run_all_when_everything_filtered = true} | ||
""" | ||
|
||
Scenario: by default, no specs are run if they are all filtered out | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "examples" do | ||
it "with no tag" do | ||
end | ||
|
||
it "with no tag as well" do | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb --tag some_tag` | ||
Then the output should contain "0 examples, 0 failures" | ||
|
||
Scenario: when the run_all_when_everything_filtered option is turned on, if there are any matches for the filtering tag, only those features are run | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require "spec_helper" | ||
describe "examples" do | ||
it "with no tag", :some_tag => true do | ||
end | ||
|
||
it "with no tag as well" do | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb --tag some_tag` | ||
Then the output should contain "1 example, 0 failures" | ||
And the output should contain "Run options: include {:some_tag=>true}" | ||
|
||
Scenario: when the run_all_when_everything_filtered option is turned on, all the specs are run when the tag has no matches | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require "spec_helper" | ||
describe "examples" do | ||
it "with no tag" do | ||
end | ||
|
||
it "with no tag as well" do | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb --tag some_tag` | ||
Then the output should contain "2 examples, 0 failures" | ||
And the output should contain "All examples were filtered out; ignoring {:some_tag=>true}" | ||
|
61 changes: 61 additions & 0 deletions
61
features/configuration/show_failures_in_pending_blocks.feature
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
Feature: show_failures_in_pending_blocks | ||
|
||
Use the show_failures_in_pending_blocks option to run the code in pending blocks while keeping the tests pending. | ||
|
||
RSpec.configure { |c| c.show_failures_in_pending_blocks = true } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure {|c| c.show_failures_in_pending_blocks = true} | ||
""" | ||
|
||
Scenario: by default, code in pending examples is not exercised | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "fails" do | ||
pending "code will not be exercised" do | ||
fail | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should not contain "Failure/Error: pending { fail }" | ||
|
||
Scenario: by default, code in pending blocks inside examples is not exercised | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "fails" do | ||
it "code will not be exercised" do | ||
pending { fail } | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should not contain "Failure/Error: pending { fail }" | ||
|
||
Scenario: when turned on, pending code blocks inside examples are exercised | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require "spec_helper" | ||
describe "fails" do | ||
it "code will be exercised" do | ||
pending { fail } | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should contain "Failure/Error: pending { fail }" | ||
|
||
Scenario: when turned on, code inside pending examples is not exercised | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require "spec_helper" | ||
describe "fails" do | ||
pending "code will not be exercised" do | ||
fail | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb` | ||
Then the output should not contain "Failure/Error: pending { fail }" |
52 changes: 52 additions & 0 deletions
52
features/configuration/treat_symbols_as_metadata_keys_with_true_values.feature
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
Feature: treat symbols as metadata keys with true values | ||
|
||
Use the treat_symbols_as_metadata_keys_with_true_values option to tell RSpec that :key is shorthand for :key => true. | ||
|
||
RSpec.configure { |c| c.treat_symbols_as_metadata_keys_with_true_values = true } | ||
|
||
Background: | ||
Given a file named "spec/spec_helper.rb" with: | ||
"""ruby | ||
RSpec.configure { |c| c.treat_symbols_as_metadata_keys_with_true_values = true } | ||
""" | ||
|
||
Scenario: by default, symbols without values are ignored and the specs are filtered out | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
describe "failed filtering" do | ||
it "this will be filted out", :some_tag do | ||
true | ||
end | ||
|
||
it "so will this" do | ||
false | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb --tag some_tag` | ||
Then the output should contain "0 examples, 0 failures" | ||
And the output should contain "All examples were filtered out" | ||
|
||
Scenario: when treat_symbols_as_metadata_keys_with_true_values is true, specs can be tagged with only a symbol | ||
Given a file named "spec/example_spec.rb" with: | ||
"""ruby | ||
require "spec_helper" | ||
describe "run me", :some_tag do | ||
it "runs" do | ||
true | ||
end | ||
end | ||
|
||
describe "run one of these" do | ||
it "run this one", :some_tag do | ||
true | ||
end | ||
|
||
it "but not me" do | ||
false | ||
end | ||
end | ||
""" | ||
When I run `rspec spec/example_spec.rb --tag some_tag` | ||
Then the output should contain "2 examples, 0 failures" | ||
And the output should contain "Run options: include {:some_tag=>true}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm glad we're getting some docs in place for the DRb stuff, but I'm not sure that these scenarios have much documentation value...they don't demonstrate what the DRb options do; they just show that when you set them, the configuration object stores the values you set.
This is going to sound funny given that I'm the lead maintainer for rspec now...but I'm not even sure how to use the DRb options (I've never used the options). I know spork uses it but it'd be nice to document how to use it w/o spork. Not sure how feasible it is to demonstrate that with a scenario here, but if it was feasible, it would be a huge improvement, I think. Or, even if it's not feasible, maybe some additional text at the top explaining why you would want to use the options and how you would use it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a common use case outside of tools like spork? I also am not too familiar with them beyond that, if so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that this doesn't provide tons of documentation value. @myronmarston, you're right that it merely shows that the DRb option can be turned on and that you can set the port (and that there is no port set by default).
I haven't the foggiest clue how to document this any further. If you guys don't know how it is used, I sure don't! So I'd vote that if you think that this documentation adds nothing, we just scrap it. I think the other documentation here is helpful, so I'd like to see it added even if we drop the DRb documentation. Do feel free to cut these features. Whatever you see fit to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll definitely merge the rest of the documentation stuff you wrote up (it's really great) but I'm hoping @dchelimsky can weigh in on the DRb options first. @dchelimsky -- if you get the chance, it'd be nice to get a description of how to use these options, if for no other reason than to provide a "here's how to use them guide" for me so if/when I refactor any code involving the DRb stuff I can verify I haven't broken it. If it's feasible to turn that into a cuke we can do that as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool. If I know how to document the DRb options I'm happy to do it. I'll watch for updates :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Basically, you add
--drb
to the command line to run via spork. That's it. AKAIK,drb
is only ever used w/ spork anddrb_port
is never used (well, it's used internally, but never by a user).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I knew spork was the primary library that relies on rspec's DRb support. Spork's usage of this should be relying upon public APIs that another bit of code or library could use as well, though, right? Can we document how the rspec DRb protocol works (as a standalone thing, without referencing spork as the only way to use it?). I'm thinking something along the lines of "start rspec with the drb option, then send it commands by doing XYZ".
Maybe it's not possible to distill how spork uses rspec down into something easily documentable, but it would be nice to do so.