config option to specify profile example count #747

greggroth · 2012-12-11T20:12:40Z

This adds a configuration option to specify the number of examples to include in the profile output.

addresses #726

dchelimsky · 2012-12-11T20:17:04Z

@greggroth almost all of the RSpec.configuration options are also available on the CLI. @myronmarston do you agree that this should be in both places?

soulcutter · 2012-12-11T20:23:09Z

I actually think this makes sense NOT having a command line option, the reasoning being that it depends on another option being set.

greggroth · 2012-12-11T20:27:03Z

@soulcutter rspec --profile enables the profile, so it would make sense to me that rspec --profile 2 would show the profile with 2 examples.

dchelimsky · 2012-12-11T20:30:00Z

@soulcutter I think it's more common to use --profile on the CLI because it's something you don't want to see every time (too much noise for TDD cycle w/ autotest/guard/etc).

How about, instead of adding a separate option, we allow --profile to accept a number:

rspec --profile 200

And in RSpec.configure we can support:

RSpec.configure {|c| c.profile_examples = 200}

soulcutter · 2012-12-11T20:31:15Z

@greggroth That's not too bad, I guess. I guess I was worried more about something like rspec --profile --profile-count 2 . I didn't think about this too long, clearly!

@dchelimsky You're right :)

greggroth · 2012-12-11T20:32:10Z

@dchelimsky I hadn't thought of it like that. Changing profile_examples from boolean to truthey/falsey should still work for whether it shows the profile and then I can pull the number from it (or if it's just true, make it 10). Do you like that more than two separate options?

dchelimsky · 2012-12-11T20:34:56Z

@greggroth yes.

greggroth · 2012-12-11T20:35:26Z

👍 I'm on it.

myronmarston · 2012-12-11T20:43:16Z

features/configuration/profile.feature

+      describe "something" do
+        it "foos take longest" do
+          container = Array.new
+          100_000.times { |n| container << n }


I'd prefer to see an explicit slowdown (e.g. using sleep) rather than an implicit one using a loop like this.

I wasn't sure if sleep would be a problem since it would slow down the test suite. I'll give it a shot.

Doing something 100,000 times slows down the test suite, too. sleep is just less obtuse and more explicit.

Just make sure to use a low value (e.g. sleep 0.1).

Not to mention the memory... good catch

… to an integer

greggroth · 2012-12-12T17:00:41Z

Can one of you check out the above two commits and let me know if I went about it the right way? Thanks!

greggroth · 2012-12-12T17:01:54Z

I'm also aware of the test failures and I'm looking into those now.

myronmarston · 2012-12-12T17:28:37Z

features/configuration/profile.feature

+        it "eleventh example" do
+          sleep 0.05
+          11.should == 11
+        end


I think it's a little confusing that you've got some examples named "foos/bazzes/bars take X" and others are named "nth example". It'd be nice to have a consistent naming scheme here. Maybe something like it "sleeps for X seconds (example #Y)"? (where X and Y would be filled in for the specific example)

myronmarston · 2012-12-12T17:47:11Z

This is looking good. One general question: should setting RSpec.configure.profile_examples = 20 always cause profiling output to be dumped, or only if the --profile flag was set? I can see a couple different ways people will want to use this:

Some will want to set a particular number to be dumped, but they won't want the output every time. They'll want the profile to be dumped if and only if the --profile flag has been passed.
Others will want it always to be dumped, and may interpret the profile_examples setting as turning on profiling.

For me personally, my workflow is more in line with the first: in my minute-by-minute TDD cycle, I run individual spec files one-off, and I don't want the profile output for that. Once I'm ready to commit and/or push, I run the entire suite, and I do want the profile output then. I kinda wonder if those semantics would be confusing, though, so it might be better to stick with the second.

@sferik -- I'm curious on your thoughts here, given that you were the original requestor of this feature.

* fix fragile spec which depends on line distance from where the spec is defined.

sferik · 2012-12-13T00:33:00Z

I'm typically in one of three modes when doing software development:

Normal Mode, where I'm writing code, writing specs, and refactoring.
Debugging Mode, where there is a specific problem I'm trying to reproduce and identify.
Optimization Mode, where I am benchmarking variations on existing code and design.

I use a different set of tools and flags in each of these modes. For example, I typically (Normal Mode) run rspec with the --fail-fast flag enabled, because I expect all specs to pass and I want to know as soon as one doesn't. Whereas, if I'm I debugging an issue that's causing many specs to fail, I won't use --fail-fast because I want to know if I'm making progress.

I also typically (Normal Mode) run specs with the --profile flag enabled to ensure that the new specs I'm writing are not unusually slow. I also use the --profile flag in Optimization Mode, especially when I'm trying to optimize the performance of my specs. However, after I optimize the slowest 10 specs as much as I can, they may still be the 10 slowest. I want the ability to see the 11th to 20th slowest specs, so I can try to optimize them after the first 10 are optimized.

This is a long way of saying, I'd like it to be easy for me to switch between these different modes.

In my opinion, setting RSpec.configure {|c| c.profile_examples = 200} should have the same behavior as setting the --profile=200 flag. I think it's confusing if the command-line flags behave differently from the configuration options. If you want to run an individual spec file without profiling, you should be able to pass the --no-profile flag on the command-line, which should override the configuration block.

greggroth · 2012-12-13T15:05:25Z

I agree with @sferik. I like the idea of a --no-profile flag that overrules the config option. I'll add that to this PR.

myronmarston · 2012-12-13T15:38:08Z

Sounds good.

greggroth · 2012-12-17T20:43:25Z

@myronmarston this is ready for review when you have a chance. Thanks!

myronmarston · 2012-12-18T06:53:06Z

lib/rspec/core/configuration.rb

@@ -506,6 +508,19 @@ def reporter
                      end
      end

+      # @api private
+      #
+      # Defaults `profile_examples` to 10 examples when `@profile_exmaples` is `true`.


s/exmaples/examples/

myronmarston · 2012-12-18T07:02:18Z

@greggroth -- thanks for letting me know. I left a couple more comments.

myronmarston · 2012-12-24T01:05:58Z

@greggroth -- do you want to take care of addressing my last couple comments before I merge? I can certainly take care of them but I always like to give the chance to contributors to address things first.

greggroth · 2012-12-24T03:01:59Z

I'll take care of this tomorrow. Sorry I've been slow -- holidays and all. ;-)

Thanks!

On Sunday, December 23, 2012 at 8:06 PM, Myron Marston wrote:

@greggroth (https://github.com/greggroth) -- do you want to take care of addressing my last couple comments before I merge? I can certainly take care of them but I always like to give the chance to contributors to address things first.

—
Reply to this email directly or view it on GitHub (#747 (comment)).

greggroth · 2012-12-24T15:24:12Z

Sorry for the delay. Those two commits should address all of your comments.

config option to specify profile example count

myronmarston · 2012-12-24T16:25:33Z

Thanks @greggroth -- this looks great.

And no need to apologize...you do this in your free time like the rest of us :).

add config option to specify profile example count

1a0accf

addresses rspec#726

myronmarston reviewed Dec 11, 2012
View reviewed changes

Greggory Rothmeier added 2 commits December 12, 2012 11:01

remove profile_example_count in favor of setting profile_examples…

db8d6e0

… to an integer

add CLI option for --profile

7aedb1d

myronmarston reviewed Dec 12, 2012
View reviewed changes

Greggory Rothmeier added 3 commits December 12, 2012 12:55

stub config optio for profile_examples

d8edaba

* fix fragile spec which depends on line distance from where the spec is defined.

improve CLI descriptoin of --profile

4ae3643

improve example spec example names

922b733

Greggory Rothmeier added 2 commits December 14, 2012 11:27

add --no-profile CLI flag

d702df7

add cucumber feature to document using --no-profile with spec_helper

eb40e25

myronmarston reviewed Dec 18, 2012
View reviewed changes

Greggory Rothmeier added 2 commits December 24, 2012 10:23

fix documentation

96d2e8f

improve clarity of conditional

569d55a

myronmarston added a commit that referenced this pull request Dec 24, 2012

Merge pull request #747 from greggroth/profile_config_options

2115f88

config option to specify profile example count

myronmarston merged commit 2115f88 into rspec:master Dec 24, 2012

myronmarston mentioned this pull request Dec 24, 2012

Allow --profile flag to take numeric argument #726

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

config option to specify profile example count #747

config option to specify profile example count #747

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

soulcutter commented Dec 11, 2012

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

soulcutter commented Dec 11, 2012

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

greggroth commented Dec 11, 2012

myronmarston Dec 11, 2012

greggroth Dec 12, 2012

myronmarston Dec 12, 2012

soulcutter Dec 12, 2012

greggroth commented Dec 12, 2012

greggroth commented Dec 12, 2012

myronmarston Dec 12, 2012

myronmarston commented Dec 12, 2012

sferik commented Dec 13, 2012

greggroth commented Dec 13, 2012

myronmarston commented Dec 13, 2012

greggroth commented Dec 17, 2012

myronmarston Dec 18, 2012

myronmarston commented Dec 18, 2012

myronmarston commented Dec 24, 2012

greggroth commented Dec 24, 2012

greggroth commented Dec 24, 2012

myronmarston commented Dec 24, 2012

config option to specify profile example count #747

config option to specify profile example count #747

Conversation

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

soulcutter commented Dec 11, 2012

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

soulcutter commented Dec 11, 2012

greggroth commented Dec 11, 2012

dchelimsky commented Dec 11, 2012

greggroth commented Dec 11, 2012

myronmarston Dec 11, 2012

Choose a reason for hiding this comment

greggroth Dec 12, 2012

Choose a reason for hiding this comment

myronmarston Dec 12, 2012

Choose a reason for hiding this comment

soulcutter Dec 12, 2012

Choose a reason for hiding this comment

greggroth commented Dec 12, 2012

greggroth commented Dec 12, 2012

myronmarston Dec 12, 2012

Choose a reason for hiding this comment

myronmarston commented Dec 12, 2012

sferik commented Dec 13, 2012

greggroth commented Dec 13, 2012

myronmarston commented Dec 13, 2012

greggroth commented Dec 17, 2012

myronmarston Dec 18, 2012

Choose a reason for hiding this comment

myronmarston commented Dec 18, 2012

myronmarston commented Dec 24, 2012

greggroth commented Dec 24, 2012

greggroth commented Dec 24, 2012

myronmarston commented Dec 24, 2012