RSpec doesn't display seed anymore

[firedev]

I haven't noticed when it started but now it is hard to pinpoint a failing test since RSpec does not show the seed it was running with. I remember before it was showing the seed value for the test run.

$ rspec ..
Run options: include ...
...
Failures:
...

Finished in 0.74256 seconds (files took 0.38212 seconds to load)
1 example, 1 failure

Failed examples:
...

[myronmarston]

The seed is not printed if it is not used. Are you running things in defined order (e.g. rspec --order defined or some equivalent config)? To run in random order, use rspec --order random (--order random can go in .rspec so you don't have to type it each time) and then the seed should be printed.

[firedev]

Thanks for the explanation. However I am not sure this is intended behaviour:

$ rspec  --order random
...
Randomized with seed 37209

$ rspec  --order random
...
Randomized with seed 37209

$ rspec  --order random
...
Randomized with seed 37209

Does not seem random enough to me

$ rspec --version
3.1.7

[firedev]

Thought it should be --order=random, but it still seeds with 37209 no matter what.

[myronmarston]

It must be something in your environment. I get 20 different seeds when running specs 20 different times:

➜ for i in {1..20}; do bin/rspec --order random | grep seed | tail -n 1; done
Randomized with seed 29520
Randomized with seed 145
Randomized with seed 33341
Randomized with seed 33375
Randomized with seed 61066
Randomized with seed 19215
Randomized with seed 15332
Randomized with seed 14487
Randomized with seed 25750
Randomized with seed 35843
Randomized with seed 62356
Randomized with seed 24741
Randomized with seed 52492
Randomized with seed 51285
Randomized with seed 60808
Randomized with seed 19049
Randomized with seed 40701
Randomized with seed 27230
Randomized with seed 11921
Randomized with seed 59059

While RSpec will default a random seed, it lets you set it so you can troubleshoot problems from a particular seed. Maybe you set it somewhere for a prior troubleshooting session and left that in place? Check --seed in .rspec, ~/.rspec and .rspec-local, and config.seed = 37209 in spec/spec_helper.rb (or whatever file you use for RSpec config). If that doesn't yield any leads, then I'm stumped. If you push up your repo so we can clone and run it locally we can take a look but w/o that I don't think there's any way for us to guess what's causing it.

[myronmarston]

Actually, here's a little monkey patch that should help you track down what is setting the seed. Put this in spec/track_seed.rb:

module TrackSeedSetter
  def seed=(value)
    puts "seed= called from:"
    puts caller.join("\n")
    puts
    super
  end
end

RSpec::Core::Configuration.prepend TrackSeedSetter

Then run rspec --require track_seed.

[firedev]

That's funny:

$  for i in {1..20}; do bin/rspec --order random | grep seed | tail -n 1; done
Randomized with seed 546
Randomized with seed 546
Randomized with seed 546
Randomized with seed 546
Randomized with seed 546

$ rspec --require track_seed
No examples found.

Finished in 0.00084 seconds (files took 0.02758 seconds to load)
0 examples, 0 failures

Randomized with seed 546

[myronmarston]

Odd. Are you using a pre-loader like zeus or spring? It's possible that with one of those running, the seed gets setup during the initial setup, and then each time you run rspec it runs with the seed you already have set.

You can see how RSpec sets the seed by default:

https://github.com/rspec/rspec-core/blob/v3.1.7/lib/rspec/core/ordering.rb#L109

To confirm that that works on your machine, try running this:

$ for 1 in {1..20}; do ruby -e "puts rand(0xFFFF)"; done

It should print a different seed each time.

[firedev]

Spring! Good one, thanks. Do you think RSpec can be amended so it works under spring? Change the randomizer to lambda or something?

$ rspec
Randomized with seed 546

$ spring stop
Spring stopped.

$ rspec
Randomized with seed 23145

[myronmarston]

We've exposed config.seed = so that anyone (either a tool like spring or a user) can set the seed anytime they want.

RSpec is blissfully unaware of Spring and we'd like to keep it that way. We've exposed public APIs that spring or other alternate runners can use to initialize and clear config and other state when they want. Alternate runners are in control of when that happens, not RSpec, so the fix would need to be there. In the meantime, you can set config.seed yourself.