Issues with rspec 3

[mikejarema]

I've incorporated rspec-todo into a spec suite for a current project of mine, it uses rspec 3. todo has no effect, namely a failure in the code still causes the example to fail.

Running rspec-todo's test suite with rspec 3 installed showed the following. It looks like rspec-todo makes some assumptions about how rspec is keeping tracking of each example under the hood which are no longer true.

My current workaround is to simply use rspec 2 until I or someone has a chance to look into this.

$ bundle exec rake

/Users/mike/.rbenv/versions/1.9.3-p448/bin/ruby -I/Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-core-3.1.7/lib:/Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-support-3.1.2/lib /Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-core-3.1.7/exe/rspec spec/rspec/todo_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
F.F.F.F.F.F.........F.F.F.F.F.F.F..F.

Failures:

  1) rspec-todo todo with unless opts are true behaves like error when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithUnlessOpts::AreTrue::BehavesLikeError::WhenExpectationFailed:0x007fd692527a18>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:134
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  2) rspec-todo todo with unless opts are false behaves like pending when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithUnlessOpts::AreFalse::BehavesLikePending::WhenExpectationFailed:0x007fd692551d90>
     Shared Example Group: "pending" called from ./spec/rspec/todo_spec.rb:120
     # ./spec/rspec/todo_spec.rb:10:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:10:in `block (3 levels) in <top (required)>'

  3) rspec-todo todo with if opts are false behaves like error when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithIfOpts::AreFalse::BehavesLikeError::WhenExpectationFailed:0x007fd69255b700>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:104
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  4) rspec-todo todo with if opts are true behaves like pending when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithIfOpts::AreTrue::BehavesLikePending::WhenExpectationFailed:0x007fd692562618>
     Shared Example Group: "pending" called from ./spec/rspec/todo_spec.rb:90
     # ./spec/rspec/todo_spec.rb:10:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:10:in `block (3 levels) in <top (required)>'

  5) rspec-todo todo without description behaves like pending when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithoutDescription::BehavesLikePending::WhenExpectationFailed:0x007fd6925645a8>
     Shared Example Group: "pending" called from ./spec/rspec/todo_spec.rb:75
     # ./spec/rspec/todo_spec.rb:10:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:10:in `block (3 levels) in <top (required)>'

  6) rspec-todo todo with description behaves like pending when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::Todo::WithDescription::BehavesLikePending::WhenExpectationFailed:0x007fd692572b30>
     Shared Example Group: "pending" called from ./spec/rspec/todo_spec.rb:61
     # ./spec/rspec/todo_spec.rb:10:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:10:in `block (3 levels) in <top (required)>'

  7) rspec-todo not todo with if opts are false behaves like error when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithIfOpts::AreFalse::BehavesLikeError::WhenExpectationFailed:0x007fd6925a8730>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:208
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  8) rspec-todo not todo with if opts are false behaves like error even if errors are specified works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithIfOpts::AreFalse::BehavesLikeError_2::EvenIfErrorsAreSpecified:0x007fd6925b1cb8>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:214
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  9) rspec-todo not todo with if opts are true behaves like error when expectation succeeded works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithIfOpts::AreTrue::BehavesLikeError::WhenExpectationSucceeded:0x007fd6925b7bb8>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:182
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  10) rspec-todo not todo with unless opts are true behaves like error when expectation failed works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithUnlessOpts::AreTrue::BehavesLikeError::WhenExpectationFailed:0x007fd6925c6690>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:250
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  11) rspec-todo not todo with unless opts are true behaves like error even if errors are specified works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithUnlessOpts::AreTrue::BehavesLikeError_2::EvenIfErrorsAreSpecified:0x007fd6925c80d0>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:256
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  12) rspec-todo not todo with unless opts are false behaves like error when expectation succeeded works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WithUnlessOpts::AreFalse::BehavesLikeError::WhenExpectationSucceeded:0x007fd6925d0848>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:224
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  13) rspec-todo not todo behaves like error when succeeded works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::BehavesLikeError::WhenSucceeded:0x007fd6925e3f88>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:144
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

  14) rspec-todo not todo when expectation raise exception behaves like error when errors are not specified works
     Failure/Error: example.metadata[:pending] = false
     NameError:
       undefined local variable or method `example' for #<RSpec::ExampleGroups::RspecTodo::NotTodo::WhenExpectationRaiseException::BehavesLikeError::WhenErrorsAreNotSpecified:0x007fd6925f70d8>
     Shared Example Group: "error" called from ./spec/rspec/todo_spec.rb:173
     # ./spec/rspec/todo_spec.rb:33:in `ensure in block (3 levels) in <top (required)>'
     # ./spec/rspec/todo_spec.rb:33:in `block (3 levels) in <top (required)>'

Deprecation Warnings:

RSpec::Core::Configuration#treat_symbols_as_metadata_keys_with_true_values= is deprecated, it is now set to true as default and setting it to false has no effect.

Using `should` from rspec-expectations' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` instead. Called from /Users/mike/workspace/rspec-todo/spec/rspec/todo_spec.rb:54:in `block (4 levels) in <top (required)>'.


If you need more of the backtrace for any of these deprecations to
identify where to make the necessary changes, you can configure
`config.raise_errors_for_deprecations!`, and it will turn the
deprecation warnings into errors, giving you the full backtrace.

2 deprecation warnings total

Finished in 0.02185 seconds (files took 0.14113 seconds to load)
37 examples, 14 failures

Failed examples:

rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo todo with unless opts are true behaves like error when expectation failed works
rspec ./spec/rspec/todo_spec.rb:5 # rspec-todo todo with unless opts are false behaves like pending when expectation failed works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo todo with if opts are false behaves like error when expectation failed works
rspec ./spec/rspec/todo_spec.rb:5 # rspec-todo todo with if opts are true behaves like pending when expectation failed works
rspec ./spec/rspec/todo_spec.rb:5 # rspec-todo todo without description behaves like pending when expectation failed works
rspec ./spec/rspec/todo_spec.rb:5 # rspec-todo todo with description behaves like pending when expectation failed works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with if opts are false behaves like error when expectation failed works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with if opts are false behaves like error even if errors are specified works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with if opts are true behaves like error when expectation succeeded works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with unless opts are true behaves like error when expectation failed works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with unless opts are true behaves like error even if errors are specified works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo with unless opts are false behaves like error when expectation succeeded works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo behaves like error when succeeded works
rspec ./spec/rspec/todo_spec.rb:26 # rspec-todo not todo when expectation raise exception behaves like error when errors are not specified works

Randomized with seed 53288

/Users/mike/.rbenv/versions/1.9.3-p448/bin/ruby -I/Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-core-3.1.7/lib:/Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-support-3.1.2/lib /Users/mike/.rbenv/versions/1.9.3-p448/lib/ruby/gems/1.9.1/gems/rspec-core-3.1.7/exe/rspec spec/rspec/todo_spec.rb failed

[mikejarema]

The above pull request is just a stopgap. Users of the gem should explicitly see the issue in the gem requirements rather than running into inexplicable failures the first time they try to use todo in their specs.