Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent block-only matchers from being used with value expectation target #1125

Conversation

@pirj
Copy link
Member

@pirj pirj commented Aug 5, 2019

The purpose of this (rather experimental) change is to restrict the usage of:

subject { -> { do_something } }
it { is_expected.to change(something).to(new_value) }

This syntax was referenced to as:

pretty obtuse and not something I’d recommend, generally.
#805 (comment)

We should mention they’re not supported.
#805 (comment)

Problems with this approach

Specs fail quite badly mostly due to the use of this syntax in RSpec Expectations' own specs:

RSpec.describe RSpec::Matchers::BuiltIn::ChangeToValue do
  k = 0
  before { k = 0 }
  it_behaves_like "an RSpec matcher", :valid_value => lambda { k = 2 },
                                      :invalid_value => lambda { k = 3 },
                                      :disallows_negation => true do
    let(:matcher) { change { k }.to(2) }
  end
end

Would you accept this if I change those specs to something more canonical? E.g. like:

RSpec.describe RSpec::Matchers::BuiltIn::ChangeToValue do
  let(:k) { 0 }
  it_behaves_like "an RSpec matcher" do
    let(:valid_value) { k += 1 }
    let(:invalid_value) { }

    def matcher
      change { k }.from(0)
    end
  end
end

The above change didn't work from the first time, but I'll keep trying if that's the right direction.

Related Links

#526
#530
#536
#1066

https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://www.reddit.com/r/ruby/comments/cejl3q/call_for_discussion_rspec_implicit_block
rubocop/rspec-style-guide#76
rubocop/rubocop-rspec#789

Example fix for matcher libraries that are forced to keep the interface to provide the syntax for their users rodjek/rspec-puppet#767

def supports_block_expectations?
true
end

def supports_value_expectations?
Copy link
Member Author

@pirj pirj Aug 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible to tell if a matcher supports block expectations, but it's not possible to detect that it doesn't support value expectations, and will be sending call on a value target.

Loading

Copy link
Member

@JonRowe JonRowe Aug 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable!

Loading

lib/rspec/matchers/built_in/change.rb Outdated Show resolved Hide resolved
Loading
Copy link
Member Author

@pirj pirj left a comment

@JonRowe Appreciate if you take a look and give some major direction on how to improve this.

Loading

spec/rspec/matchers/built_in/compound_spec.rb Outdated Show resolved Hide resolved
Loading
spec/support/shared_examples/block_matcher.rb Outdated Show resolved Hide resolved
Loading
spec/support/shared_examples/block_matcher.rb Outdated Show resolved Hide resolved
Loading
@pirj
Copy link
Member Author

@pirj pirj commented Aug 22, 2019

I'm still up to adjust this the way you see fit but need some guidance on specs.

Loading

spec/spec_helper.rb Outdated Show resolved Hide resolved
Loading
Copy link
Member

@JonRowe JonRowe left a comment

Sorry this took so long to get back to you!

Loading

# 1) output.to_stderr matcher behaves like an RSpec block-only matcher preserves the symmetric property of `==`
# Failure/Error: raise "Warnings were generated: #{output}" if has_output?
# RuntimeError:
# Warnings were generated: foo
Copy link
Member

@JonRowe JonRowe Aug 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is because warnings are generated by the specs, it should be possible to rewrite the relevant specs warning free...

Loading

Copy link
Member Author

@pirj pirj Aug 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, valid values were defined as e.g.:

  it_behaves_like "an RSpec matcher", :valid_value => lambda { k += 1 },

and that were supposed to check that

expect { k += 1 }.to change { k }

The matcher here is initialized by change { k }, and the target is a block.

So the whole point of those two expectations is to make sure that == on both the lambda and the matcher return false when compared with each other.

After this change the target block is defined using a method, and the reason for warnings to appear is that those output/throw_symbol/raise_error targets are actually get called when target is compared, since they are not a block.

Do you think that writing those examples as the following will achieve the same goal?

    expect(matcher).not_to eq(-> { valid_block })
    expect(-> { valid_block }).not_to eq(matcher)

Loading

spec/support/shared_examples/block_matcher.rb Outdated Show resolved Hide resolved
Loading
spec/support/shared_examples/block_matcher.rb Outdated Show resolved Hide resolved
Loading
@pirj pirj force-pushed the prevent-block-matchers-from-being-used-with-value-expectation-target branch 2 times, most recently from 1da2e36 to 9970f0e Aug 24, 2019
@pirj
Copy link
Member Author

@pirj pirj commented Aug 25, 2019

@JonRowe Builds are green, all specs are in place.
It took a while to work around Ruby 1.8.7 block signature strictness, moved stuff around.
No rush, please take another look when you have a moment.

Loading

spec/support/shared_examples/block_matcher.rb Outdated Show resolved Hide resolved
Loading
spec/support/shared_examples/block_matcher.rb Show resolved Hide resolved
Loading
@pirj pirj force-pushed the prevent-block-matchers-from-being-used-with-value-expectation-target branch from 43520c6 to 0418c67 Aug 29, 2019
@pirj
Copy link
Member Author

@pirj pirj commented Aug 29, 2019

Squashed commits.

Loading

@pirj
Copy link
Member Author

@pirj pirj commented Sep 11, 2019

@JonRowe Do you think this needs some adjustment?

Loading

@pirj pirj self-assigned this Sep 11, 2019
@JonRowe
Copy link
Member

@JonRowe JonRowe commented Sep 11, 2019

@JonRowe Do you think this needs some adjustment?

I promise this is high on my list for a re-review!

Loading

@JonRowe
Copy link
Member

@JonRowe JonRowe commented Sep 12, 2019

Hey @pirj great work, looks good. Do you want to merge and then write up a change log?

Loading

@pirj
Copy link
Member Author

@pirj pirj commented Sep 14, 2019

Thanks!

I'm holding off from merging this, found a problem with rspec-collection_matchers, still getting to the bottom of it.

     NoMethodError:
       undefined method `supports_value_expectations?' for #<#<Class:0x00007fb20c0a7128>:0x00007fb20c0a7088>
     # ./lib/rspec/collection_matchers/have.rb:71:in `determine_collection'
     # ./lib/rspec/collection_matchers/have.rb:41:in `matches?'
     # ./spec/rspec/collection_matchers/have_spec.rb:532:in `block (4 levels) in <top (required)>'

Loading

@pirj pirj force-pushed the prevent-block-matchers-from-being-used-with-value-expectation-target branch 2 times, most recently from 05edeb7 to 2e3a7f5 Sep 14, 2019
…rget

https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Ruby 1.8-specific workarounds:

    multiple values for a block parameter (0 for 1)

If a block expects an argument, it ought to be provided an argument in 1.8
@pirj pirj force-pushed the prevent-block-matchers-from-being-used-with-value-expectation-target branch from 1779c57 to 771da6d Sep 14, 2019
matcher.supports_block_expectations?
rescue NoMethodError
false
if matcher.respond_to?(:supports_block_expectations?)
Copy link
Member Author

@pirj pirj Sep 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is due to have matcher from rspec-collection_matchers that is sensitive to methods called on it, i.e. if we call supports_value_expectations? on have(3).players, it starts to think that it should match supports_value_expectations? association, not players.

The problem is with supports_value_expectations?, changing supports_block_expectations? for consistency.

Loading

@pirj pirj merged commit 110243e into rspec:master Sep 14, 2019
2 checks passed
Loading
@pirj pirj deleted the prevent-block-matchers-from-being-used-with-value-expectation-target branch Sep 14, 2019
JonRowe added a commit that referenced this issue Oct 2, 2019
…used-with-value-expectation-target

Prevent block-only matchers from being used with value expectation target
@mvz
Copy link
Contributor

@mvz mvz commented Oct 3, 2019

Just found this change and it also breaks the following code. Not sure if that was intended:

      call = lambda do
        Reek::CLI::Silencer.silently do
          described_class.new ['--foo']
        end
      end
      expect(call).to raise_error(SystemExit) do |error|
        expect(error.status).to eq Reek::CLI::Status::DEFAULT_ERROR_EXIT_CODE
      end

Loading

@mvz
Copy link
Contributor

@mvz mvz commented Oct 3, 2019

@pirj Yes, both are fine options, although I would prefer the second one rather than chaining multi-line blocks. I was just surprised my existing specs started failing.

I fixed my code by changing expect(call) to expect(&call).

Loading

@JonRowe
Copy link
Member

@JonRowe JonRowe commented Oct 7, 2019

This PR has been reverted for now.

Loading

JonRowe added a commit that referenced this issue Oct 7, 2019
…m-being-used-with-value-expectation-target"

This reverts commit f5bf9d8.
@JonRowe
Copy link
Member

@JonRowe JonRowe commented Oct 7, 2019

I would like to include this again, but we need to address the following situations:

  • Transitionally allow blocks to be in-explicitly passed in for future deprecation and warning. This is quite easy to achieve but has knock on effects for the next thing I'd like to address.
  • Still allow passed in blocks to be introspected as values.

Loading

JonRowe added a commit that referenced this issue Oct 7, 2019
…m-being-used-with-value-expectation-target"

This reverts commit f5bf9d8.
@pirj
Copy link
Member Author

@pirj pirj commented Oct 7, 2019

@JonRowe Sounds good. I can tackle this in the near future.
What I intend to do is to warn in enforce_value_expectation instead of failing.
Not sure I completely understand what you mean by block introspection, can you please elaborate a bit?

Loading

@JonRowe
Copy link
Member

@JonRowe JonRowe commented Oct 8, 2019

Things like expect(-> { }).to be_a(Proc)

Loading

JonRowe added a commit that referenced this issue May 8, 2020
…m-being-used-with-value-expectation-target"

This reverts commit f5bf9d8.
pirj added a commit that referenced this issue Jan 28, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Jan 29, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Jan 29, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Jan 29, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Feb 7, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Feb 7, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Feb 7, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
pirj added a commit that referenced this issue Feb 8, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  #1139
  #1125
yujinakayama pushed a commit to yujinakayama/rspec-monorepo that referenced this issue Oct 6, 2021
…pirj/prevent-block-matchers-from-being-used-with-value-expectation-target

Prevent block-only matchers from being used with value expectation target

---
This commit was imported from rspec/rspec-expectations@f5bf9d8.
yujinakayama pushed a commit to yujinakayama/rspec-monorepo that referenced this issue Oct 6, 2021
… from pirj/prevent-block-matchers-from-being-used-with-value-expectation-target"

This reverts commit f5bf9d8264ffd786a66fe25c3bb1125c93b59d89.

---
This commit was imported from rspec/rspec-expectations@9423442.
yujinakayama pushed a commit to yujinakayama/rspec-monorepo that referenced this issue Oct 6, 2021
… from pirj/prevent-block-matchers-from-being-used-with-value-expectation-target"

This reverts commit f5bf9d8264ffd786a66fe25c3bb1125c93b59d89.

---
This commit was imported from rspec/rspec-expectations@c3b567d.
yujinakayama pushed a commit to yujinakayama/rspec-monorepo that referenced this issue Oct 19, 2021
https://blog.rubystyle.guide/rspec/2019/07/17/rspec-implicit-block-syntax.html
https://rspec.rubystyle.guide/#implicit-block-expectations

Spec changes are due to:

    matcher.matches?(invalid_value)

doesn't work with block-only matchers, as no block is passed in, and it
fails with a message:

    1) RSpec::Matchers::BuiltIn::Change behaves like an RSpec block-only matcher uses the `ObjectFormatter` for `failure_message`
       Failure/Error: expect(message).to include("detailed inspect")
         expected "expected `@k` to have changed, but was not given a block" to include "detailed inspect"

The redundant (due to existing check in ExpectationTarget) `Proc ===
@event_proc` checks could not be removed safely as well, since
@actual_after is not initialized yet when we haven't executed the block:

     RuntimeError:
       Warnings were generated: rspec-dev/repos/rspec-expectations/lib/rspec/matchers/built_in/change.rb:407: warning: instance variable @actual_after not initialized

Also see:
  rspec/rspec-expectations#1139
  rspec/rspec-expectations#1125

---
This commit was imported from rspec/rspec-expectations@aaf93ad.
@pirj pirj mentioned this pull request Nov 29, 2021
51 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants