Rework AggregateExamples cop

The "one expectation per example" rule has been relaxed and allows for several expectations to be set in the same example.
https://github.com/rubocop-hq/rspec-style-guide#expectations-per-example

In cases the examples don't have any setup, metadata, or even a docstring, and may be aggregated into one thus saving on sometimes expensive context setup.

The cop still does report the cases when the examples might be aggregated.

Block expectation syntax is deliberately not supported due to:
 - `subject { -> { ... } }` syntax being hard to detect
 - aggregation should use composition with `.and`
 - aggregation of the `not_to` is barely possible when a matcher doesn't
 provide a negated variant
 - aggregation of block syntax with non-block syntax should be in a
 specific order

Known caveats:
The usages if `its` that are testing private methods/readers will result in spec failure. Test only public interface!

Matchers with side effects might affects following expectations when aggregated.

Originally submitted as rubocop/rubocop-rspec#726

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## master (unreleased)
 
 - Fix `let_it_be` issue when initialized with an array/enumerable or an AR relation. ([@pirj][])
+- Improve `RSpec/AggregateExamples` (formerly `RSpec/AggregateFailures`) cop. ([@pirj][])
 
 ## 0.10.2 (2020-01-07) 🎄
 

config/default.yml

@@ -0,0 +1,35 @@
+---
+AllCops:
+  RSpec:
+    Patterns:
+    - _spec.rb
+    - "(?:^|/)spec/"
+
+RSpec/AggregateExamples:
+  Description: Checks if example groups contain two or more aggregatable examples.
+  Enabled: true
+  StyleGuide: https://rspec.rubystyle.guide/#expectation-per-example
+  AddAggregateFailuresMetadata: true
+  MatchersWithSideEffects:
+    - allow_value
+    - allow_values
+    - validate_presence_of
+    - validate_absence_of
+    - validate_length_of
+    - validate_inclusion_of
+    - validates_exclusion_of
+
+# TODO: remove this one we hit 1.0
+RSpec/AggregateFailures:
+  Description: Checks if example groups contain two or more aggregatable examples.
+  Enabled: true
+  StyleGuide: https://rspec.rubystyle.guide/#expectation-per-example
+  AddAggregateFailuresMetadata: true
+  MatchersWithSideEffects:
+    - allow_value
+    - allow_values
+    - validate_presence_of
+    - validate_absence_of
+    - validate_length_of
+    - validate_inclusion_of
+    - validates_exclusion_of

docs/rubocop.md

@@ -2,36 +2,33 @@
 
 TestProf comes with the [RuboCop](https://github.com/bbatsov/rubocop) cops that help you write more performant tests.
 
-To enable them:
-
-- Require `test_prof/rubocop` in your RuboCop configuration:
+To enable them, require `test_prof/rubocop` in your RuboCop configuration:
 
 ```yml
 # .rubocop.yml
 require:
  - 'test_prof/rubocop'
 ```
 
-- Enable cops:
+To configure cops to your needs:
 
 ```yml
-RSpec/AggregateFailures:
-  Enabled: true
-  Include:
-    - 'spec/**/*.rb'
+RSpec/AggregateExamples:
+  AddAggregateFailuresMetadata: false
 ```
 
 Or you can just require it dynamically:
 
 ```sh
-bundle exec rubocop -r 'test_prof/rubocop' --only RSpec/AggregateFailures
+bundle exec rubocop -r 'test_prof/rubocop' --only RSpec/AggregateExamples
 ```
 
-## RSpec/AggregateFailures
+## RSpec/AggregateExamples
 
 This cop encourages you to use one of the greatest features of the recent RSpec – aggregating failures within an example.
 
-Instead of writing one example per assertion, you can group _independent_ assertions together, thus running all setup hooks only once. That can dramatically increase your performance (by reducing the total number of examples).
+Instead of writing one example per assertion, you can group _independent_ assertions together, thus running all setup hooks only once.
+That can dramatically increase your performance (by reducing the total number of examples).
 
 Consider an example:
 
@@ -51,8 +48,10 @@ it "returns the second page", :aggregate_failures do
 end
 ```
 
+Auto-correction will typically add `:aggregate_failures` to examples, but if your project enables it globally, or selectively by e.g. deriving metadata from file location, you may opt-out of adding it using `AddAggregateFailuresMetadata` config option.
+
 This cop supports auto-correct feature, so you can automatically refactor you legacy tests!
 
 **NOTE**: `its` examples shown here have been deprecated as of RSpec 3, but users of the [rspec-its gem](https://github.com/rspec/rspec-its) can leverage this cop to cut out that dependency.
 
-**NOTE**: auto-correction may break your tests (especially the ones using block-matchers, such as `change`).
+**NOTE**: auto-correction of examples using block matchers, such as `change` is deliberately not supported.

forspell.dict

@@ -11,6 +11,7 @@ flamegraphs
 Gitlab
 instrumenter
 Isolator
+matchers
 Modifers
 profilers
 speedscope

lib/test_prof/cops/inject.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+# This is shamelessly borrowed from RuboCop RSpec
+# https://github.com/rubocop-hq/rubocop-rspec/blob/master/lib/rubocop/rspec/inject.rb
+module RuboCop
+  # Because RuboCop doesn't yet support plugins, we have to monkey patch in a
+  # bit of our configuration.
+  module Inject
+    PROJECT_ROOT = Pathname.new(__dir__).parent.parent.parent.expand_path.freeze
+    CONFIG_DEFAULT = PROJECT_ROOT.join("config", "default.yml").freeze
+
+    def self.defaults!
+      path = CONFIG_DEFAULT.to_s
+      hash = ConfigLoader.send(:load_yaml_configuration, path)
+      config = Config.new(hash, path)
+      puts "configuration from #{path}" if ConfigLoader.debug?
+      config = ConfigLoader.merge_with_default(config, path)
+      ConfigLoader.instance_variable_set(:@default_configuration, config)
+    end
+  end
+end
+
+RuboCop::Inject.defaults!

lib/test_prof/cops/rspec/aggregate_examples.rb

@@ -0,0 +1,199 @@
+# frozen_string_literal: true
+
+require_relative "aggregate_examples/line_range_helpers"
+require_relative "aggregate_examples/metadata_helpers"
+require_relative "aggregate_examples/node_matchers"
+
+require_relative "aggregate_examples/its"
+require_relative "aggregate_examples/matchers_with_side_effects"
+
+module RuboCop
+  module Cop
+    module RSpec
+      # Checks if example groups contain two or more aggregatable examples.
+      #
+      # @see https://github.com/rubocop-hq/rspec-style-guide#expectations-per-example
+      #
+      # This cop is primarily for reducing the cost of repeated expensive
+      # context initialization.
+      #
+      # @example
+      #
+      #   # bad
+      #   describe do
+      #     specify do
+      #       expect(number).to be_positive
+      #       expect(number).to be_odd
+      #     end
+      #
+      #     it { is_expected.to be_prime }
+      #   end
+      #
+      #   # good
+      #   describe do
+      #     specify do
+      #       expect(number).to be_positive
+      #       expect(number).to be_odd
+      #       is_expected.to be_prime
+      #     end
+      #   end
+      #
+      #   # fair - subject has side effects
+      #   describe do
+      #     specify do
+      #       expect(multiply_by(2)).to be_multiple_of(2)
+      #     end
+      #
+      #     specify do
+      #       expect(multiply_by(3)).to be_multiple_of(3)
+      #     end
+      #   end
+      #
+      # Block expectation syntax is deliberately not supported due to:
+      #
+      # 1. `subject { -> { ... } }` syntax being hard to detect, e.g. the
+      #    following looks like an example with non-block syntax, but it might
+      #    be, depending on how the subject is defined:
+      #
+      #        it { is_expected.to do_something }
+      #
+      #    If the subject is defined in a `shared_context`, it's impossible to
+      #    detect that at all.
+      #
+      # 2. Aggregation should use composition with an `.and`. Also, aggregation
+      #    of the `not_to` expectations is barely possible when a matcher
+      #    doesn't provide a negated variant.
+      #
+      # 3. Aggregation of block syntax with non-block syntax should be in a
+      #    specific order.
+      #
+      # RSpec [comes with an `aggregate_failures` helper](https://relishapp.com/rspec/rspec-expectations/docs/aggregating-failures)
+      # not to fail the example on first unmet expectation that might come
+      # handy with aggregated examples.
+      # It can be [used in metadata form](https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures#use-%60:aggregate-failures%60-metadata),
+      # or [enabled globally](https://relishapp.com/rspec/rspec-core/docs/expectation-framework-integration/aggregating-failures#enable-failure-aggregation-globally-using-%60define-derived-metadata%60).
+      #
+      # @example Globally enable `aggregate_failures`
+      #
+      #   # spec/spec_helper.rb
+      #   config.define_derived_metadata do |metadata|
+      #     unless metadata.key?(:aggregate_failures)
+      #       metadata[:aggregate_failures] = true
+      #     end
+      #   end
+      #
+      # To match the style being used in the spec suite, AggregateExamples
+      # can be configured to add `:aggregate_failures` metadata to the
+      # example or not. The option not to add metadata can be also used
+      # when it's not desired to make expectations after previously failed
+      # ones, commonly known as fail-fast.
+      #
+      # The terms "aggregate examples" and "aggregate failures" not to be
+      # confused. The former stands for putting several expectations to
+      # a single example. The latter means to run all the expectations in
+      # the example instead of aborting on the first one.
+      #
+      # @example AddAggregateFailuresMetadata: true (default)
+      #
+      #   # Metadata set using a symbol
+      #   specify(:aggregate_failures) do
+      #     expect(number).to be_positive
+      #     expect(number).to be_odd
+      #   end
+      #
+      # @example AddAggregateFailuresMetadata: false
+      #
+      #   specify do
+      #     expect(number).to be_positive
+      #     expect(number).to be_odd
+      #   end
+      #
+      class AggregateExamples < Cop
+        include LineRangeHelpers
+        include MetadataHelpers
+        include NodeMatchers
+
+        # Methods from the following modules override and extend methods of this
+        # class, extracting specific behavior.
+        prepend Its
+        prepend MatchersWithSideEffects
+
+        MSG = "Aggregate with the example at line %d."
+
+        def on_block(node)
+          example_group_with_several_examples(node) do |all_examples|
+            example_clusters(all_examples).each do |_, examples|
+              examples[1..-1].each do |example|
+                add_offense(example,
+                  location: :expression,
+                  message: message_for(example, examples[0]))
+              end
+            end
+          end
+        end
+
+        def autocorrect(example_node)
+          clusters = example_clusters_for_autocorrect(example_node)
+          return if clusters.empty?
+
+          lambda do |corrector|
+            clusters.each do |metadata, examples|
+              range = range_for_replace(examples)
+              replacement = aggregated_example(examples, metadata)
+              corrector.replace(range, replacement)
+              examples[1..-1].map { |example| drop_example(corrector, example) }
+            end
+          end
+        end
+
+        private
+
+        # Clusters of examples in the same example group, on the same nesting
+        # level that can be aggregated.
+        def example_clusters(all_examples)
+          all_examples
+            .select { |example| example_with_expectations_only?(example) }
+            .group_by { |example| metadata_without_aggregate_failures(example) }
+            .select { |_, examples| examples.count > 1 }
+        end
+
+        # Clusters of examples that can be aggregated without losing any
+        # information (e.g. metadata or docstrings)
+        def example_clusters_for_autocorrect(example_node)
+          examples_in_group = example_node.parent.each_child_node(:block)
+            .select { |example| example_for_autocorrect?(example) }
+          example_clusters(examples_in_group)
+        end
+
+        def message_for(_example, first_example)
+          format(MSG, first_example.loc.line)
+        end
+
+        def drop_example(corrector, example)
+          aggregated_range = range_by_whole_lines(example.source_range,
+            include_final_newline: true)
+          corrector.remove(aggregated_range)
+        end
+
+        def aggregated_example(examples, metadata)
+          base_indent = " " * examples.first.source_range.column
+          metadata = metadata_for_aggregated_example(metadata)
+          [
+            "#{base_indent}specify#{metadata} do",
+            *examples.map { |example| transform_body(example, base_indent) },
+            "#{base_indent}end\n"
+          ].join("\n")
+        end
+
+        # Extracts and transforms the body, keeping proper indentation.
+        def transform_body(node, base_indent)
+          "#{base_indent}  #{new_body(node)}"
+        end
+
+        def new_body(node)
+          node.body.source
+        end
+      end
+    end
+  end
+end