Add some regression tests for some popular custom formatter gems. #1292

Closed
wants to merge 14 commits into from

3 participants

@myronmarston
RSpec member

For #1291.

So currently fuubar, nyancat and fivemat are all broken against the latest formatter changes:

➜  rspec-core git:(add-formatter-regression-tests) ✗ be cucumber --profile wip
Using the wip profile...
@wip @announce
Feature: Regression tests for legacy custom formatters

  Background:                                                   # features/formatters/regression_tests.feature:4
    Given a file named "spec/passing_and_failing_spec.rb" with: # aruba-0.5.4/lib/aruba/cucumber.rb:24
      $ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
      """
      RSpec.describe "Some examples" do
        it "passes" do
          expect(1).to eq(1)
        end

        it "fails" do
          expect(1).to eq(2)
        end

        context "nested" do
          it "passes" do
            expect(1).to eq(1)
          end

          it "fails" do
            expect(1).to eq(2)
          end
        end
      end
      """
    And a file named "spec/bar_spec.rb" with:                   # aruba-0.5.4/lib/aruba/cucumber.rb:24
      """
      RSpec.describe "Some pending examples" do
        context "pending" do
          it "is reported as pending" do
            pending { expect(1).to eq(2) }
          end

          it "is reported as failing" do
            pending { expect(1).to eq(1) }
          end
        end

        context "skip" do
          it "does not run the example" do
            skip
          end
        end
      end
      """

  Scenario: Use fuubar formatter                                # features/formatters/regression_tests.feature:48
    When I run `rspec --format Fuubar`                          # aruba-0.5.4/lib/aruba/cucumber.rb:84
      $ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
      $ rspec --format Fuubar

      /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for #<Fuubar:0x007fcf8cac1a80> (NameError)
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
        from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
    Then the output should contain "TBD"                        # aruba-0.5.4/lib/aruba/cucumber.rb:131
      expected "/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for #<Fuubar:0x007fcf8cac1a80> (NameError)\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'\n" to include "TBD"
      Diff:
      @@ -1,2 +1,15 @@
      -TBD
      +/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fuubar-1.3.2/lib/fuubar.rb:18:in `initialize': undefined local variable or method `example_count' for #<Fuubar:0x007fcf8cac1a80> (NameError)
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `new'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters.rb:99:in `add'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration.rb:583:in `add_formatter'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `block in load_formatters_into'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `each'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:92:in `load_formatters_into'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/configuration_options.rb:18:in `configure'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:20:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
       (RSpec::Expectations::ExpectationNotMetError)
      features/formatters/regression_tests.feature:50:in `Then the output should contain "TBD"'

  Scenario: Use fivemat formatter                               # features/formatters/regression_tests.feature:52
    When I run `rspec --format Fivemat`                         # aruba-0.5.4/lib/aruba/cucumber.rb:84
      $ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
      $ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
      $ rspec --format Fivemat
      Some pending examples **FFFF
      /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)
        from /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
        from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
    Then the output should contain "TBD"                        # aruba-0.5.4/lib/aruba/cucumber.rb:131
      expected "Some pending examples **FFFF\n\n/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)\n\tfrom /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'\n" to include "TBD"
      Diff:
      @@ -1,2 +1,18 @@
      -TBD
      +Some pending examples **FFFF
      +
      +/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `instance_method': undefined method `start_dump' for class `RSpec::Core::Formatters::BaseTextFormatter' (NameError)
      + from /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/fivemat-1.2.1/lib/fivemat/rspec.rb:63:in `start_dump'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
       (RSpec::Expectations::ExpectationNotMetError)
      features/formatters/regression_tests.feature:54:in `Then the output should contain "TBD"'

  Scenario: Use nyancat formatter                               # features/formatters/regression_tests.feature:56
    When I run `rspec --format NyanCatFormatter`                # aruba-0.5.4/lib/aruba/cucumber.rb:84
      $ export SPEC_OPTS="-r/Users/myron/code/rspec-dev/repos/rspec-core/features/support/require_expect_syntax_in_aruba_specs.rb"
      $ cd /Users/myron/code/rspec-dev/repos/rspec-core/tmp/aruba
      $ rspec --format NyanCatFormatter
      *F
      An error occurred
        NoMethodError: undefined method `-' for nil:NilClass
        occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'


      An error occurred
        NoMethodError: undefined method `-' for nil:NilClass
        occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'

      F
      stty: stdin isn't a terminal
      stty: stdin isn't a terminal
      stty: stdin isn't a terminal
      /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
        from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
        from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
        from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
    Then the output should contain "TBD"                        # aruba-0.5.4/lib/aruba/cucumber.rb:131
      expected "*F\nAn error occurred \n  NoMethodError: undefined method `-' for nil:NilClass\n  occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'\n\n\nAn error occurred \n  NoMethodError: undefined method `-' for nil:NilClass\n  occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'\n\nF\nstty: stdin isn't a terminal\nstty: stdin isn't a terminal\nstty: stdin isn't a terminal\n/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'\n\tfrom /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'\n" to include "TBD"
      Diff:
      @@ -1,2 +1,30 @@
      -TBD
      +*F
      +An error occurred
      +  NoMethodError: undefined method `-' for nil:NilClass
      +  occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'
      +
      +
      +An error occurred
      +  NoMethodError: undefined method `-' for nil:NilClass
      +  occurred at /Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter.rb:97:in `terminal_width'
      +
      +F
      +stty: stdin isn't a terminal
      +stty: stdin isn't a terminal
      +stty: stdin isn't a terminal
      +/Users/myron/code/rspec-dev/repos/bundle/ruby/1.9.1/gems/nyan-cat-formatter-0.5.2/lib/nyan_cat_formatter/rspec2.rb:26:in `start_dump': wrong number of arguments (1 for 0) (ArgumentError)
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:67:in `start_dump'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/formatters/legacy_formatter.rb:177:in `start_dump'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:114:in `block in notify'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `each'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:113:in `notify'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:96:in `finish'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `ensure in report'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/reporter.rb:49:in `report'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/command_line.rb:24:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:100:in `run'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/lib/rspec/core/runner.rb:31:in `invoke'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/exe/rspec:4:in `<top (required)>'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `load'
      + from /Users/myron/code/rspec-dev/repos/rspec-core/bin/rspec:12:in `<main>'
       (RSpec::Expectations::ExpectationNotMetError)
      features/formatters/regression_tests.feature:58:in `Then the output should contain "TBD"'

Failing Scenarios:
cucumber -p wip features/formatters/regression_tests.feature:48 # Scenario: Use fuubar formatter
cucumber -p wip features/formatters/regression_tests.feature:52 # Scenario: Use fivemat formatter
cucumber -p wip features/formatters/regression_tests.feature:56 # Scenario: Use nyancat formatter

3 scenarios (3 failed)
12 steps (3 failed, 9 passed)
0m1.988s

The --wip switch was used, so the failures were expected. All is good.

@JonRowe -- do you want to take a stab at fixing these or would you like me to?

@myronmarston
RSpec member

Actually, you've got the deprecation for for rspec/rspec-expectations#423 so I'll take a stab at this.

@JonRowe
RSpec member

I got Fuubar working, on my machine and in the cuke. I've also got Nyan "running" by delaying formatter instansiation for legacy formatters until we've mixed in the module, but it's got an issue due to inheritance (our module takes precedence over theres :/) Fivemat is more of a wtf

@JonRowe
RSpec member

Fivemat doesn't itself implement a formatter, instead returning on on new, so doesn't fit trigger the ancestor detection approach :/

@myronmarston
RSpec member

Fivemat is pretty crazy. Besides the Fivemat.new craziness, there's the mother of all brittle hacks in it's start_dump implementation:

https://github.com/tpope/fivemat/blob/5586092ae26241484f5e71fd27de350dfff0a1fa/lib/fivemat/rspec.rb#L61-L64

I don't think there's anyway we can support formatters that use those kind of hacks without putting special cases in rspec-core specific to the formatter, which I don't think we should do. I'm going to remove fivemat from the cuke here. Are there other popular formatters we should support?

@soulcutter
RSpec member

TeamCity uses a custom formatter which is broken

@myronmarston
RSpec member

TeamCity uses a custom formatter which is broken

Gotta link to it? I'll through it on the list here as well. It's really easy to add these to the cuke.

@myronmarston myronmarston Add scenarios for rspec-extra-formatters.
The tap formatter is failing due to an attempted frozen string mutation.
c06895a
@myronmarston
RSpec member

I added a few more scenarios. rspec-instafail works. So does the junit formatter from rspec-extra-formatters. The tap formatter from that gem fails though (due to an attempted mutation of a frozen string). The rspec_spinner formatter fails as well -- looks potentially similar to the nyancat failure.

@myronmarston
RSpec member

@JonRowe -- heads up, I'm going to rebase this against master (and maybe squash/amend the commits a bit). There's some failing specs in this branch due to the recent removal of support for and_return { } in rspec-mocks so I need to rebase anyway to keep things green. Usually I wouldn't do that to a branch two of us are working on but it's needed in this case. So if you have any changes in a local branch you'll need to re-apply them after I push (by cherry-picking or whatever). I'll let you know when I've pushed.

@JonRowe
RSpec member

I've made some progress on this using some meta programming magic, let me know when you want me to push it... (my local repo works fine cause i haven't updated mocks/expectations yet)

@myronmarston
RSpec member
@JonRowe
RSpec member

So this code now suffers two issues,

1) the undefine method code doesnt work quite correctly, so nyan for example is implementing methods twice :/ so superclassing is still broken.
2) the lack of a real TTY is actually breaking nyancat on the cukes...

But the nyancat formatter now loads and runs so it's closer... any ideas?

@JonRowe
RSpec member

Fixed! Only Nyan now fails in the cuke due to the TTY issue.

@JonRowe JonRowe commented on the diff Feb 9, 2014
lib/rspec/core/formatters/legacy_formatter.rb
+ super
+
+ # restore the clashing methods on top of ours
+ override_module = Module.new do
+ class << self
+ attr_accessor :__clashing_methods
+ end
+ def self.included(other)
+ __clashing_methods.each do |(name, meth)|
+ other.send :define_method, name, meth
+ end
+ end
+ end
+ override_module.__clashing_methods = clashing_methods
+ other.send :include, override_module
+ end
@JonRowe
RSpec member
JonRowe added a note Feb 9, 2014

I'm worryingly proud of the mightiness of this hax ;) let me know if you need me to explain it...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@JonRowe
RSpec member

Also I don't mind rebasing it if you want?

@myronmarston
RSpec member

Also I don't mind rebasing it if you want?

No, please don't. I've already got some ammended commits in my local branch (e.g. adding some specs for the cases you fixed before). I was busy most of the day and didn't get a chance to push what I had. I'll work off of what you have here and take care of the rebase since I'm already in process.

@JonRowe
RSpec member

No worries :)

@myronmarston
RSpec member

Closing in favor of #1309. I'm not going to delete this branch, though, so the commits can be cherry-picked as needed in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment