rerun_formatter.feature is failing

[mattwynne]

bundle exec cucumber features/docs/formatters/rerun_formatter.feature

It looks like the problem is that Scenario doesn't have failed? declared.

@mattwynne @tooky what is the right way to get status of a scenario?

[mattwynne]

@os97673 long story! The report API from the core has an after_test_case message which would be perfect for this formatter. It passes the Test::Case which you can call #location on to get the file:line of the failed scenario or example row, and Test::Result which tells you the result of the test case.

It would be great to re-write the rerun formatter using this API, since it's much simpler and would require less code. However this would mean we'll need to invent a way to declare which formatters use the new report API, and which still need wrapping with the Cucumber::Formatter::ReportAdapter.

Might be a good idea to discuss this on the call this week?

[mattwynne]

The report API I mentioned is spec'd here: https://github.com/cucumber/cucumber-ruby-core/blob/master/spec/cucumber/core/test/runner_spec.rb. Probably needs better documentation.

[os97673]

Might be a good idea to discuss this on the call this week?

yes, I think it is a good idea. I'm going to check reasons of other failures we have in our fetures for formatters (perhaps there are some other obvious problems in this area)

[mattwynne]

Right, @tooky and I did some work on this in a pairing session today. We've written a new Cucumber::Reports::Rerun that can replace the old Cucumber::Formatter::Rerun.

The challenge now is to work on the Runtime so that it can cope with both legacy formatters and new-style reports.

I've thought about this a bit, and my best idea so far is to add a facade for each legacy formatter in the Cucumber::Reports namespace. This could still be awkward as the old-style formatters need additional constructor parameters.

Maybe you can have a play with this @os97673?

[tooky]

@mattwynne I've integrated the new rerun report - the scenario that was added for #503 is still failing, but I don't think --expand is working yet.

I've added a method to the LegacyFormatter which generates a new report class which will wrap a specific formatter. At the moment I'm using this to wrap all user supplied formatters, but it would be nice if we could detect which API they conform to and wrap if needed?

I added a factory method configure to the new report api - perhaps we could use to detect if we have a legacy formatter or a report?

[mattwynne]

Looking at these names again, I'm thinking we have three things:

1. the core report API
2. the extended report API for Cucumber (CORE_REPORT_API + [:embed, :puts])
3. the user interface API for Cucumber ([:ask]) which only needs to be sent to the user interface, not all the formatters / reports

Oh and
4. the legacy formatter API

We could refer to (2) as the 'new formatter API', and go back to keeping all this stuff in lib/formatters, moving the older stuff into lib/formatters/legacy. That would give us a clear distinction between (1) and (2), and hopefully enough distinction between (1), (2) and (4).

I think at least we should talk about (1) as CORE_REPORT_API and then (2) as REPORT_API. That's probably clear enough, and certainly having a new name for this will make the change very clear to our users. Are we going to change the CLI too, and deprecate the --formatter switch in favour of --report ?

[mattwynne]

New rerun formatter is now live as of 2.0.0.beta.5

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.