Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

output from test reports doesn't seem right #50

Open
ppazos opened this issue Dec 14, 2018 · 4 comments
Open

output from test reports doesn't seem right #50

ppazos opened this issue Dec 14, 2018 · 4 comments

Comments

@ppazos
Copy link

ppazos commented Dec 14, 2018

Hi I'm testing this plugin on Grails 2.5.6 (the latest 2.x version), and I have this feature file:

Feature: xyz
  some description here

  Scenario: aaa
    Given ...
    When ...
    Then ...

  Scenario: bbb
    Given ...
    When ...
    Then ...

  Scenario: ccc
    Given ...
    When ...
    Then ...

On grails test-app :cucumber I the output for Scenario aaa in the test html report is "Feature xyz ...", then for Scenario bbb the output is "Scenario aaa ....", and for Scenario ccc the output is "Scenario bbb .... Scenario ccc".

It seems the outputs are not mapped to the correspondent Scenario being executed.

@hauner
Copy link
Owner

hauner commented Dec 16, 2018

uhh, looks like the reporting in cucumber or grails has changed so that the current mapping from cucumber to junit/grails doesn't fit anymore. See CucumberTestType, you could use the DebugFormatter here to find out where it is going wrong.

@ppazos
Copy link
Author

ppazos commented Dec 20, 2018

Thanks @hauner do you know how can I include the modified plugin on an app to test this?

@ppazos
Copy link
Author

ppazos commented Dec 20, 2018

Got it working adding this to the buildconfig grails.plugin.location.cucumber = '/home/pablo/GitHub/grails-cucumber'

I'm seeing everything that outputs with R is empty, only the F steps have some info:

F(scenario): (Upload an ADL 1.4-based OPT)
F(step): (current count of OPTs is 0)
F(step): (the client uploads an OPT to the I_DEFINITION_ADL14 with template_id "encounter")
F(step): (current count of OPTs is 1)
R(match)
R(result):
R(match)
R(result):
R(match)
R(result):
R(after)
R(after)
AFTER2
R(after)
F(endOfScenarioLifeCycle): (Upload an ADL 1.4-based OPT)
F(startOfScenarioLifeCycle): (Get template by template_id)
R(before)

Also seeing the behavior described in #52 all the Before and After methods are called on every scenario, even if those are declared on different step files.

But can't see what is going on in the Cucumber Formatter with the report of the features and scenarios, there is too much IoC there to understand the workflow.

One thing I found is the activeFeature = null was missing from here:

private void endFeature () {
        if (activeFeature) {
            log.trace ("end feature ()\n")

            report.endFeature ()
            publisher.testCaseEnd (activeFeature.name)
            activeFeature = null
        }
    }

Not sure if that fixed anything. On the HTML report I get the output of the feature and first scenario on the second scenario, and the first scenario has no output. Also getting the output of the second and third scenarios on the third scenario.

I'm not sure how the HTML report is created and how to associate the specific outputs with the correct scenario. Also not sure if the feature output should be there at all, since there is no row for the feature, just seeing rows for the scenarios. It is easier with an image:

unit test results all tests

@hauner
Copy link
Owner

hauner commented Feb 10, 2019

I Looked a minute at the code, the CucumberFormatter is the one that tries to translate from cucumber to junit and feeds the GrailsTestEventPublisher that is probably the source of the report generated by grails.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants