one page Error Summary custom report, last failed report when retry flag is set

[hcsan]

Hi Team,

We have around 500 test cases. 50% script fails, Most of the time we need to rerun/retry min 2 to 3 times, those tests failed because of environment issues . we use (--max-retries-count=3 ) flag . Team is spending around 45 min to 1 hrs only to review , navigating through each and every test scenarios. ( Gauge report left nav is not getting highlighted/selected fixed one issue)

so we have created tool which will capture only failed scenarios using web-scrapping , which is working great one page Error Summary custom report.

Instead of creating outside of Gauge project we thought of integrating this in our gauge project. we successfully created using is scenario failed
if (context.getCurrentStep().getIsFailing()) {
SuiteDataStore.put("getStackTrace", context.getCurrentStep().getStackTrace().toString());
SuiteDataStore.put("getErrorMessage", context.getCurrentStep().getErrorMessage());
SuiteDataStore.put("getDynamicText", "
" + ScenarioDataStore.get("getDynamicText") + "
");
}

Now the problem is, say for ex if 22 scenarios are getting failed and we have Gauge retry for 3 times, as we set retry flag set to 3 (--max-retries-count=3 ) so will get 66 failed report and some may be passed after rerun will also got captured in the error report which is in correct.

so is there any way we can capture the flag retry count and only capture the last failed report at scenario-level?
or using - gauge_clear_state_level not helped. Any other methods or solution ?

Thanks
Shc

[zabil]

I don't think there's a way to capture the retry count. As an alternative to scraping maybe you can parsing xml and json using the xml and json report plugins. This however will still need to be done post gauge execution.

[sriv]

When running with --max-retries-count=3 the html report should highlight in the reports page that the scenario was retried.

The left nav is at a spec level, hence you do not see any indicators (we could add it there, to indicate that the spec run report has at least one scenario which was retried).

The retry count is not currently passed to the execution context, maybe this could be added.

Ideally, test failure analysis should come out of the box, so anything that can help here could be included in gauge / the reports. Do you have any inputs on what sort of enhancement could we do to bring this information in the html-reports?