Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Total Duration displayed in the report seems to be incorrect when the feature files are executed in parallel #1071

Open
MadhuQAAutomation opened this issue Sep 1, 2022 · 3 comments

Comments

@MadhuQAAutomation
Copy link

MadhuQAAutomation commented Sep 1, 2022

Hi - The duration value displayed in the cucumber report seems to be incorrect when we feature files were executed in parallel. The time taken for the complete set of feature files to get executed is ~ 39 minutes. But in the report, it shows 3:9:52.041

But this is the actual value :

Summary: 94% passed, 6% failed
Duration: 39 min, 9 sec
Run Level: Feature
Total: 54
Passed: 51
Failed: 3

Same was observed while running the test from JENKINS as well.

File reportOutputDirectory=new File("reports/cucumber/html-reports"); List<String> jsonFiles=new ArrayList<>(); jsonFiles.add("reports/cucumber/json/cucumber.json"); String projectName="cucumber-test-automation"; Configuration configuration=new Configuration(reportOutputDirectory,projectName); //configuration.addPresentationModes(PresentationMode.RUN_WITH_JENKINS); // do not make scenario failed when step has status SKIPPED configuration.setNotFailingStatuses(Collections.singleton(Status.PENDING)); configuration.setNotFailingStatuses(Collections.singleton(Status.UNDEFINED)); configuration.setNotFailingStatuses(Collections.singleton(Status.SKIPPED)); configuration.setSortingMethod(SortingMethod.NATURAL); /*configuration.setTrendsStatsFile(new File("target/trends/trends.json")); configuration.setTrends(new File("target/cucumber/trends/trends.json"),5);*/ configuration.addReducingMethod(ReducingMethod.HIDE_EMPTY_HOOKS); configuration.addReducingMethod(ReducingMethod.MERGE_FEATURES_BY_ID); configuration.addPresentationModes(PresentationMode.PARALLEL_TESTING); configuration.addPresentationModes(PresentationMode.EXPAND_ALL_STEPS); // addidtional metadata presented on main page configuration.addClassifications("Platform","Windows"); configuration.addClassifications("Java","11.0.15"); configuration.addClassifications("Cucumber","7.3.4"); ReportBuilder reportBuilder=new ReportBuilder(jsonFiles,configuration); Reportable result=reportBuilder.generateReports();

image

image

@damianszczepanik
Copy link
Owner

Report takes all duration and sum them together to have total values. It has no information if the report was generated at once or in parallel.

@gerardosanchezqa
Copy link

Hello there, @damianszczepanik !

First of all thanks for working on this reporting tool, it's been super useful in our project.

Question about your last reply:
We're running into the same issue as the original poster. Is there a way to add information to cucumber.json to let the report know that it was run in parallel? We tried with PresentationMode.PARALLEL_TESTING but didn't see the duration being adjusted.

@damianszczepanik
Copy link
Owner

I don't know answer for that question. Probably depends on the framework you use

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants