Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for multiple reporters #3003

Conversation

vaibhavsingh97
Copy link
Member

Thanks in advance for your contribution. Please follow the below steps in submitting a pull request, as it will help us with reviewing it quicker.

  • Create a new branch from master (e.g. features/my-new-feature or issue/123-my-bugfix);
  • If you're fixing a bug also create an issue if one doesn't exist yet;
  • If it's a new feature explain why do you think it's necessary. Please check with the maintainers beforehand to make sure it is something that we will accept. Usually we only accept new features if we feel that they will benefit the entire community;
  • Please avoid sending PRs which contain drastic or low level changes. If you are certain that the changes are needed, please discuss them beforehand and indicate what the impact will be;
  • If your change is based on existing functionality please consider refactoring first. Pull requests that duplicate code will most likely be ignored;
  • Do not include changes that are not related to the issue at hand;
  • Follow the same coding style with regards to spaces, semicolons, variable naming etc.;
  • Always add unit tests - PRs without tests are most of the times ignored.

fixes #2588

@vaibhavsingh97 vaibhavsingh97 marked this pull request as draft January 20, 2022 13:12
@vaibhavsingh97 vaibhavsingh97 marked this pull request as ready for review January 21, 2022 05:45
Copy link
Member

@gravityvi gravityvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Along with customReporter (if provided) could we also generate builtInReporter? I think the issue also mentions that. In this case, we would need to pass in the path of default (builtin) reporter.

@vaibhavsingh97
Copy link
Member Author

Along with customReporter (if provided) could we also generate builtInReporter? I think the issue also mentions that. In this case, we would need to pass in the path of default (builtin) reporter.

yes, it work for both builtInReporter and customReporter.

Comment on lines 117 to 126
loadFile(reporterFile) {
const builtInReporterFileName = path.join(__dirname, 'reporters', reporterFile + '.js');

return Utils.fileExists(builtInReporterFileName).then((fileExists) => {
if (!fileExists) {
return this.loadCustomReporter(reporterFile);
}

return require(builtInReporterFileName);
});
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gravityvi This takes care of both built-in reporter and custom reporter

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it generates a single report with one --reporter CLI value. Could we update it to generate the default builtInReporter every time along with the reporter mentioned via CLI

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, we can do that. I will update the PR

return require(builtInReporterFileName);
});
if (!builtInReporter.includes(this.reporterFile)) {
this.reporterFile = ['junit', this.reporterFile];
Copy link
Member

@gravityvi gravityvi Feb 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this.reporterFile when sending --reporter html.js,junit from CLI comes out as a string 'html.js,junit', while an array is expected over here. We need to take care of that while getting cli args. Can you add a test for that as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At max, this implementation can generate two reporters for a single test run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not meant to be used like that, we need to pass --reporter junit --reporter html.js, but I will amend it to make use of the example you mentioned, as commonly people would prefer that

Copy link
Member

@beatfactor beatfactor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The behaviour of using multiple reporters should be:

  • pass the reporters via nightwatch --reporter test_reporter.js --reporter junit

This is necessary because we have to keep the existing behaviour where you are using a custom reporter without using the junit reporter.

The test needs to reflect this so you need to write something like here: https://github.com/nightwatchjs/nightwatch/blob/main/test/src/runner/testRunnerSessionCreate.js#L133

runTests({
  reporter: ['junit', 'other-reporter-file.js']
}, {
  output_folder: true
});

});
});

it('test run tests with multiple reporters - html, junit, json', function () {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test is incomplete.

@vaibhavsingh97 vaibhavsingh97 changed the base branch from main to features/multiple-reporters June 13, 2022 12:18
@beatfactor beatfactor merged commit a8d254b into nightwatchjs:features/multiple-reporters Jun 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support more that one reporter in the same test run
3 participants