Output JUnit XML metadata for CircleCi #12023

benbraou · 2018-01-15T17:28:22Z

Fixes #11949

Summary

Currently, when the build fails, there is no way to know what happened other than going through the logs in CircleCI.

The goal of this PR is to output JUnit XML metadata for every build step. This allows CircleCI summary page to display exactly which step has failed.

What this PR does:

Handle Jest JUnit reports generation
jest-junit reporter is used to output JUnit report reports in /junit/jest-results.xml
⚠️ test_entry_point.sh is calling yarn test several times. So, a unit test failure will be duplicated in the build summary
Handle ESLint JUnit reports generation
ESLint script can either use the default stylish formatter or junit formatter.
⚠️ in JUnit mode, ESlint results are in XML format. Not only are they written to report fils, but also they are logged to console. This may be a noise to the build logs.

⚠️ For the tasks described below, the package junit-report-builder has been used. Initially, I was writing the JUnit report on my own.

Handle flow JUnit reports generation
Flow result is treated as a single test that has a single test output
Handle prettier JUnit reports generation
Handle check_license.sh JUnit report generation
Handle check_modules.sh JUnit report generation
test_print_warnings.sh
Nothing to be done as this script is simply used to extract all messages from `warning()
Handle track_stats JUnit.sh report generation
Handle build.sh JUnit.sh report generation
Handle upload_build.sh JUnit report generation
Handle JUnit report generation for coveralls
Handle JUnit report generation for Danger

facebook-github-bot · 2018-01-15T17:28:36Z

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

facebook-github-bot · 2018-01-15T17:40:16Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

Fixes facebook#12023

nhunzaker · 2018-08-03T20:06:52Z

Hey! I know we're getting to this pretty late, but do you mind upstreaming this? I took a pass but got stuck on some of the Flow stuff and thought you might have an easier time with it.

Thanks!

benbraou · 2018-08-05T15:13:18Z

@nhunzaker sure. This week, I have limited access to my laptop unfortunately. I will work on it next week.

nhunzaker · 2018-08-05T16:49:03Z

@benbraou no worries at all! I'll check back in about two weeks, but please let me know if you have any questions!

benbraou · 2018-08-15T08:42:38Z

@nhunzaker thanks. I finished rebasing my branch on top of master. No questions for now 😃. I am just curious to see how you are going to validate my changes. Up until now, I was forcing a build failure and then checking that the correct message is displayed in CircleCi summary section (I was also checking the locally generated junit report)

nhunzaker · 2018-08-17T13:40:43Z

scripts/circleci/test_entry_point.sh

  COMMANDS_TO_RUN+=('./scripts/circleci/check_license.sh')
  COMMANDS_TO_RUN+=('./scripts/circleci/check_modules.sh')
  COMMANDS_TO_RUN+=('./scripts/circleci/test_print_warnings.sh')
  COMMANDS_TO_RUN+=('./scripts/circleci/track_stats.sh')
 fi

 if [ $((1 % CIRCLE_NODE_TOTAL)) -eq "$CIRCLE_NODE_INDEX" ]; then
-  COMMANDS_TO_RUN+=('yarn test-prod --maxWorkers=2')
+  COMMANDS_TO_RUN+=("yarn test-prod --maxWorkers=2 --testResultsProcessor=${JEST_PROCESSOR}")


Will this override the reporter on line 13?

I noticed that it does not override but rather duplicate the junit error messages displayed in CircleCi. I am removing this, and keeping the reporter only in line 13.

nhunzaker · 2018-08-17T13:41:13Z

scripts/circleci/test_entry_point.sh

-  COMMANDS_TO_RUN+=('yarn test-build --maxWorkers=2')
-  COMMANDS_TO_RUN+=('yarn test-build-prod --maxWorkers=2')
+  COMMANDS_TO_RUN+=("yarn test-build --maxWorkers=2 --testResultsProcessor=${JEST_PROCESSOR}")
+  COMMANDS_TO_RUN+=("yarn test-build-prod --maxWorkers=2 --testResultsProcessor=${JEST_PROCESSOR}")


Same question here. Do these need to generate unique reports?

Indeed, no need for unique reports. I am removing this, and keeping the reporting only in line 13.

nhunzaker · 2018-08-17T13:42:34Z

scripts/circleci/write_junit_report.sh

+
+set -e
+
+node ./scripts/tasks/junit "$1" "$2" "$3"


Could you leave a comment in this script about what these arguments are?

nhunzaker · 2018-08-17T14:05:59Z

Fixes facebook#11949

benbraou · 2018-08-18T11:11:02Z

@nhunzaker thank you for the review. I have updated the PR with the requested changes. Please let me know if further actions are needed on my side.

stale · 2020-01-10T08:52:41Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution.

stale · 2020-01-17T08:58:55Z

Closing this pull request after a prolonged period of inactivity. If this issue is still present in the latest release, please ask for this pull request to be reopened. Thank you!

facebook-github-bot added the CLA Signed label Jan 15, 2018

benbraou force-pushed the issue11949 branch 2 times, most recently from 484c8fb to 55b83ec Compare January 20, 2018 16:49

benbraou added a commit to benbraou/react that referenced this pull request Jan 20, 2018

Output JUnit XML metada for Danger

9789d96

Fixes facebook#12023

benbraou mentioned this pull request Jan 20, 2018

Output JUnit XML test metadata for CircleCI #11949

Closed

benbraou force-pushed the issue11949 branch 2 times, most recently from 81ec4b8 to c7a10e6 Compare August 15, 2018 08:05

nhunzaker reviewed Aug 17, 2018

View reviewed changes

Output JUnit XML metadata for CircleCi

74ff0a2

Fixes facebook#11949

benbraou force-pushed the issue11949 branch from c7a10e6 to 74ff0a2 Compare August 18, 2018 10:56

stale bot added the Resolution: Stale Automatically closed due to inactivity label Jan 10, 2020

stale bot closed this Jan 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Output JUnit XML metadata for CircleCi #12023

Output JUnit XML metadata for CircleCi #12023

benbraou commented Jan 15, 2018 •

edited

facebook-github-bot commented Jan 15, 2018

facebook-github-bot commented Jan 15, 2018

nhunzaker commented Aug 3, 2018

benbraou commented Aug 5, 2018

nhunzaker commented Aug 5, 2018

benbraou commented Aug 15, 2018

nhunzaker Aug 17, 2018

benbraou Aug 18, 2018

nhunzaker Aug 17, 2018

benbraou Aug 18, 2018

nhunzaker Aug 17, 2018

benbraou Aug 18, 2018

nhunzaker commented Aug 17, 2018

benbraou commented Aug 18, 2018

stale bot commented Jan 10, 2020

stale bot commented Jan 17, 2020

Output JUnit XML metadata for CircleCi #12023

Output JUnit XML metadata for CircleCi #12023

Conversation

benbraou commented Jan 15, 2018 • edited

Summary

What this PR does:

facebook-github-bot commented Jan 15, 2018

facebook-github-bot commented Jan 15, 2018

nhunzaker commented Aug 3, 2018

benbraou commented Aug 5, 2018

nhunzaker commented Aug 5, 2018

benbraou commented Aug 15, 2018

nhunzaker Aug 17, 2018

Choose a reason for hiding this comment

benbraou Aug 18, 2018

Choose a reason for hiding this comment

nhunzaker Aug 17, 2018

Choose a reason for hiding this comment

benbraou Aug 18, 2018

Choose a reason for hiding this comment

nhunzaker Aug 17, 2018

Choose a reason for hiding this comment

benbraou Aug 18, 2018

Choose a reason for hiding this comment

nhunzaker commented Aug 17, 2018

benbraou commented Aug 18, 2018

stale bot commented Jan 10, 2020

stale bot commented Jan 17, 2020

benbraou commented Jan 15, 2018 •

edited