Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QA] Code coverage: unskip tests, collect tests results, exclude bundles from report #64477

Merged

Conversation

dmlemeshko
Copy link
Member

@dmlemeshko dmlemeshko commented Apr 25, 2020

Summary

This PR has some improvements for code coverage:

  • collect test results for functional tests with failure screenshots
  • run sed for coverage json files only before merge to speedup process
  • nyc config files for functional and jest tests

Coverage job to verify changes: https://kibana-ci.elastic.co/job/elastic+kibana+code-coverage/777/

@dmlemeshko dmlemeshko changed the title [skip CI] Improve code coverage reporting Improve code coverage reporting Apr 27, 2020
@dmlemeshko dmlemeshko force-pushed the improve-code-coverage-reporting branch from 9ad9c79 to de43ac9 Compare April 29, 2020 18:04
@dmlemeshko dmlemeshko changed the title Improve code coverage reporting Code coverage: unskip tests, collect tests results, exclude bundles from report Apr 29, 2020
@dmlemeshko dmlemeshko added backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes v8.0.0 labels Apr 30, 2020
Copy link
Contributor

@LeeDr LeeDr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - based on Jenkins pass

@dmlemeshko
Copy link
Member Author

@elasticmachine merge upstream

@dmlemeshko dmlemeshko force-pushed the improve-code-coverage-reporting branch from bfd9254 to 638e94e Compare May 6, 2020 13:51

echo "### Merge coverage reports"
for x in jest functional; do
yarn nyc report --temp-dir $COMBINED_EXTRACT_DIR/${x} --report-dir $EXTRACT_END_DIR/${x}-combined --reporter=html --reporter=json-summary
yarn nyc report --nycrc-path src/dev/code_coverage/nyc_config/nyc.${x}.config.js
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wayneseymour maybe you have an idea why it can fail with

17:58:16  ### Merge coverage reports
17:58:16  yarn run v1.22.4
17:58:16  $ /var/lib/jenkins/workspace/elastic+kibana+coverage-ingestion/kibana/node_modules/.bin/nyc report --nycrc-path src/dev/code_coverage/nyc_config/nyc.jest.config.js
17:58:16  ENOENT: no such file or directory, scandir '/var/lib/jenkins/workspace/elastic+kibana+coverage-ingestion/kibana/target/kibana-coverage/jest'
17:58:16  error Command failed with exit code 1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can list the contents of all the paths in question?

Copy link
Member Author

@dmlemeshko dmlemeshko Jun 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue was caused by mixed path during sed operation. I put it out of this PR, we anyway need to think about the smart way of replacing path in jsons

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is working fine now

@dmlemeshko dmlemeshko marked this pull request as ready for review May 15, 2020 10:20
@dmlemeshko dmlemeshko requested a review from a team as a code owner May 15, 2020 10:20
@dmlemeshko dmlemeshko requested a review from a team as a code owner May 15, 2020 18:03
@LeeDr
Copy link
Contributor

LeeDr commented May 18, 2020

@elasticmachine merge upstream

Copy link
Contributor

@spalger spalger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any issues from an operations perspective

test/functional/page_objects/common_page.ts Outdated Show resolved Hide resolved
@dmlemeshko
Copy link
Member Author

@elasticmachine merge upstream

@dmlemeshko dmlemeshko force-pushed the improve-code-coverage-reporting branch from c1b6461 to 2f3261f Compare June 4, 2020 17:56
@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@dmlemeshko
Copy link
Member Author

@wayneseymour could you check it? Coverage pipeline passed

@wayneseymour wayneseymour changed the title Code coverage: unskip tests, collect tests results, exclude bundles from report [QA] Code coverage: unskip tests, collect tests results, exclude bundles from report Jun 5, 2020
@wayneseymour wayneseymour merged commit 5064ae4 into elastic:master Jun 5, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jun 8, 2020
* master: (57 commits)
  Add app arch team as owner of datemath package (elastic#66880)
  [Observability] Landing page for Observability (elastic#67467)
  [SIEM] Fix timeline buildGlobalQuery (elastic#68320)
  Optimize saved objects getScopedClient and HTTP API (elastic#68221)
  [Maps] Fix mb-style interpolate style rule (elastic#68413)
  update script to always download node (elastic#68421)
  [SECURITY SOLEIL] Fix selection of event type when no siem index signal created (elastic#68291)
  [DOCS] Adds note about configuring File Data Visualizer (elastic#68407)
  [DOCS] Adds link from remote clusters to index patterns (elastic#68406)
  [QA] slack notify on failure (elastic#68126)
  upgrade eslint-plugin-react-hooks from 2.3.0 to 4.0.4 (elastic#68295)
  moving to jira to a gold license (elastic#67178)
  [DOCS] Revises doc on adding data (elastic#68038)
  [APM] Add ThemeProvider to support dark mode (elastic#68242)
  Make welcome screen disabling first action in loginIfPrompted (elastic#68238)
  [QA] Code coverage: unskip tests, collect tests results, exclude bundles from report (elastic#64477)
  [ML] Functional tests - disable flaky regression and classification creation test
  [Alerting] change eventLog ILM requests to absolute URLs (elastic#68331)
  Report page load asset size (elastic#66224)
  [SIEM][CASE] Change SIEM to Security (elastic#68365)
  ...
@dmlemeshko dmlemeshko deleted the improve-code-coverage-reporting branch January 31, 2022 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants