Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: playwright coverage report showing incorrect results [WD-9070] #671

Merged
merged 1 commit into from
Feb 26, 2024
Merged

fix: playwright coverage report showing incorrect results [WD-9070] #671

merged 1 commit into from
Feb 26, 2024

Conversation

mas-who
Copy link
Contributor

@mas-who mas-who commented Feb 24, 2024

Done

  • Resolved issue with playwright coverage report not displaying all results.

QA

  1. Run the LXD-UI:
    • On the demo server via the link posted by @webteam-app below. This is only available for PRs created by collaborators of the repo. Ask @mas-who or @edlerd for access.
    • With a local copy of this branch, run as described here.
  2. Perform the following QA steps:
    • Run the test coverage script locally and inspect that the report produced is now correct

@mas-who
fix: playwright coverage report showing incorrect results
be86f86 
Signed-off-by: Mason Hu <mason.hu@canonical.com>
@webteam-app
Copy link

Demo starting at https://lxd-ui-671.demos.haus

mas-who
mas-who commented Feb 24, 2024
View reviewed changes
playwright.config.ts
@@ -23,7 +23,7 @@ const config: PlaywrightTestConfig<TestOptions> = {
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : 3,
workers: 1,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the tests reuse the same resources, running those tests in parallel will fail sometimes

mas-who
mas-who commented Feb 24, 2024
View reviewed changes
tests/fixtures/lxd-test.ts

test.afterEach(async ({ page, hasCoverage }) => {
await finishCoverage(page, hasCoverage);
runCoverage: [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I found the issue wasn't with nyc, turns out global beforeEach and afterEach hooks in playwright should be done this way instead of using test.beforeEach and test.afterEach. This was discovered after debugging where finishCoverage didn't run for every test.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent!

@mas-who mas-who changed the title fix: playwright coverage report showing incorrect results fix: playwright coverage report showing incorrect results [WD-9070] Feb 26, 2024
edlerd
edlerd approved these changes Feb 26, 2024
View reviewed changes
Copy link
Collaborator

@edlerd edlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for solving this!

tests/fixtures/lxd-test.ts

test.afterEach(async ({ page, hasCoverage }) => {
await finishCoverage(page, hasCoverage);
runCoverage: [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent!

@edlerd edlerd merged commit 95a38d3 into canonical:main Feb 26, 2024
10 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 26, 2024
workflow: remove all reports for PR #671
be83fd7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edlerd edlerd edlerd approved these changes

Assignees
No one assigned
Labels
Review: Code needed Review: QA needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@mas-who @webteam-app @edlerd