Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(test): added tests for multiple lxd server versions [WD-8616] #653

Merged
merged 1 commit into from
Feb 20, 2024

Conversation

mas-who
Copy link
Contributor

@mas-who mas-who commented Feb 16, 2024

Done

  • Adjusted CI to run e2e tests against multiple lxd server versions
  • Added tests for lxd v5.0/stable
  • Adjusted existing tests with features that does not exist in lxd v5.0/stable to be skipped
  • Added test fixture to annotate tests with lxd version

QA

  1. Run the LXD-UI:
    • On the demo server via the link posted by @webteam-app below. This is only available for PRs created by collaborators of the repo. Ask @lorumic or @edlerd for access.
    • With a local copy of this branch, run as described here.
  2. Perform the following QA steps:
    • ensure CI passes
    • check that the output playwright report looks fine

@webteam-app
Copy link

Demo starting at https://lxd-ui-653.demos.haus

@mas-who mas-who changed the title chore(test): added tests for multiple lxd server versions chore(test): added tests for multiple lxd server versions [WD-8616] Feb 16, 2024
@mas-who mas-who force-pushed the multi-lxd-version-test branch 2 times, most recently from e79994a to e0820a7 Compare February 16, 2024 13:55
tests/iso-volumes.spec.ts Show resolved Hide resolved
tests/projects.spec.ts Show resolved Hide resolved
tests/fixtures/lxd-test.ts Outdated Show resolved Hide resolved
@mas-who mas-who force-pushed the multi-lxd-version-test branch 6 times, most recently from b752a68 to 8875114 Compare February 19, 2024 08:39
Copy link
Collaborator

@edlerd edlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As just discussed in person: Let's combine the browser and lxd-backend verison as project, that is communicated to playwright. This way we can avoid passing the LXD_VERSION as environment variable and get it on the playwright report on top level.

tests/fixtures/lxd-test.ts Outdated Show resolved Hide resolved
.github/workflows/pr.yaml Outdated Show resolved Hide resolved
Signed-off-by: Mason Hu <mason.hu@canonical.com>
Copy link
Collaborator

@edlerd edlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA from the report looks good. One question for a possible simplification below.

"test-js": "vitest --run"
"test-js": "vitest --run",
"test-e2e-edge": "npx playwright test --project chromium:lxd-latest-edge firefox:lxd-latest-edge",
"test-e2e-stable": "npx playwright test --project chromium:lxd-5.0-stable firefox:lxd-5.0-stable"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice additions here 👍

};

export const test = base.extend<TestOptions>({
lxdVersion: ["latest-edge", { option: true }],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am confused as the value passed here seems to be hardcoded. Is the actual value coming from the use block in playwright config? Maybe we don't need the value here, then?

Though, please keep this lxd-test file, because I am going to need it for the coverage very soon -- even if it turns out to be empty and just passes the base test at this point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The value set here is the default for the project use parameter. I referenced the playwright docs for parameterising projects. I think { option: true } is necessary but not sure if we can pass in a empty value for the first parameter of the array. Although passing in an empty value will look strange as well I rate, wdyt?

Copy link
Collaborator

@edlerd edlerd Feb 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we need this definition here, I think keeping latest-edge as a default is reasonable.

Maybe we should document the test setup, as it lately became a bit more complex. Could you add a future story to extend the architecture file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do, I did add a bit of context in the HACKING.md, but I think adding to the architecture file is also necessary. Will create a task for it 👍

Copy link
Collaborator

@edlerd edlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I like the solution :)

@edlerd edlerd merged commit d4abd51 into canonical:main Feb 20, 2024
10 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants