-
Notifications
You must be signed in to change notification settings - Fork 25
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: test examples in real browsers (#242)
Pulls example tests out to own dir so we can run the same test for each ui library! fixes: #222 You can see it run in CI at https://github.com/web3-storage/w3ui/actions/runs/3901558574/jobs/6663550379#step:6:13 and you can try it out locally from this branch by running: ``` $ pnpm install # one time download of test browsers $ pnpx playwright install # fails for vanilla currently, but is ok, proceed. $ pnpm build:examples $ pnpm test:examples Scope: 20 of 33 workspace projects examples/test/playwright test$ playwright test [30 lines collapsed] │ [30/33] [webkit] › sign-up-in.spec.ts:4:3 › vue: sign in │ [31/33] [webkit] › uploads-list.spec.ts:4:3 › react: uploads list │ [32/33] [webkit] › uploads-list.spec.ts:4:3 › solid: uploads list │ [33/33] [webkit] › uploads-list.spec.ts:4:3 › vue: uploads list │ │ 33 passed (7s) │ To open last HTML report run: │ │ npx playwright show-report │ └─ Done in 7.6s ``` _note: the build fails currently as our vanilla examples fail to build currently, which is why we need these tests!_ Adds and updates run scripts to make it easy to run package or example tests. Most of the time we'd want to run package tests locally, so this is set as the default. ```bash # runs package tests $ pnpm test # builkd examples. must be run before test so the dist dir exits for each $ pnpm build:examples # run example test $ pnpm test:examples # serve examples, so you can poke around as a dev. $ pnpm serve:examples ``` Updates examples to produce relative URLs by setting `base: ''` in vite config. This allows us to serve all the examples from a single static server in the `examples` dir, and `serve` is added with a `serve.json` config to rewrite the paths so the `dist` dir is served for each example. Adds playwright config to fire up the example server Adds github workflow to set up and run the browser tests only if we make a change to the examples. We may change this to run for every change in the future, as we should probably check if the examples still work if we change a package! Reduces the scope of our local github action to just handle setting up node and pnpm which is needed by every workflow. `lint` and `test` are now specified explicitly in workflows that should run them. We don't need to run those steps when we're doing browser testing. - Vanilla JS examples are not currently tested as they fail to build #243 - we can fix in a separate PR. - We need a mechanism to be able to register a space from browser, or add pre-registered test account fixture - #244 - We are missing a `multi-file-upload` example - #245 <img width="707" alt="Screenshot 2023-01-12 at 02 03 41" src="https://user-images.githubusercontent.com/58871/211958353-b4a587a7-ead7-4957-b028-45b17c41f9a3.png"> <img width="1582" alt="Screenshot 2023-01-12 at 01 57 39" src="https://user-images.githubusercontent.com/58871/211957738-0bfad4e5-a0e4-4ba6-8a13-ee05602f3adc.png"> **whoo!** <img width="871" alt="Screenshot 2023-01-12 at 11 00 28" src="https://user-images.githubusercontent.com/58871/212049726-68f86951-c244-4ed3-bde9-4ca3bfaa25ca.png"> License: MIT Signed-off-by: Oli Evans <oli@protocol.ai> Signed-off-by: Oli Evans <oli@protocol.ai>
- Loading branch information
Showing
37 changed files
with
649 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
name: Examples | ||
on: | ||
push: | ||
branches: | ||
- main | ||
paths: | ||
- "examples/**" | ||
pull_request: | ||
branches: | ||
- main | ||
paths: | ||
- "examples/**" | ||
jobs: | ||
test: | ||
name: Test | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: ./.github/actions/pnpm | ||
- run: pnpm build:examples | ||
- run: pnpx playwright install --with-deps # install browsers | ||
- run: pnpm test:examples | ||
|
||
# save test report | ||
# - uses: actions/upload-artifact@v3 | ||
# if: always() | ||
# with: | ||
# name: playwright-report | ||
# path: playwright-report/ | ||
# retention-days: 30 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"unlisted": [ | ||
"README.md", | ||
"test", | ||
"serve.json", | ||
"template" | ||
], | ||
"rewrites": [ | ||
{ "source": "/:ui/:eg", "destination": "/:ui/:eg/dist/index.html" }, | ||
{ "source": "/:ui/:eg/", "destination": "/:ui/:eg/dist/index.html" }, | ||
{ "source": "/:ui/:eg/favicon.ico", "destination": "/:ui/:eg/dist/favicon.ico" }, | ||
{ "source": "/:ui/:eg/assets/:x", "destination": "/:ui/:eg/dist/assets/:x" } | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
node_modules/ | ||
/test-results/ | ||
/playwright-report/ | ||
/playwright/.cache/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# W3UI Playwright tests | ||
|
||
_Test our examples across all UI libraties we support, in real browsers!_ | ||
|
||
## Getting started | ||
|
||
Install the deps and the browsers from the root of the monorepo | ||
|
||
```bash | ||
# update deps | ||
$ pnpm i | ||
|
||
# fetch test browsers | ||
$ pnpx playwright install | ||
``` | ||
|
||
Build the examples to their many dist folders | ||
|
||
```bash | ||
$ pnpm build:examples | ||
``` | ||
|
||
Run the tests | ||
|
||
```bash | ||
$ pnpm test:examples | ||
|
||
examples/test/playwright test$ playwright test | ||
[30 lines collapsed] | ||
│ [30/33] [webkit] › sign-up-in.spec.ts:4:3 › vue: sign in | ||
│ [31/33] [webkit] › uploads-list.spec.ts:4:3 › react: uploads list | ||
│ [32/33] [webkit] › uploads-list.spec.ts:4:3 › solid: uploads list | ||
│ [33/33] [webkit] › uploads-list.spec.ts:4:3 › vue: uploads list | ||
│ | ||
│ 33 passed (7s) | ||
│ To open last HTML report run: | ||
│ | ||
│ npx playwright show-report | ||
│ | ||
└─ Done in 7.6s | ||
``` | ||
|
||
## Debugging | ||
|
||
You can run the test server manually to see what requests are made, and the tests will re-use your running server on http://localhost:1337 - it's powered by `serve` a static webserver running over the examples dir, configured in `examples/serve.json` | ||
|
||
```bash | ||
$ pnpm serve:examples | ||
``` | ||
|
||
You can also run the tests locally from this project, and have it show you the browser as the tests run to aid debugging | ||
|
||
```bash | ||
$ cd examples/test/playwright | ||
$ pnpm test | ||
|
||
# or to see what's going on in the browser as the test runs | ||
$ pnpm test:debug | ||
``` | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{ | ||
"name": "@w3ui/example-test-playwright", | ||
"version": "1.0.0", | ||
"type": "module", | ||
"scripts": { | ||
"serve": "serve ../../", | ||
"test": "playwright test", | ||
"test:debug": "playwright test --project=chromium --headed" | ||
}, | ||
"author": "olzilla", | ||
"license": "Apache-2.0 OR MIT", | ||
"devDependencies": { | ||
"@playwright/test": "^1.29.2", | ||
"serve": "^14.1.2" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,101 @@ | ||
import type { PlaywrightTestConfig } from '@playwright/test' | ||
import { devices } from '@playwright/test' | ||
|
||
/** | ||
* See https://playwright.dev/docs/test-configuration. | ||
*/ | ||
const config: PlaywrightTestConfig = { | ||
testDir: './src', | ||
timeout: 30 * 1000, | ||
expect: { | ||
/** | ||
* Maximum time expect() should wait for the condition to be met. | ||
* For example in `await expect(locator).toHaveText();` | ||
*/ | ||
timeout: 5000 | ||
}, | ||
/* Run tests in files in parallel */ | ||
fullyParallel: true, | ||
/* Fail the build on CI if you accidentally left test.only in the source code. */ | ||
forbidOnly: !!process.env.CI, // eslint-disable-line @typescript-eslint/strict-boolean-expressions | ||
/* Retry on CI only */ | ||
retries: process.env.CI ? 2 : 0, // eslint-disable-line @typescript-eslint/strict-boolean-expressions | ||
/* Opt out of parallel tests on CI. */ | ||
workers: process.env.CI ? 1 : undefined, // eslint-disable-line @typescript-eslint/strict-boolean-expressions | ||
/* Reporter to use. See https://playwright.dev/docs/test-reporters */ | ||
reporter: 'html', | ||
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */ | ||
use: { | ||
/* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */ | ||
actionTimeout: 0, | ||
/* Base URL to use in actions like `await page.goto('/')`. */ | ||
baseURL: 'http://localhost:1337', | ||
|
||
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */ | ||
trace: 'on-first-retry' | ||
}, | ||
|
||
/* Run your local dev server before starting the tests */ | ||
webServer: { | ||
reuseExistingServer: !process.env.CI, // eslint-disable-line @typescript-eslint/strict-boolean-expressions | ||
command: 'pnpm run serve -p 1337', | ||
port: 1337 | ||
}, | ||
|
||
/* Configure projects for major browsers */ | ||
projects: [ | ||
{ | ||
name: 'chromium', | ||
use: { | ||
...devices['Desktop Chrome'] | ||
} | ||
}, | ||
|
||
{ | ||
name: 'firefox', | ||
use: { | ||
...devices['Desktop Firefox'] | ||
} | ||
}, | ||
|
||
{ | ||
name: 'webkit', | ||
use: { | ||
...devices['Desktop Safari'] | ||
} | ||
} | ||
|
||
/* Test against mobile viewports. */ | ||
// { | ||
// name: 'Mobile Chrome', | ||
// use: { | ||
// ...devices['Pixel 5'], | ||
// }, | ||
// }, | ||
// { | ||
// name: 'Mobile Safari', | ||
// use: { | ||
// ...devices['iPhone 12'], | ||
// }, | ||
// }, | ||
|
||
/* Test against branded browsers. */ | ||
// { | ||
// name: 'Microsoft Edge', | ||
// use: { | ||
// channel: 'msedge', | ||
// }, | ||
// }, | ||
// { | ||
// name: 'Google Chrome', | ||
// use: { | ||
// channel: 'chrome', | ||
// }, | ||
// }, | ||
] | ||
|
||
/* Folder for test artifacts such as screenshots, videos, traces, etc. */ | ||
// outputDir: 'test-results/', | ||
} | ||
|
||
export default config |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
import { test, expect } from '@playwright/test' | ||
|
||
for (const ui of ['react', 'solid', 'vue'/* 'vanilla' not work atm */]) { | ||
test(`${ui}: file upload`, async ({ page }) => { | ||
await page.goto(`/${ui}/file-upload/`) | ||
await expect(page).toHaveTitle('W3UI File Upload Example App') | ||
|
||
const input = page.getByRole('textbox', { name: 'Email address:' }) | ||
await input.fill('test@example.org') | ||
await input.press('Enter') | ||
await expect(page.getByText('Verify your email address!')).toBeVisible() | ||
}) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
import { test, expect } from '@playwright/test' | ||
|
||
for (const ui of ['react', 'solid'/* `vue` not exit yet, 'vanilla' not work atm */]) { | ||
test(`${ui}: multi file upload`, async ({ page }) => { | ||
await page.goto(`/${ui}/multi-file-upload/`) | ||
await expect(page).toHaveTitle('W3UI Multi File Upload Example App') | ||
|
||
const input = page.getByRole('textbox', { name: 'Email address:' }) | ||
await input.fill('test@example.org') | ||
await input.press('Enter') | ||
await expect(page.getByText('Verify your email address!')).toBeVisible() | ||
}) | ||
} |
Oops, something went wrong.