One playwright config to rule them all by zomars · Pull Request #4072 · calcom/cal.diy

zomars · 2022-09-01T02:56:19Z

What does this PR do?

Consolidates App Store and Web E2E test config
Planning on consolidating embed as well in a follow-up
Makes correct usage of Playwright's Project config
Stops using turbo for testing as there was little benefit and it was getting messy
Add special .e2e.ts file naming to distinguish between Unit and E2E tests for App Store only, I'm planning to adopt this in the whole monorepo later on

NOTE: Tests are failing because GitHub is trying to run the old YAML files

Type of change

Chore (refactoring code, technical debt, workflow improvements)

How should this be tested?

yarn test
yarn test-e2e
yarn test-e2e:app-store

vercel · 2022-09-01T02:56:23Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
cal	✅ Ready (Inspect)	Visit Preview	Sep 2, 2022 at 1:51AM (UTC)
swagger	❌ Failed (Inspect)		Sep 2, 2022 at 1:51AM (UTC)

zomars · 2022-09-01T03:05:23Z

-
-export * from "@calcom/app-store/_apps-playwright/lib/testUtils";
-export async function cleanUpForms() {
-  await prisma.app_RoutingForms_Form.deleteMany({


No need for this since deleting test users cascades forms as well

zomars · 2022-09-01T03:05:31Z

-}
-
-export async function cleanUpSeededForm(formId: string) {
-  return await prisma.app_RoutingForms_FormResponse.deleteMany({


zomars · 2022-09-01T03:19:13Z

+      - run: yarn workspace @calcom/prisma db-seed
+      - run: yarn build


No need for turbo here

zomars · 2022-09-01T03:20:05Z

@@ -1,7 +1,7 @@
 name: E2E Test - Integrations with Third Party
 on:


This was running the same command as normal yarn test-e2e am I missing something? @hariombalhara

zomars · 2022-09-01T03:20:29Z

+            - job: test
+              paths:
+                - /apps/web/**
+                - /packages/**


Jest runs in parallel now. Doesn't depend on E2E

zomars · 2022-09-01T03:20:50Z

-    "test-e2e-integrations": "NEXT_PUBLIC_IS_E2E=1 yarn playwright test --config=playwright-integrations/config/playwright.config.ts --project=chromium",
-    "test-e2e-integrations-quick": "QUICK=true E2E_DEV_SERVER=1 yarn test-e2e-integrations",
-    "db-setup-tests": "dotenv -e ./test/.env.test -- yarn workspace @calcom/prisma prisma generate",
-    "playwright-report": "playwright show-report playwright/reports/playwright-html-report",


No more individual runners

zomars · 2022-09-01T03:21:56Z

-  "scripts": {
-    "app-e2e": "yarn playwright test --config=playwright/config/playwright.config.ts",
-    "app-e2e-quick": "QUICK=true yarn app-e2e"
-  },


Apps don't need to setup their own runners anymore. You just need to add a *.e2e.ts file and playwright should pick it up.

hariombalhara · 2022-09-01T04:05:01Z

@zomars To get the status of tests for the branch, maybe we can add fixes/e2e-consolidation as branch push target

hariombalhara · 2022-09-01T04:07:10Z

I like the idea of having just one playwright.config 🙌

zlwaterfield

This looks good! I do agree with @hariombalhara, it might be good to update the actions config so the E2E can run on this branch to make sure they pass before merging.

zomars · 2022-09-01T17:36:18Z

@zlwaterfield @hariombalhara push tests are passing now 🙏🏽

leog

yarn test worked perfectly
For yarn test-e2e I had to run npx playwright install first which makes sense, and then got an error that I had to build before running it. Did that and got an error regarding the database: Please make sure your database server is running at localhost:5450. This try was with PLAYWRIGHT_HEADLESS=true.
For yarn test-e2e:app-store I got time outs. Also run with PLAYWRIGHT_HEADLESS=true.

Really great job on putting effort here to make everything more understandable and accessible. The much needed groundwork for your suggestion around having a Testing Expert.

zomars · 2022-09-01T21:20:15Z

@leog you're right leog. This is a breaking change since yarn test-e2e doesn't run the seeder and build anymore

zomars added 4 commits August 31, 2022 15:37

Migrates api tests configs

e5303e4

Allows root test to run in parallel before e2e

21b023f

Replaces deprecated turbo syntax

5c4737b

Centralizes playwright config

11ebdb7

vercel bot had a problem deploying to Preview – swagger September 1, 2022 02:59 Failure

vercel bot deployed to Preview – cal September 1, 2022 03:05 View deployment

zomars commented Sep 1, 2022

View reviewed changes

WIP

364580e

zomars commented Sep 1, 2022

View reviewed changes

vercel bot had a problem deploying to Preview – swagger September 1, 2022 03:21 Failure

zomars commented Sep 1, 2022

View reviewed changes

vercel bot deployed to Preview – cal September 1, 2022 03:25 View deployment

zomars added 2 commits August 31, 2022 21:27

Merge branch 'main' into fixes/e2e-consolidation

0858f78

Update turbo.json

ef2379d

zomars requested review from emrysal, hariombalhara and zlwaterfield September 1, 2022 03:28

zomars marked this pull request as ready for review September 1, 2022 03:33

zomars changed the title ~~Fixes/e2e consolidation~~ One playwright config to rule them all Sep 1, 2022

zomars added the 👀 ready for review label Sep 1, 2022

zomars added this to the v.2.0 milestone Sep 1, 2022

zomars self-assigned this Sep 1, 2022

zomars requested a review from a team September 1, 2022 03:34

vercel bot deployed to Preview September 1, 2022 03:44 View deployment

zlwaterfield reviewed Sep 1, 2022

View reviewed changes

zomars added 5 commits September 1, 2022 10:51

Merge branch 'main' into fixes/e2e-consolidation

b6f8040

Updates checks to run on this PR

25f17b7

Update jest.config.ts

11d4113

Fixes tests seeding

dda6c43

Update test.yml

ec411cd

vercel bot deployed to Preview September 1, 2022 17:29 View deployment

Re-enables playwirght caching on CI

c324c25

zlwaterfield approved these changes Sep 1, 2022

View reviewed changes

leog reviewed Sep 1, 2022

View reviewed changes

zomars added 4 commits September 1, 2022 18:17

No more seeded data in e2e tests

1d7c013

Renames e2e tests

462e951

Updates snapshot

0e32f1b

Update integrations.e2e.ts

037b310

vercel bot deployed to Preview September 2, 2022 00:52 View deployment

Workaround for SAML test

2c315f8

vercel bot deployed to Preview September 2, 2022 01:51 View deployment

zomars merged commit d27b7ab into main Sep 2, 2022

zomars deleted the fixes/e2e-consolidation branch September 2, 2022 02:00

		- run: yarn workspace @calcom/prisma db-seed
		- run: yarn build

		@@ -1,7 +1,7 @@
		name: E2E Test - Integrations with Third Party
		on:

Conversation

zomars commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Type of change

How should this be tested?

Uh oh!

vercel bot commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

zomars Sep 1, 2022

Choose a reason for hiding this comment

Uh oh!

hariombalhara commented Sep 1, 2022

Uh oh!

hariombalhara commented Sep 1, 2022

Uh oh!

zlwaterfield left a comment

Choose a reason for hiding this comment

Uh oh!

zomars commented Sep 1, 2022

Uh oh!

leog left a comment

Choose a reason for hiding this comment

Uh oh!

zomars commented Sep 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zomars commented Sep 1, 2022 •

edited

Loading

vercel bot commented Sep 1, 2022 •

edited

Loading