-
Notifications
You must be signed in to change notification settings - Fork 6.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: flaky webhook e2e test #14850
fix: flaky webhook e2e test #14850
Conversation
Thank you for following the naming conventions! 🙏 Feel free to join our discord and post your PR link. |
The latest updates on your projects. Learn more about Vercel for Git ↗︎
2 Ignored Deployments
|
Graphite Automations"Add consumer team as reviewer" took an action on this PR • (05/02/24)1 reviewer was added to this PR based on Keith Williams's automation. |
📦 Next.js Bundle Analysis for @calcom/webThis analysis was generated by the Next.js Bundle Analysis action. 🤖 This PR introduced no changes to the JavaScript bundle! 🙌 |
Current Playwright Test Results Summary✅ 316 Passing - Run may still be in progress, this comment will be updated as current testing workflow or job completes... (Last updated on 05/03/2024 02:57:44pm UTC) Run DetailsRunning Workflow PR Update on Github Actions Commit: 7987c4c Started: 05/03/2024 02:52:47pm UTC
|
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Event Types tests -- legacy user Different Locations Tests Can remove location from multiple locations that are saved
Retry 2 • Retry 1 • Initial Attempt |
6.20% (16)16 / 258 runsfailed over last 7 days |
14.34% (37)37 / 258 runsflaked over last 7 days |
📄 apps/web/playwright/teams.e2e.ts • 1 Flake
Test Case Results
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Teams - NonOrg -- future Team Onboarding Invite Members
Retry 1 • Initial Attempt |
9% (26)26 / 289 runsfailed over last 7 days |
25.95% (75)75 / 289 runsflaked over last 7 days |
📄 packages/embeds/embed-core/playwright/tests/preview.e2e.ts • 1 Flake
Test Case Results
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Preview Preview - embed-core should load if correct embedLibUrl is provided
Retry 1 • Initial Attempt |
0% (0)0 / 29 runsfailed over last 7 days |
34.48% (10)10 / 29 runsflaked over last 7 days |
📄 packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 9 Flakes
Top 1 Common Error Messages
|
9 Test Cases Affected |
Test Case Results
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Popup Tests should open embed iframe on click - Configured with light theme
Retry 1 • Initial Attempt |
12.81% (36)36 / 281 runsfailed over last 7 days |
50.53% (142)142 / 281 runsflaked over last 7 days |
Popup Tests should be able to reschedule
Retry 1 • Initial Attempt |
-137.86% (-142)-142 / 103 runsfailed over last 7 days |
137.86% (142)142 / 103 runsflaked over last 7 days |
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe according to system theme when no theme is configured through Embed API
Retry 1 • Initial Attempt |
-136.27% (-139)-139 / 102 runsfailed over last 7 days |
136.27% (139)139 / 102 runsflaked over last 7 days |
Popup Tests should open Routing Forms embed on click
Retry 1 • Initial Attempt |
-139.22% (-142)-142 / 102 runsfailed over last 7 days |
139.22% (142)142 / 102 runsflaked over last 7 days |
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe according to system theme when configured with 'auto' theme using Embed API
Retry 1 • Initial Attempt |
-136.27% (-139)-139 / 102 runsfailed over last 7 days |
136.27% (139)139 / 102 runsflaked over last 7 days |
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe(Event Booking Page) with dark theme when configured with dark theme using Embed API
Retry 1 • Initial Attempt |
-136.27% (-139)-139 / 102 runsfailed over last 7 days |
136.27% (139)139 / 102 runsflaked over last 7 days |
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe(Booker Profile Page) with dark theme when configured with dark theme using Embed API
Retry 1 • Initial Attempt |
-136.27% (-139)-139 / 102 runsfailed over last 7 days |
136.27% (139)139 / 102 runsflaked over last 7 days |
Popup Tests prendered embed should be loaded and apply the config given to it
Retry 1 • Initial Attempt |
-136.27% (-139)-139 / 102 runsfailed over last 7 days |
136.27% (139)139 / 102 runsflaked over last 7 days |
Popup Tests should open on clicking child element
Retry 1 • Initial Attempt |
-136.36% (-120)-120 / 88 runsfailed over last 7 days |
136.36% (120)120 / 88 runsflaked over last 7 days |
📄 apps/web/playwright/organization/booking.e2e.ts • 1 Flake
Test Case Results
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Bookings Team Event Can create a booking for Round Robin EventType
Retry 1 • Initial Attempt |
16.20% (46)46 / 284 runsfailed over last 7 days |
24.65% (70)70 / 284 runsflaked over last 7 days |
📄 packages/app-store/routing-forms/playwright/tests/basic.e2e.ts • 1 Flake
Test Case Results
Test Case | Last 7 days Failures | Last 7 days Flakes |
---|---|---|
Routing Forms Seeded Routing Form Router URL should work
Retry 1 • Initial Attempt |
0% (0)0 / 273 runsfailed over last 7 days |
9.89% (27)27 / 273 runsflaked over last 7 days |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -427,7 +427,7 @@ test.describe("BOOKING_RESCHEDULED", async () => { | |||
|
|||
await page.locator('[data-testid="confirm-reschedule-button"]').click(); | |||
|
|||
await expect(page).toHaveURL(/.*booking/); | |||
await expect(page.getByTestId("success-page")).toBeVisible(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the possible reason for toHaveUrl
to be not working?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure why it happened, but the issue was that sometimes .toHaveURL(/.*booking/)
was already satisfied but the booking was not yet in the db
What does this PR do?
Fixes flaky webhook e2e tests.
Run:
yarn e2e webhook.e2e.ts