fix: Flakiness in location tests #13557
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
5 Ignored Deployments
|
|
Thank you for following the naming conventions! 🙏 Feel free to join our discord and post your PR link. |
📦 Next.js Bundle Analysis for @calcom/webThis analysis was generated by the Next.js Bundle Analysis action. 🤖 This PR introduced no changes to the JavaScript bundle! 🙌 |
Current Playwright Test Results Summary✅ 440 Passing - Run may still be in progress, this comment will be updated as current testing workflow or job completes... (Last updated on 02/06/2024 02:52:15pm UTC) Run DetailsRunning Workflow PR Update on Github Actions Commit: 0bdc380 Started: 02/06/2024 02:41:23pm UTC
|
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Booking With Multiple Email Question and Each Other Question Booking With Multiple Email Question and Address Question Multiple Email required and Address required
Retry 1 • Initial Attempt |
4.14% (12)12 / 290 runsfailed over last 7 days |
5.52% (16)16 / 290 runsflaked over last 7 days |
📄 apps/web/playwright/booking/longTextQuestion.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Booking With Long Text Question and Each Other Question Booking With Long Text Question and checkbox Question Long Text and checkbox required
Retry 1 • Initial Attempt |
0% (0)0 / 265 runsfailed over last 7 days |
5.66% (15)15 / 265 runsflaked over last 7 days |
📄 apps/web/playwright/booking/collectiveEventType/collectiveEventType.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Collective event type Book a Collective event type (with added guest)
Retry 1 • Initial Attempt |
0% (0)0 / 261 runsfailed over last 7 days |
2.68% (7)7 / 261 runsflaked over last 7 days |
📄 apps/web/playwright/booking/responsiveBooking.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Booking page with no questions Booking page with 640x480 resolution
Retry 1 • Initial Attempt |
0% (0)0 / 254 runsfailed over last 7 days |
5.51% (14)14 / 254 runsflaked over last 7 days |
📄 apps/web/playwright/booking/addressQuestione2e/addressQuestion.e2e.ts • 2 Flakes
Top 1 Common Error Messages
|
|
2 Test Cases Affected |
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Booking With Address Question and Each Other Question Booking With Address Question and Multi email Question Address and Multi email not required
Retry 1 • Initial Attempt |
0% (0)0 / 282 runsfailed over last 7 days |
6.74% (19)19 / 282 runsflaked over last 7 days |
|
Booking With Address Question and Each Other Question Booking With Address Question and Radio group Question Address and Radio group not required
Retry 1 • Initial Attempt |
0% (0)0 / 278 runsfailed over last 7 days |
5.04% (14)14 / 278 runsflaked over last 7 days |
📄 apps/web/playwright/event-types.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Event Types tests -- legacy user Different Locations Tests Can add Organzer Phone Number location and book with it
Retry 1 • Initial Attempt |
0% (0)0 / 295 runsfailed over last 7 days |
2.71% (8)8 / 295 runsflaked over last 7 days |
📄 apps/web/playwright/organization/organization-invitation.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Organization Email not matching orgAutoAcceptEmail Team invitation
Retry 1 • Initial Attempt |
15.71% (44)44 / 280 runsfailed over last 7 days |
14.29% (40)40 / 280 runsflaked over last 7 days |
📄 packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 8 Flakes
Top 1 Common Error Messages
|
|
8 Test Cases Affected |
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Popup Tests should open embed iframe on click - Configured with light theme
Retry 1 • Initial Attempt |
-3.53% (-10)-10 / 283 runsfailed over last 7 days |
48.06% (136)136 / 283 runsflaked over last 7 days |
|
Popup Tests should be able to reschedule
Retry 1 • Initial Attempt |
-87.18% (-136)-136 / 156 runsfailed over last 7 days |
87.18% (136)136 / 156 runsflaked over last 7 days |
|
Popup Tests should open Routing Forms embed on click
Retry 1 • Initial Attempt |
-75.64% (-118)-118 / 156 runsfailed over last 7 days |
78.21% (122)122 / 156 runsflaked over last 7 days |
|
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe according to system theme when no theme is configured through Embed API
Retry 1 • Initial Attempt |
-80.26% (-122)-122 / 152 runsfailed over last 7 days |
80.26% (122)122 / 152 runsflaked over last 7 days |
|
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe according to system theme when configured with 'auto' theme using Embed API
Retry 1 • Initial Attempt |
-80.26% (-122)-122 / 152 runsfailed over last 7 days |
80.26% (122)122 / 152 runsflaked over last 7 days |
|
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe(Booker Profile Page) with dark theme when configured with dark theme using Embed API
Retry 1 • Initial Attempt |
-80.26% (-122)-122 / 152 runsfailed over last 7 days |
80.26% (122)122 / 152 runsflaked over last 7 days |
|
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe(Event Booking Page) with dark theme when configured with dark theme using Embed API
Retry 1 • Initial Attempt |
-80.26% (-122)-122 / 152 runsfailed over last 7 days |
80.26% (122)122 / 152 runsflaked over last 7 days |
|
Popup Tests prendered embed should be loaded and apply the config given to it
Retry 1 • Initial Attempt |
-80.26% (-122)-122 / 152 runsfailed over last 7 days |
80.26% (122)122 / 152 runsflaked over last 7 days |
📄 apps/web/playwright/booking/phoneQuestion.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Booking With Phone Question and Each Other Question Booking With Phone Question and Address Question Booking With Phone Question and select Question Phone and select required
Retry 1 • Initial Attempt |
0.37% (1)1 / 272 runfailed over last 7 days |
5.88% (16)16 / 272 runsflaked over last 7 days |
📄 apps/web/playwright/reschedule.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Reschedule Tests Unpaid rescheduling should go to payment page
Retry 1 • Initial Attempt |
2.40% (7)7 / 292 runsfailed over last 7 days |
1.03% (3)3 / 292 runsflaked over last 7 days |
📄 packages/embeds/embed-core/playwright/tests/inline.e2e.ts • 1 Flake
Test Case Results
| Test Case | Last 7 days Failures | Last 7 days Flakes |
|---|---|---|
|
Inline Iframe Inline Iframe - Configured with Dark Theme
Retry 1 • Initial Attempt |
0.36% (1)1 / 281 runfailed over last 7 days |
38.43% (108)108 / 281 runsflaked over last 7 days |
exception
approved these changes
Feb 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes # (issue) Test failure messages similar to this
Type of change