test: testing workflow triggers #12823

CarinaWolli · 2023-12-15T22:27:20Z

What does this PR do?

Add tests for testing workflow triggers when new booking is created and booking is rescheduled.

Follow up issues:
#12882
#12881

Type of change

Tests (Unit/Integration/E2E or any other test)

How should this be tested?

Run:

yarn test fresh-booking.test.ts
yarn test reschedule.test.ts

Mandatory Tasks

Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected.

vercel · 2023-12-15T22:27:29Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
api	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 20, 2023 6:32pm
dev	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Dec 20, 2023 6:32pm

5 Ignored Deployments

Name	Status	Preview	Updated (UTC)
ai	⬜️ Ignored (Inspect)	Visit Preview	Dec 20, 2023 6:32pm
cal	⬜️ Ignored (Inspect)	Visit Preview	Dec 20, 2023 6:32pm
cal-demo	⬜️ Ignored (Inspect)	Visit Preview	Dec 20, 2023 6:32pm
qa	⬜️ Ignored (Inspect)	Visit Preview	Dec 20, 2023 6:32pm
ui	⬜️ Ignored (Inspect)	Visit Preview	Dec 20, 2023 6:32pm

github-actions · 2023-12-15T22:27:34Z

Thank you for following the naming conventions! 🙏 Feel free to join our discord and post your PR link.

github-actions · 2023-12-15T22:34:54Z

📦 Next.js Bundle Analysis for @calcom/web

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

deploysentinel · 2023-12-16T01:07:08Z

Current Playwright Test Results Summary

✅ 334 Passing - ❌ 1 Failing - ⚠️ 19 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 12/18/2023 03:26:42pm UTC)

Run Details

Running Workflow PR Update on Github Actions

Commit: 0efc646

Started: 12/18/2023 03:17:26pm UTC

❌ Failures

📄 packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 1 Failure

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Popup Tests should be able to reschedule Retry 2 • Retry 1 • Initial Attempt Error: Timed out 30000ms waiting for expect(received).toBeVisible()... `Timed out 30000ms waiting for expect(received).toBeVisible() Call log: - expect.toBeVisible with timeout 30000ms - waiting for locator('[data-testid=success-page]') - waiting for locator('[data-testid=success-page]')`	13.97% (38) 38 / 272 runs failed over last 7 days	80.15% (218) 218 / 272 runs flaked over last 7 days

⚠️ Flakes

📄 apps/web/playwright/team/team-invitation.e2e.ts • 2 Flakes

Top 1 Common Error Messages

`null`	2 Test Cases Affected

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Team Invitation (non verified) Retry 1 • Initial Attempt	4.12% (11) 11 / 267 runs failed over last 7 days	93.26% (249) 249 / 267 runs flaked over last 7 days
Team Invitation (verified) Retry 1 • Initial Attempt	0% (0) 0 / 267 runs failed over last 7 days	94.76% (253) 253 / 267 runs flaked over last 7 days

📄 apps/web/playwright/locale.e2e.ts • 13 Flakes

Top 1 Common Error Messages

`null`	13 Test Cases Affected

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
unauthorized user sees correct translations (de) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (ar) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (zh) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (zh-CN) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (zh-TW) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (pt) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (pt-br) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
unauthorized user sees correct translations (es-419) should use correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
authorized user sees correct translations (de) should return correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
authorized user sees correct translations (pt-br) should return correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
authorized user sees correct translations (ar) should return correct translations and html attributes Retry 1 • Initial Attempt	-12.64% (-34) -34 / 269 runs failed over last 7 days	12.64% (34) 34 / 269 runs flaked over last 7 days
authorized user sees changed translations (de->ar) should return correct translations and html attributes Retry 1 • Initial Attempt	-2.97% (-8) -8 / 269 runs failed over last 7 days	11.52% (31) 31 / 269 runs flaked over last 7 days
authorized user sees changed translations (de->pt-BR) [locale1] should return correct translations and html attributes Retry 1 • Initial Attempt	-8.54% (-21) -21 / 246 runs failed over last 7 days	11.79% (29) 29 / 246 runs flaked over last 7 days

📄 apps/web/playwright/event-types.e2e.ts • 1 Flake

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Event Types tests user -- legacy Different Locations Tests can add single organizer address location without display location public option Retry 1 • Initial Attempt	0% (0) 0 / 245 runs failed over last 7 days	1.22% (3) 3 / 245 runs flaked over last 7 days

📄 packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 3 Flakes

Top 1 Common Error Messages

`null`	3 Test Cases Affected

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Popup Tests should open embed iframe on click - Configured with light theme Retry 1 • Initial Attempt	1.10% (3) 3 / 272 runs failed over last 7 days	61.03% (166) 166 / 272 runs flaked over last 7 days
Popup Tests Floating Button Popup Pro User - Configured in App with default setting of system theme should open embed iframe according to system theme when no theme is configured through Embed API Retry 1 • Initial Attempt	0.37% (1) 1 / 272 run failed over last 7 days	83.82% (228) 228 / 272 runs flaked over last 7 days
Popup Tests should open Routing Forms embed on click Retry 1 • Initial Attempt	0.37% (1) 1 / 272 run failed over last 7 days	50.37% (137) 137 / 272 runs flaked over last 7 days

View Detailed Build Results

CarinaWolli · 2023-12-19T20:49:40Z

apps/web/test/utils/bookingScenario/expects.ts

+  emails: Fixtures["emails"];
+  organizer: { email: string; name: string; timeZone: string };
+}) {
+  const subjectPattern = /^Reminder: /i;


For now I am only testing workflows with reminder template, so subject will always start with 'Reminder: '

CarinaWolli · 2023-12-19T20:51:35Z

packages/features/bookings/lib/handleNewBooking/test/fresh-booking.test.ts

@@ -1374,7 +1439,7 @@ describe("handleNewBooking", () => {
            status: BookingStatus.PENDING,
          });

-          expectWorkflowToBeTriggered();
+          expectWorkflowToBeNotTriggered({ organizer, emails });


status is pending because event type requires confirmation, workflow is triggered when booking is confirmed

CarinaWolli · 2023-12-19T20:52:25Z

packages/features/bookings/lib/handleNewBooking/test/recurring-event.test.ts

@@ -202,7 +202,7 @@ describe("handleNewBooking", () => {
          });
        }

-        expectWorkflowToBeTriggered();
+        // expectWorkflowToBeTriggered();


Will do this in a follow-up: #12882

CarinaWolli · 2023-12-19T20:53:12Z

packages/features/bookings/lib/handleNewBooking/test/reschedule.test.ts

@@ -815,7 +850,7 @@ describe("handleNewBooking", () => {
              },
            });

-            expectWorkflowToBeTriggered();
+            // expectWorkflowToBeTriggered({emails, organizer});


this fails and is a bug, I created an issue for it: #12881

CarinaWolli · 2023-12-19T20:53:40Z

...ages/features/bookings/lib/handleNewBooking/test/team-bookings/collective-scheduling.test.ts

@@ -198,7 +198,7 @@ describe("handleNewBooking", () => {
              ],
            });

-            expectWorkflowToBeTriggered();
+            // expectWorkflowToBeTriggered();


also follow up: #12882

CarinaWolli · 2023-12-20T18:10:11Z

packages/features/ee/workflows/lib/reminders/providers/sendgridProvider.ts

+
+  const testMode = process.env.NEXT_PUBLIC_IS_E2E || process.env.INTEGRATION_TEST_MODE ? true : false;
+  if (testMode) {
+    if (!mailData.sendAt) {


for now only 'when event is booked' trigger is tested, no scheduling of email in test mode possible and needed for now

CarinaWolli · 2023-12-20T18:11:23Z

turbo.json

@@ -327,6 +327,7 @@
    "TELEMETRY_DEBUG",
    "TWILIO_MESSAGING_SID",
    "TWILIO_PHONE_NUMBER",
+    "TWILIO_WHATSAPP_PHONE_NUMBER",


this was missing, couldn't commit without that. Not sure why this was no issue before

CarinaWolli added 8 commits December 13, 2023 17:57

add workflows to bookingScenario

7933cd6

activate sandbox mode for unit/integreation tests

f863ab4

add sendgrid specific code to SendgridProvider

73a0166

Refactor WIP

eaded74

remove duplicate sendgridProvider file

0b931da

first implementation for testing workflows

81c348f

Merge branch 'main' into tests/workflows

f2f3856

revert unintended changes

2c4b04c

zomars added the core area: core, team members only label Dec 15, 2023

vercel bot deployed to Preview – api December 15, 2023 22:32 View deployment

vercel bot deployed to Preview – dev December 15, 2023 22:45 View deployment

comment out Workflow trigger tests

db79de7

vercel bot deployed to Preview – api December 16, 2023 00:38 View deployment

move sendgrid check after test mode

6c7766f

vercel bot deployed to Preview – api December 16, 2023 00:47 View deployment

vercel bot deployed to Preview – dev December 16, 2023 00:50 View deployment

CarinaWolli added 2 commits December 15, 2023 19:56

Update signup.tsx

f0032e9

fix esLint

0623fe9

vercel bot deployed to Preview – api December 16, 2023 01:05 View deployment

vercel bot deployed to Preview – dev December 16, 2023 01:13 View deployment

Merge branch 'main' into tests/workflows

0efc646

vercel bot deployed to Preview – ai December 18, 2023 15:14 View deployment

vercel bot deployed to Preview – api December 18, 2023 15:15 View deployment

vercel bot deployed to Preview – dev December 18, 2023 15:19 View deployment

Merge branch 'main' into tests/workflows

3056972

vercel bot deployed to Preview – ai December 19, 2023 19:24 View deployment

vercel bot deployed to Preview – dev December 19, 2023 19:29 View deployment

CarinaWolli added 3 commits December 19, 2023 14:39

test webhooks on all tests in fresh-booking.test.ts

0c4c8a7

fix subjectPattern as title can be different

5e6411f

add workflow tests to reschedule.test.ts

5ced759

vercel bot had a problem deploying to Preview – dev December 19, 2023 20:31 Failure

vercel bot deployed to Preview – api December 19, 2023 20:36 View deployment

code clean up

ce50450

vercel bot had a problem deploying to Preview – dev December 19, 2023 20:49 Failure

vercel bot deployed to Preview – api December 19, 2023 20:53 View deployment

CarinaWolli commented Dec 19, 2023

View reviewed changes

code clean up

a1ffcac

vercel bot had a problem deploying to Preview – dev December 20, 2023 17:04 Failure

vercel bot deployed to Preview – api December 20, 2023 17:08 View deployment

fix sendgrid credentials missing message

99acfe1

vercel bot had a problem deploying to Preview – dev December 20, 2023 17:54 Failure

vercel bot deployed to Preview – api December 20, 2023 17:59 View deployment

CarinaWolli commented Dec 20, 2023

View reviewed changes

CarinaWolli marked this pull request as ready for review December 20, 2023 18:11

Merge branch 'main' into tests/workflows

65f1d20

CarinaWolli requested review from a team December 20, 2023 18:11

vercel bot deployed to Preview – ai December 20, 2023 18:15 View deployment

vercel bot deployed to Preview – api December 20, 2023 18:17 View deployment

vercel bot deployed to Preview – dev December 20, 2023 18:20 View deployment

code clean up

ada156b

vercel bot deployed to Preview – api December 20, 2023 18:30 View deployment

vercel bot deployed to Preview – dev December 20, 2023 18:32 View deployment

Udit-takkar approved these changes Dec 21, 2023

View reviewed changes

Udit-takkar merged commit 076868d into main Dec 21, 2023
24 checks passed

Udit-takkar deleted the tests/workflows branch December 21, 2023 13:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: testing workflow triggers #12823

test: testing workflow triggers #12823

CarinaWolli commented Dec 15, 2023 •

edited

vercel bot commented Dec 15, 2023 •

edited

github-actions bot commented Dec 15, 2023 •

edited

github-actions bot commented Dec 15, 2023 •

edited

deploysentinel bot commented Dec 16, 2023 •

edited

Test Case Results

Top 1 Common Error Messages

Test Case Results

Top 1 Common Error Messages

Test Case Results

Test Case Results

Top 1 Common Error Messages

Test Case Results

CarinaWolli Dec 19, 2023

CarinaWolli Dec 19, 2023

CarinaWolli Dec 19, 2023

CarinaWolli Dec 19, 2023

CarinaWolli Dec 19, 2023

CarinaWolli Dec 20, 2023

CarinaWolli Dec 20, 2023

test: testing workflow triggers #12823

test: testing workflow triggers #12823

Conversation

CarinaWolli commented Dec 15, 2023 • edited

What does this PR do?

Type of change

How should this be tested?

Mandatory Tasks

vercel bot commented Dec 15, 2023 • edited

github-actions bot commented Dec 15, 2023 • edited

github-actions bot commented Dec 15, 2023 • edited

📦 Next.js Bundle Analysis for @calcom/web

deploysentinel bot commented Dec 16, 2023 • edited

Current Playwright Test Results Summary

❌ Failures

Test Case Results

⚠️ Flakes

Top 1 Common Error Messages

Test Case Results

Top 1 Common Error Messages

Test Case Results

Test Case Results

Top 1 Common Error Messages

Test Case Results

CarinaWolli Dec 19, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 19, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 19, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 19, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 19, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 20, 2023

Choose a reason for hiding this comment

CarinaWolli Dec 20, 2023

Choose a reason for hiding this comment

CarinaWolli commented Dec 15, 2023 •

edited

vercel bot commented Dec 15, 2023 •

edited

github-actions bot commented Dec 15, 2023 •

edited

github-actions bot commented Dec 15, 2023 •

edited

deploysentinel bot commented Dec 16, 2023 •

edited