perf: /router endpoint perf improvement by hariombalhara · Pull Request #18366 · calcom/cal.diy

hariombalhara · 2024-12-24T09:46:41Z

What does this PR do?

Using possibly faster findUnique for form query instead of findFirst
Abstracted out handleResponse from response.handler.ts to be reused in /router - It allows already available form to be passed directly to handleResponse and avoids one sequential query to DB.
Passing orgId directly to findTeamMembersMatchingAttributeLogic which helps in restricting the number of records being queried in attributes related tables.
Completely different querying approach for getAttributesAssignmentData avoiding some queries in Prisma and using some in memory calculations.
Also added /router endpoint to vercel.json with more memory to give it more vCPUs
Moved /router and trpc /response endpoint out of app-store as it was not allowing to use the repositories. Prisma was getting imported client side - Note: Moving /router to app-router can be done later.
Already computed timings(and a few more) would be logged in Axiom now. Later to be moved to Sentry.
Sample
Server-Timing formQuery;dur=4.965374946594238, profileEnrichment;dur=1.2969579696655273, getSerializableForm;dur=14.026041984558105, ttgetAttributesQueryBuilderConfigHavingListofLabels;dur=0.6570830345153809, ttTeamMembersMatchingAttributeLogic;dur=23.702916860580444, ttGetAttributesForLogic;dur=292.05258297920227

Improvement seen locally:

With 2000 members in Acme and half of them being part of the Insights team, querying time went from 442ms(minimum) to 271ms(minimum). The improvements should be more visible with production data which has much much higher number of records. Note that even with 2000 members the time that we are optimizing is still low because total number of records across different tables are very few compared to production data.

Mandatory Tasks (DO NOT REMOVE)

I have self-reviewed the code (A decent size PR without self-review might be rejected).
I have updated the developer docs in /docs if this PR makes changes that would require a documentation change. If N/A, write N/A here and check the checkbox.
I confirm automated tests are in place that prove my fix is effective or that my feature works.

How should this be tested?

Test Preview of Routing Form
/router endpoint should also work
A regular routing form submission should also work

Followup

A teamMembershipId column in attributeToUser could possibly save some querying time, if in-memory computation for orgMembershipId to userId doesn't work

github-actions · 2024-12-24T09:46:56Z

Hey there and thank you for opening this pull request! 👋🏼

We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted.

Details:

No release type found in pull request title "Add stats and allow showing embed without loader". Add a prefix to indicate what kind of release this pull request corresponds to. For reference, see https://www.conventionalcommits.org/

Available types:
 - feat: A new feature
 - fix: A bug fix
 - docs: Documentation only changes
 - style: Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
 - refactor: A code change that neither fixes a bug nor adds a feature
 - perf: A code change that improves performance
 - test: Adding missing tests or correcting existing tests
 - build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
 - ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
 - chore: Other changes that don't modify src or test files
 - revert: Reverts a previous commit

vercel · 2024-12-24T09:46:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
cal	⬜️ Ignored (Inspect)	Visit Preview		Dec 27, 2024 7:09am
calcom-web-canary	⬜️ Ignored (Inspect)	Visit Preview		Dec 27, 2024 7:09am

hariombalhara · 2024-12-24T10:35:50Z

    );

+    // TODO: To be done using sentry tracing
+    console.log("Server-Timing", getServerTimingHeader(timeTaken));


timeTaken is already being computed, so I will just use the same approach for now for quick release. I will later move it to Sentry.

hariombalhara · 2024-12-24T10:37:24Z

-  const { createContext } = await import("@calcom/trpc/server/createContext");
-  const ctx = await createContext(context);
-
-  const { default: trpcRouter } = await import("@calcom/app-store/routing-forms/trpc/_router");
-  const caller = trpcRouter.createCaller(ctx);


Avoid using another trpcHandler and instead use the utility directly.

Also, needed to do this so that I can easily pass on already queried form to handleResponse and avoid repeat querying of same data

hariombalhara · 2024-12-24T11:09:27Z

+    include: {
+      team: {
+        select: {
+          parentId: true,


New requirement, it was already queried in /router endpoint for some other requirement

hariombalhara · 2024-12-24T11:37:13Z

-
-  const form = await prisma.app_RoutingForms_Form.findFirst({
+async function findFormById(formId: string, prisma: AppPrisma) {
+  return await prisma.app_RoutingForms_Form.findUnique({


Using findUnique instead of findFirst

github-actions · 2024-12-24T14:54:45Z

E2E results are ready!

Workflow #42449.2 latest results

hariombalhara · 2024-12-25T10:27:36Z

-        source: "/router/:path*",
-        destination: "/apps/routing-forms/router/:path*",
-      },


Rewrite not needed because /router is actually a route now

hariombalhara · 2024-12-25T10:31:16Z

    form: z.string(),
-    slug: z.string(),
-    pages: z.array(z.string()),
  })


Slug pages was there because it was earlier under app-store. Now it is not needed.

hariombalhara · 2024-12-25T12:53:19Z

+  const { form: formId, embed, ...fieldsResponses } = queryParsed.data;
+  const { currentOrgDomain } = orgDomainConfig(context.req);
+
+  let timeTaken: Record<string, number | null> = {};


Commented at other places as well but comment here as well. Time Taken is an existing approach that I am adding more data on. This later needs to move to Sentry performance tracing

hariombalhara · 2024-12-25T12:54:11Z

-    const result = await caller.public.response({
-      formId: form.id,
+    const result = await handleResponse({
+      form: serializableForm,


Passing already queried form now.

hariombalhara · 2024-12-25T12:56:33Z

-        : null;
+
+      const teamMembersMatchingAttributeLogicWithResult =
+        formTeamId && formOrgId


Team Members matching logic is applicable only for organization

…n embed

hariombalhara · 2024-12-26T06:18:43Z

    "pages/api/trpc/appRoutingForms/[trpc].ts": {
      "memory": 2048
+    },
+    "pages/router/embed.tsx": {


Not sure if page route will work but I am assuming it would.

Giving it more vCPUs, because there are a lot of computations that happen through this route's getServerSideProps -> handleResponse -> findTeamMembersMatchingAttributeLogic

We could add non embed route as well later but for now this is necessary.

graphite-app · 2024-12-26T06:30:11Z

Graphite Automations

"Add consumer team as reviewer" took an action on this PR • (12/26/24)

1 reviewer was added to this PR based on Keith Williams's automation.

"Add foundation team as reviewer" took an action on this PR • (12/26/24)

1 reviewer was added to this PR based on Keith Williams's automation.

Udit-takkar

Screen.Recording.2024-12-26.at.1.53.45.PM.mov

Whenever i type a letter the input looses focus and i have to click again to type the next letter.

Not sure if this bug was introduced in this PR

hariombalhara · 2024-12-26T08:53:26Z

Screen.Recording.2024-12-26.at.1.53.45.PM.mov
Whenever i type a letter the input looses focus and i have to click again to type the next letter.

Not sure if this bug was introduced in this PR

That is weird. I don't think I touched anything in there.

keithwillcode

Why aren't we using the Sentry perf tracing instead of log messages?

Udit-takkar

LGTM.

This issue #18366 (review) is not related to this PR

hariombalhara · 2024-12-26T16:28:15Z

@keithwillcode Time taken was already computed earlier,I am just logging thode in this PR and added 2 more cases I believe

Will tackle move to sentry in a followup

hariombalhara force-pushed the routing-perf branch from acba639 to 9379810 Compare December 24, 2024 09:46

keithwillcode added core area: core, team members only enterprise area: enterprise, audit log, organisation, SAML, SSO labels Dec 24, 2024

hariombalhara changed the title ~~Add stats and allow showing embed without loader~~ perf: /router endpoint perf improvement Dec 24, 2024

vercel Bot had a problem deploying to Preview – api December 24, 2024 09:51 Failure

hariombalhara commented Dec 24, 2024

View reviewed changes

hariombalhara force-pushed the routing-perf branch from 9379810 to 86bb2ba Compare December 24, 2024 11:06

github-actions Bot added the ❗️ migrations contains migration files label Dec 24, 2024

hariombalhara commented Dec 24, 2024

View reviewed changes

vercel Bot deployed to Preview – api December 24, 2024 11:14 View deployment

hariombalhara force-pushed the routing-perf branch from 86bb2ba to 1d9d8cb Compare December 24, 2024 11:36

hariombalhara commented Dec 24, 2024

View reviewed changes

vercel Bot deployed to Preview – api December 24, 2024 11:43 View deployment

hariombalhara force-pushed the routing-perf branch from 1d9d8cb to 6f56f8e Compare December 24, 2024 11:55

vercel Bot deployed to Preview – api December 24, 2024 12:02 View deployment

hariombalhara added the ready-for-e2e label Dec 24, 2024

Improve /router performance

3dff0ca

hariombalhara force-pushed the routing-perf branch from 6f56f8e to 3dff0ca Compare December 24, 2024 12:20

vercel Bot deployed to Preview – api December 24, 2024 12:27 View deployment

Fix prisma import error onm client

a6aebf4

vercel Bot deployed to Preview – api December 24, 2024 14:41 View deployment

hariombalhara commented Dec 25, 2024

View reviewed changes

hariombalhara added 2 commits December 26, 2024 11:13

Give more vCPUs to /router/embed which is used for headless routing i…

e2b21d8

…n embed

Make the iterations super fast

5cba5e4

hariombalhara commented Dec 26, 2024

View reviewed changes

vercel Bot deployed to Preview – api December 26, 2024 06:25 View deployment

hariombalhara marked this pull request as ready for review December 26, 2024 06:25

graphite-app Bot requested review from a team December 26, 2024 06:25

hariombalhara requested a review from Udit-takkar December 26, 2024 06:25

dosubot Bot added performance area: performance, page load, slow, slow endpoints, loading screen, unresponsive routing-forms area: routing forms, routing, forms labels Dec 26, 2024

Merge branch 'main' into routing-perf

db4f533

vercel Bot deployed to Preview – api December 26, 2024 06:44 View deployment

Udit-takkar added this to the v4.9 milestone Dec 26, 2024

PeerRich added the High priority Created by Linear-GitHub Sync label Dec 26, 2024

Udit-takkar reviewed Dec 26, 2024

View reviewed changes

keithwillcode reviewed Dec 26, 2024

View reviewed changes

Udit-takkar approved these changes Dec 26, 2024

View reviewed changes

hariombalhara mentioned this pull request Dec 27, 2024

fix: Route Builder field input looses focus on typing #18381

Merged

3 tasks

Merge branch 'main' into routing-perf

c43eb2c

hariombalhara requested a review from a team December 27, 2024 07:11

vercel Bot deployed to Preview – api December 27, 2024 07:17 View deployment

hariombalhara enabled auto-merge (squash) December 27, 2024 08:51

hariombalhara merged commit b5bec79 into main Dec 27, 2024

hariombalhara deleted the routing-perf branch December 27, 2024 08:56

joeauyeung pushed a commit that referenced this pull request Jan 5, 2025

perf: /router endpoint perf improvement (#18366)

212c50e

MuhammadAimanSulaiman pushed a commit to hit-pay/cal.com that referenced this pull request Jan 10, 2025

perf: /router endpoint perf improvement (calcom#18366)

c97fafd

MuhammadAimanSulaiman pushed a commit to hit-pay/cal.com that referenced this pull request Feb 20, 2025

perf: /router endpoint perf improvement (calcom#18366)

5759c07

Conversation

hariombalhara commented Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Mandatory Tasks (DO NOT REMOVE)

How should this be tested?

Followup

Uh oh!

github-actions Bot commented Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel Bot commented Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hariombalhara Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Dec 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E results are ready!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hariombalhara Dec 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

graphite-app Bot commented Dec 26, 2024

Graphite Automations

Uh oh!

Udit-takkar left a comment

Choose a reason for hiding this comment

Uh oh!

hariombalhara commented Dec 26, 2024

Uh oh!

keithwillcode left a comment

Choose a reason for hiding this comment

Uh oh!

Udit-takkar left a comment

Choose a reason for hiding this comment

Uh oh!

hariombalhara commented Dec 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hariombalhara commented Dec 24, 2024 •

edited

Loading

github-actions Bot commented Dec 24, 2024 •

edited

Loading

vercel Bot commented Dec 24, 2024 •

edited

Loading

hariombalhara Dec 24, 2024 •

edited

Loading

github-actions Bot commented Dec 24, 2024 •

edited

Loading

hariombalhara Dec 26, 2024 •

edited

Loading