Skip to content

test: Hotfix/Orgs rewrite causing 404 with 3 or more dots domains#9803

Merged
roae merged 5 commits into
mainfrom
hotfix/404-orgs
Jun 27, 2023
Merged

test: Hotfix/Orgs rewrite causing 404 with 3 or more dots domains#9803
roae merged 5 commits into
mainfrom
hotfix/404-orgs

Conversation

@hariombalhara
Copy link
Copy Markdown
Member

@hariombalhara hariombalhara commented Jun 26, 2023

What does this PR do?

Fixes #9768
Added tests as well. Helped in testing and helps in avoiding future bugs.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How should this be tested?

  • Add a hosts entry for calcom.app.company.com and acme.calcom.app.company.com to 127.0.0.1
  • Visit calcom.app.company.com and see a 404. With this branch that 404 is fixed. Also, test acme.calcom.app.company.com, it should also not be 404.
  • Ensure that env variables are like this
Screenshot 2023-06-26 at 7 25 16 PM

Here are the tests done with app.cal.com and vwo org.

Screenshot 2023-06-27 at 12 36 32 PM Screenshot 2023-06-27 at 12 36 47 PM Screenshot 2023-06-27 at 12 37 00 PM Screenshot 2023-06-27 at 12 37 28 PM

Mandatory Tasks

  • Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected.

@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 26, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
api ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 27, 2023 6:55am
cal ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 27, 2023 6:55am
web-staging ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jun 27, 2023 6:55am
1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
ui ⬜️ Ignored (Inspect) Jun 27, 2023 6:55am

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 26, 2023

Thank you for following the naming conventions! 🙏

@@ -0,0 +1,15 @@
const getDefaultSubdomain = (url) => {
if (!url.startsWith("http:") && !url.startsWith("https:")) {
// Make it a valid URL. Mabe we can simply return null and opt-out from orgs support till the use a URL scheme.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Someone reported they were using cal.com without specifying https:// earlier. So I just added it, didn't verify what else fails if someone uses that.

Comment thread apps/web/next.config.js
Comment on lines -74 to -75
const getSubdomain = () => {
const _url = new URL(process.env.NEXT_PUBLIC_WEBAPP_URL);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved outside to allow writing unit tests

@hariombalhara hariombalhara changed the title Hotfix: Orgs rewrite causing 404 Hotfix: Orgs rewrite causing 404 with 3 or more dots domains Jun 26, 2023
@hariombalhara hariombalhara requested a review from roae June 26, 2023 13:57
@hariombalhara hariombalhara changed the title Hotfix: Orgs rewrite causing 404 with 3 or more dots domains test: Hotfix/Orgs rewrite causing 404 with 3 or more dots domains Jun 26, 2023
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 26, 2023

📦 Next.js Bundle Analysis for @calcom/web

This analysis was generated by the Next.js Bundle Analysis action. 🤖

New Pages Added

The following pages were added to the bundle from the code in this PR:

Page Size (compressed) First Load % of Budget (350 KB)
/[user]/calendar-cache/[month] 325 B 153.41 KB 43.83%
/[user]/calendar-cache/[month]/[orgSlug] 357 B 153.44 KB 43.84%

Ninety-eight Pages Changed Size

The following pages changed size from the code in this PR compared to its base branch:

Page Size (compressed) First Load % of Budget (350 KB)
/404 90.37 KB 243.46 KB 69.56% (🟢 -6.88%)
/500 88.5 KB 241.6 KB 69.03% (🟢 -7.24%)
/[user] 136.08 KB 289.17 KB 82.62% (🟡 +0.31%)
/[user]/[type] 183.95 KB 337.04 KB 96.30% (🟡 +0.29%)
/[user]/[type]/embed 183.97 KB 337.07 KB 96.30% (🟡 +0.30%)
/[user]/book 252.94 KB 406.03 KB 116.01% (🟡 +0.19%)
/[user]/embed 136.13 KB 289.22 KB 82.63% (🟡 +0.31%)
/apps 295.32 KB 448.41 KB 128.12% (🟢 -1.36%)
/apps/[slug] 313.3 KB 466.39 KB 133.26% (🟢 -1.37%)
/apps/[slug]/[...pages] 436.51 KB 589.6 KB 168.46% (🟢 -0.17%)
/apps/[slug]/setup 88.09 KB 241.19 KB 68.91% (🟢 -6.87%)
/apps/categories 258.07 KB 411.16 KB 117.48% (🟢 -1.37%)
/apps/categories/[category] 278.56 KB 431.65 KB 123.33% (🟢 -1.36%)
/apps/installed/[category] 294.94 KB 448.03 KB 128.01% (🟢 -1.36%)
/auth/error 100.52 KB 253.62 KB 72.46% (🟢 -3.64%)
/auth/forgot-password 134.65 KB 287.74 KB 82.21% (🟢 -6.86%)
/auth/forgot-password/[id] 141.85 KB 294.94 KB 84.27% (🟢 -4.83%)
/auth/login 150.21 KB 303.3 KB 86.66% (🟢 -3.65%)
/auth/logout 88.89 KB 241.98 KB 69.14% (🟢 -6.88%)
/auth/saml-idp 82.86 KB 235.95 KB 67.41% (🟢 -7.23%)
/auth/setup 172.7 KB 325.79 KB 93.08% (🟢 -3.29%)
/auth/signin 82.93 KB 236.03 KB 67.44% (🟢 -7.24%)
/auth/sso/[provider] 82.99 KB 236.08 KB 67.45% (🟢 -7.23%)
/auth/sso/direct 82.9 KB 236 KB 67.43% (🟢 -7.24%)
/auth/verify 100.59 KB 253.68 KB 72.48% (🟢 -4.00%)
/auth/verify-email 88.35 KB 241.45 KB 68.98% (🟢 -7.23%)
/availability 263.39 KB 416.48 KB 118.99% (🟢 -1.42%)
/availability/[schedule] 349.55 KB 502.64 KB 143.61% (🟢 -1.36%)
/availability/troubleshoot 258.81 KB 411.9 KB 117.69% (🟢 -1.36%)
/booking/[uid] 217.38 KB 370.47 KB 105.85% (🟡 +0.26%)
/d/[link]/[slug] 183.54 KB 336.64 KB 96.18% (🟡 +0.29%)
/d/[link]/[slug]/embed 183.57 KB 336.67 KB 96.19% (🟡 +0.29%)
/d/[link]/book 252.58 KB 405.68 KB 115.91% (🟡 +0.19%)
/event-types 445.11 KB 598.2 KB 170.92% (🟡 +0.18%)
/getting-started/[[...step]] 440.12 KB 593.22 KB 169.49% (🟢 -2.17%)
/insights 465.24 KB 618.33 KB 176.67% (🟢 -1.37%)
/maintenance 83.07 KB 236.17 KB 67.48% (🟢 -7.24%)
/more 257.7 KB 410.79 KB 117.37% (🟢 -1.36%)
/new-booker/[user]/embed 136.13 KB 289.23 KB 82.64% (🟡 +0.31%)
/new-booker/team/[slug]/embed 224.36 KB 377.46 KB 107.84% (🟡 +0.29%)
/org/[orgSlug] 224.35 KB 377.44 KB 107.84% (🟡 +0.28%)
/org/[orgSlug]/[user] 243.68 KB 396.77 KB 113.36% (🟡 +0.30%)
/org/[orgSlug]/[user]/[type] 183.98 KB 337.08 KB 96.31% (🟡 +0.29%)
/org/[orgSlug]/team/[slug] 224.36 KB 377.45 KB 107.84% (🟡 +0.29%)
/org/[orgSlug]/team/[slug]/[type] 183.57 KB 336.67 KB 96.19% (🟡 +0.29%)
/payment/[uid] 121.23 KB 274.33 KB 78.38% (🟢 -1.78%)
/settings/admin 263.53 KB 416.62 KB 119.04% (🟢 -1.37%)
/settings/admin/apps 275.29 KB 428.39 KB 122.40% (🟢 -0.94%)
/settings/admin/apps/[category] 275.28 KB 428.37 KB 122.39% (🟢 -0.93%)
/settings/admin/flags 266.32 KB 419.42 KB 119.83% (🟢 -1.36%)
/settings/admin/impersonation 263.81 KB 416.9 KB 119.12% (🟢 -1.37%)
/settings/admin/users 264.95 KB 418.04 KB 119.44% (🟢 -1.36%)
/settings/admin/users/[id]/edit 351.33 KB 504.43 KB 144.12% (🟡 +0.42%)
/settings/admin/users/add 351.02 KB 504.11 KB 144.03% (🟡 +0.42%)
/settings/billing 263.69 KB 416.78 KB 119.08% (🟢 -1.36%)
/settings/developer/api-keys 292.24 KB 445.33 KB 127.24% (🟢 -1.36%)
/settings/developer/webhooks 266.67 KB 419.77 KB 119.93% (🟢 -1.45%)
/settings/developer/webhooks/[id] 268.43 KB 421.52 KB 120.43% (🟢 -1.36%)
/settings/developer/webhooks/new 268.29 KB 421.38 KB 120.39% (🟢 -1.36%)
/settings/my-account/appearance 284.84 KB 437.94 KB 125.12% (🟡 +0.34%)
/settings/my-account/calendars 270.16 KB 423.25 KB 120.93% (🟢 -1.36%)
/settings/my-account/conferencing 269.4 KB 422.49 KB 120.71% (🟢 -1.36%)
/settings/my-account/general 348.53 KB 501.63 KB 143.32% (🟢 -1.36%)
/settings/my-account/profile 391.02 KB 544.12 KB 155.46% (🟢 -1.53%)
/settings/organizations/[id]/about 168.69 KB 321.78 KB 91.94% (🟢 -3.79%)
/settings/organizations/[id]/add-teams 168.73 KB 321.82 KB 91.95% (🟢 -3.79%)
/settings/organizations/[id]/onboard-admins 168.74 KB 321.83 KB 91.95% (🟢 -3.78%)
/settings/organizations/[id]/set-password 168.69 KB 321.78 KB 91.94% (🟢 -3.79%)
/settings/organizations/appearance 276.25 KB 429.34 KB 122.67% (🟢 -1.36%)
/settings/organizations/general 344.14 KB 497.23 KB 142.07% (🟢 -1.37%)
/settings/organizations/members 381.06 KB 534.16 KB 152.62% (🟡 +0.23%)
/settings/organizations/new 168.65 KB 321.75 KB 91.93% (🟢 -3.79%)
/settings/organizations/profile 386.15 KB 539.24 KB 154.07% (🟢 -1.36%)
/settings/security/impersonation 265.49 KB 418.58 KB 119.59% (🟢 -1.36%)
/settings/security/password 274.96 KB 428.05 KB 122.30% (🟡 +0.43%)
/settings/security/sso 272.96 KB 426.06 KB 121.73% (🟢 -1.36%)
/settings/security/two-factor-auth 267.91 KB 421 KB 120.29% (🟢 -1.37%)
/settings/teams 263.3 KB 416.39 KB 118.97% (🟢 -1.36%)
/settings/teams/[id]/appearance 276.26 KB 429.35 KB 122.67% (🟢 -1.36%)
/settings/teams/[id]/billing 263.54 KB 416.63 KB 119.04% (🟢 -1.36%)
/settings/teams/[id]/members 383.34 KB 536.44 KB 153.27% (🟡 +0.23%)
/settings/teams/[id]/onboard-members 163.4 KB 316.49 KB 90.43% (🟢 -3.82%)
/settings/teams/[id]/profile 453.86 KB 606.95 KB 173.41% (🟢 -1.70%)
/settings/teams/[id]/sso 273.06 KB 426.15 KB 121.76% (🟢 -1.36%)
/settings/teams/new 202.97 KB 356.06 KB 101.73% (🟢 -3.77%)
/signup 134.14 KB 287.23 KB 82.07% (🟢 -6.88%)
/team/[slug] 224.3 KB 377.39 KB 107.83% (🟡 +0.28%)
/team/[slug]/[type] 183.54 KB 336.64 KB 96.18% (🟡 +0.29%)
/team/[slug]/[type]/embed 183.57 KB 336.67 KB 96.19% (🟡 +0.29%)
/team/[slug]/book 252.59 KB 405.68 KB 115.91% (🟡 +0.19%)
/team/[slug]/embed 224.35 KB 377.45 KB 107.84% (🟡 +0.28%)
/teams 257.79 KB 410.88 KB 117.40% (🟢 -1.37%)
/video/[uid] 239.2 KB 392.29 KB 112.08% (🟢 -1.96%)
/video/meeting-ended/[uid] 95.87 KB 248.96 KB 71.13% (🟢 -4.84%)
/video/meeting-not-started/[uid] 95.51 KB 248.6 KB 71.03% (🟢 -4.85%)
/video/no-meeting-found 87.79 KB 240.88 KB 68.82% (🟢 -6.87%)
/workflows 268.8 KB 421.89 KB 120.54% (🟢 -1.73%)
/workflows/[workflow] 384.79 KB 537.88 KB 153.68% (🟡 +0.31%)
Details

Only the gzipped size is provided here based on an expert tip.

First Load is the size of the global bundle plus the bundle for the individual page. If a user were to show up to your website and land on a given page, the first load size represents the amount of javascript that user would need to download. If next/link is used, subsequent page loads would only need to download that page's bundle (the number in the "Size" column), since the global bundle has already been downloaded.

Any third party scripts you have added directly to your app using the <script> tag are not accounted for in this analysis

The "Budget %" column shows what percentage of your performance budget the First Load total takes up. For example, if your budget was 100kb, and a given page's first load size was 10kb, it would be 10% of your budget. You can also see how much this has increased or decreased compared to the base branch of your PR. If this percentage has increased by 20% or more, there will be a red status indicator applied, indicating that special attention should be given to this. If you see "+/- <0.01%" it means that there was a change in bundle size, but it is a trivial enough amount that it can be ignored.

Copy link
Copy Markdown
Member Author

@hariombalhara hariombalhara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self reviewed.

Comment thread apps/web/next.config.js
source: `/:user((?!${pages.join("|")}|_next|public))/:path*`,
destination: "/:user/:path*",
},
...(process.env.ORGANIZATIONS_ENABLED
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though I have tried to fix the RegExp, still using this existing env variable to opt-out of the behaviour and not impact existing deployments by default.

const subdomainRegExp = getSubdomainRegExp('https://calcom.example.com');
expect(pagesRewriteRegex(subdomainRegExp).exec('calcom.example.com')).toEqual(null)
expect(pagesRewriteRegex(subdomainRegExp).exec('acme.calcom.example.com')?.groups?.orgSlug).toEqual('acme')
// The following also matches which causes anything other than the domain in NEXT_PUBLIC_WEBAPP_URL to give 404
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what to do about this case?

Comment on lines +6 to +9
const _url = new URL(url);
const regex = new RegExp(/^([a-z]+\:\/{2})?((?<subdomain>[\w-.]+)\.[\w-]+\.\w+)$/);
//console.log(_url.hostname, _url.hostname.match(regex));
return _url.hostname.match(regex)?.groups?.subdomain || null;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied as is.

Comment on lines +12 to +14
const defaultSubdomain = getDefaultSubdomain(url);
const subdomain = defaultSubdomain ? `(?!${defaultSubdomain})[^.]+` : "[^.]+";
return subdomain;
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied as is.

url = `https://${url}`;
}
const _url = new URL(url);
const regex = new RegExp(/^([a-z]+\:\/{2})?((?<subdomain>[\w-.]+)\.[\w-]+\.\w+)$/);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only this regex changed with inclusion of dot in subdomain capturing group

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, when the subdomain is org.app.cal.com the slug will be org.app?

Copy link
Copy Markdown
Member Author

@hariombalhara hariombalhara Jun 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subdomain group would be org.app but orgSlug group would be org. This is because we are simply doing negative lookahead for subdomain.
Added a testcase as well.
Screenshot 2023-06-27 at 11 01 03 AM

@deploysentinel
Copy link
Copy Markdown

deploysentinel Bot commented Jun 26, 2023

Current Playwright Test Results Summary

✅ 114 Passing - ⚠️ 2 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 06/27/2023 06:57:10am UTC)

Run Details

Running Workflow PR Update on Github Actions

Commit: 50755b5

Started: 06/27/2023 06:54:51am UTC

⚠️ Flakes

📄   apps/web/playwright/reschedule.e2e.ts • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Reschedule Tests -- old-booker Attendee should be able to reschedule a booking
Retry 1Initial Attempt
0% (0) 0 / 259 runs
failed over last 7 days
14.67% (38) 38 / 259 runs
flaked over last 7 days

📄   packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 1 Flake

Test Case Results

Test Case Last 7 days Failures Last 7 days Flakes
Popup Tests should be able to reschedule
Retry 1Initial Attempt
27.17% (47) 47 / 173 runs
failed over last 7 days
70.52% (122) 122 / 173 runs
flaked over last 7 days

View Detailed Build Results


Copy link
Copy Markdown
Contributor

@zomars zomars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Delegatin final testing to @roae

Copy link
Copy Markdown
Contributor

@roae roae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rewrites related to organization are not working. e.g. org.cal.local/team-slug return 404 error

@hariombalhara
Copy link
Copy Markdown
Member Author

hariombalhara commented Jun 27, 2023

Seems to be working for me @roae. Though, I would like you take it over as I am not fully aware of the logic.
Screenshot 2023-06-27 at 10 56 56 AM

Comment thread apps/web/next.config.js
Comment on lines +102 to +130
const matcherConfigRootPath = {
has: [
{
type: "host",
value: orgHostRegExp,
},
],
source: "/",
};

const matcherConfigOrgMemberPath = {
has: [
{
type: "host",
value: orgHostRegExp,
},
],
source: `/:user((?!${pages.join("|")}|_next|public)[a-zA-Z0-9\-_]+)`,
};

const matcherConfigUserPath = {
has: [
{
type: "host",
value: `^(?<orgSlug>${subdomainRegExp}[^.]+)\\..*`,
},
],
source: `/:user((?!${pages.join("|")}|_next|public))/:path*`,
};
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Abstracted out to shared with headers

Comment thread apps/web/next.config.js
},
],
},
...[
Copy link
Copy Markdown
Member Author

@hariombalhara hariombalhara Jun 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added response header X-Cal-Org-path to easily identify to which path the rewrite occured. It would also help in writing checkly tests where we could just check the header to test if the rewrite is working correctly.

@hariombalhara
Copy link
Copy Markdown
Member Author

Here are the tests done with app.cal.com and vwo org.
Screenshot 2023-06-27 at 12 36 32 PM
Screenshot 2023-06-27 at 12 36 47 PM
Screenshot 2023-06-27 at 12 37 00 PM
Screenshot 2023-06-27 at 12 37 28 PM

@hariombalhara hariombalhara requested a review from roae June 27, 2023 17:51
@hariombalhara hariombalhara marked this pull request as ready for review June 27, 2023 17:52
@roae
Copy link
Copy Markdown
Contributor

roae commented Jun 27, 2023

@hariombalhara it is not working on my end, I'm taking over of this.

imagen

@hariombalhara
Copy link
Copy Markdown
Member Author

@roae I have added a response header to confirm the route(X-Cal-Org-Path). Can you check what the value for this is for you

@roae
Copy link
Copy Markdown
Contributor

roae commented Jun 27, 2023

@roae I have added a response header to confirm the route(X-Cal-Org-Path). Can you check what the value for this is for you

Yes, I notice that, it seems there was something bad in my BD I just reseted it and create a new organization and teams and is working now!

Copy link
Copy Markdown
Contributor

@roae roae left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hariombalhara, nice job.

It's working on my end.

@roae roae merged commit 8384c3f into main Jun 27, 2023
@roae roae deleted the hotfix/404-orgs branch June 27, 2023 19:01
@mariushosting
Copy link
Copy Markdown

I still experience the same identical issue on a new fresh installation. The fix it's already in :main ?

@carsonbird
Copy link
Copy Markdown

I'm also still experiencing this on a fresh install inside docker. My full host domain only has 2 periods. Everything else works great, except for trying to access the root / path.

@PeerRich PeerRich added the core area: core, team members only label Jul 24, 2023
fritterhoff pushed a commit to hm-edu/cal.com that referenced this pull request Jul 25, 2023
…lcom#9803)

* Fix RegExp

* Used ORGANIZATIONS_ENABLED to opt-out of orgs behaviour

* Add comments

* Add a new testcase

* Add response headers for easier debugging and Checkly tests in production
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core area: core, team members only

Projects

No open projects
Status: No status

Development

Successfully merging this pull request may close these issues.

/ Redirect doesnt appear to be working on self hosting

6 participants