feat: download insights raw data as csv by alannnc · Pull Request #11645 · calcom/cal.diy

alannnc · 2023-10-01T22:05:07Z

What does this PR do?

Adds button to insights that triggers new api endpoint that return file download of same data used for all the insights graphs.

Fixes #10747

Requirement/Documentation

Type of change

New feature (non-breaking change which adds functionality)

How should this be tested?

Are there environment variables that should be set? No
What are the minimal test data to have? using yarn seed-insights from packages/playwright would be useful to have an entire insights setup with bookings/users/event-types ready.
What is expected (happy path) to have (input and output)? Download data that was used for the same count for the graphs
Any other important info that could help to test that PR

Mandatory Tasks

Make sure you have self-reviewed the code. A decent size PR without self-review might be rejected.

linear · 2023-10-01T22:05:11Z

CAL-2347 /insights: be able to access raw data via csv api endpoint

same as metabase, it would be great if you can copy the link to the real-time raw data as a csv:

vercel · 2023-10-01T22:05:16Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
ai	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 12, 2023 1:12pm
api	❌ Failed (Inspect)			Oct 12, 2023 1:12pm
cal	🔄 Building (Inspect)	Visit Preview	💬 Add feedback	Oct 12, 2023 1:12pm
cal-demo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 12, 2023 1:12pm
dev	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 12, 2023 1:12pm
ui	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 12, 2023 1:12pm

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
qa	⬜️ Ignored (Inspect)	Visit Preview		Oct 12, 2023 1:12pm

github-actions · 2023-10-01T22:05:18Z

Thank you for following the naming conventions! 🙏 Feel free to join our discord and post your PR link to collect XP and win prizes!

alannnc · 2023-10-01T22:06:33Z

+    "u"."username" AS "username"
+   FROM "Booking"
+     LEFT JOIN "EventType" et ON "Booking"."eventTypeId" = et.id
+     LEFT JOIN users u ON u.id = "Booking"."userId";


Removed unused membership left join here and added new users to get user->email and username

alannnc · 2023-10-01T22:07:19Z

+    });
+
+    const csvAsString = objectToCsv(csvData);
+    const downloadAs = `Insights-${dayjs(startDate).format("YYYY-MM-DD")}-${dayjs(endDate).format(


Felt that this name convention was right.

alannnc · 2023-10-01T22:07:54Z

+    return "";
+  }
+  const header = Object.keys(data[0]).join(",") + "\n";
+  const rows = data.map((obj: any) => Object.values(obj).join(",") + "\n");


returning props of object as first line for headers.

alannnc · 2023-10-01T22:08:17Z

+
+test.describe.configure({ mode: "parallel" });
+
+const createTeamsAndMembership = async (userIdOne: number, userIdTwo: number) => {


There is another PR that have more test here, reused some logic but only added required test here.

alannnc · 2023-10-01T22:10:16Z

+        endTime: true,
+        paid: true,
+        userEmail: true,
+        username: true,


We should be fine for now for this values.

alannnc · 2023-10-01T22:11:05Z

+   * @param props
+   * @returns
+   */
+  static obtainWhereConditional = async (


Reusing logic from insights router, to be used later and avoid duplicated code.

alannnc · 2023-10-01T22:11:17Z

+   * - memberUserId
+   * - eventTypeId
+   * - userId
+   * - teamId


Important to know.

github-actions · 2023-10-01T22:11:30Z

📦 Next.js Bundle Analysis for @calcom/web

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

deploysentinel · 2023-10-01T22:12:09Z

Current Playwright Test Results Summary

✅ 148 Passing - ⚠️ 2 Flaky

Run may still be in progress, this comment will be updated as current testing workflow or job completes...

(Last updated on 10/12/2023 01:14:53pm UTC)

Run Details

Running Workflow PR Update on Github Actions

Commit: d5be128

Started: 10/12/2023 01:12:00pm UTC

⚠️ Flakes

📄 packages/embeds/embed-core/playwright/tests/action-based.e2e.ts • 1 Flake

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Popup Tests should be able to reschedule Retry 1 • Initial Attempt	5.28% (17) 17 / 322 runs failed over last 7 days	94.10% (303) 303 / 322 runs flaked over last 7 days

📄 packages/embeds/embed-core/playwright/tests/inline.e2e.ts • 1 Flake

Test Case Results

Test Case	Last 7 days Failures	Last 7 days Flakes
Inline Iframe Inline Iframe - Configured with Dark Theme Retry 1 • Initial Attempt	2.80% (9) 9 / 322 runs failed over last 7 days	9.94% (32) 32 / 322 runs flaked over last 7 days

View Detailed Build Results

PeerRich · 2023-10-02T10:09:38Z

getting a real-time URL comes later, right?

CarinaWolli

Tested and works, but please address my comments first 🙏

alannnc · 2023-10-04T21:49:35Z

I'm gonna try the trpc approach while also adding public url link for this PR.

…-csv-api-endpoint-10747-cal-2347

CarinaWolli

Looks good, my requested changes are fixed 👍

api endpoint and test for csv

a7407b7

alannnc linked an issue Oct 1, 2023 that may be closed by this pull request

[CAL-2347] /insights: be able to access raw data via csv api endpoint #10747

Closed

zomars added the core area: core, team members only label Oct 1, 2023

vercel Bot deployed to Preview – ai October 1, 2023 22:06 View deployment

vercel Bot deployed to Preview – ui October 1, 2023 22:06 View deployment

alannnc commented Oct 1, 2023

View reviewed changes

vercel Bot deployed to Preview – api October 1, 2023 22:08 View deployment

vercel Bot deployed to Preview – cal-demo October 1, 2023 22:09 View deployment

alannnc commented Oct 1, 2023

View reviewed changes

vercel Bot deployed to Preview – dev October 1, 2023 22:11 View deployment

PeerRich requested review from a team and zomars October 2, 2023 10:10

vercel Bot deployed to Preview – api October 3, 2023 18:16 View deployment

vercel Bot deployed to Preview – dev October 3, 2023 18:17 View deployment

vercel Bot deployed to Preview – cal-demo October 3, 2023 18:18 View deployment

alannnc self-assigned this Oct 3, 2023

CarinaWolli requested changes Oct 4, 2023

View reviewed changes

Comment thread packages/features/insights/filters/Download/index.tsx

Comment thread packages/features/insights/filters/Download/index.tsx Outdated

Comment thread apps/web/pages/api/insights/download.ts Outdated

Comment thread apps/web/pages/api/insights/download.ts Outdated

github-actions Bot added the 🚨 merge conflict This PR has a merge conflict that has to be addressed label Oct 4, 2023

use trpc instead of api/download

6fdf482

vercel Bot deployed to Preview – ai October 5, 2023 05:37 View deployment

vercel Bot had a problem deploying to Preview – api October 5, 2023 05:37 Failure

vercel Bot deployed to Preview – dev October 5, 2023 05:39 View deployment

vercel Bot deployed to Preview – cal-demo October 5, 2023 05:41 View deployment

alannnc removed their assignment Oct 10, 2023

PeerRich added the Medium priority Created by Linear-GitHub Sync label Oct 11, 2023

fix type error

80dc3c2

vercel Bot deployed to Preview – ai October 12, 2023 12:50 View deployment

vercel Bot deployed to Preview – api October 12, 2023 12:52 View deployment

vercel Bot deployed to Preview – dev October 12, 2023 12:53 View deployment

vercel Bot deployed to Preview – cal-demo October 12, 2023 12:54 View deployment

Merge branch 'main' into feat-insights-be-able-to-access-raw-data-via…

d5be128

…-csv-api-endpoint-10747-cal-2347

github-actions Bot removed the 🚨 merge conflict This PR has a merge conflict that has to be addressed label Oct 12, 2023

vercel Bot deployed to Preview – ai October 12, 2023 13:07 View deployment

vercel Bot had a problem deploying to Preview – api October 12, 2023 13:08 Failure

vercel Bot deployed to Preview – ui October 12, 2023 13:08 View deployment

vercel Bot deployed to Preview – cal-demo October 12, 2023 13:11 View deployment

vercel Bot deployed to Preview – dev October 12, 2023 13:12 View deployment

CarinaWolli self-assigned this Oct 16, 2023

CarinaWolli approved these changes Oct 16, 2023

View reviewed changes

CarinaWolli merged commit fefb6ac into main Oct 16, 2023

CarinaWolli deleted the feat-insights-be-able-to-access-raw-data-via-csv-api-endpoint-10747-cal-2347 branch October 16, 2023 11:27


		test.describe.configure({ mode: "parallel" });

		const createTeamsAndMembership = async (userIdOne: number, userIdTwo: number) => {

Conversation

alannnc commented Oct 1, 2023

What does this PR do?

Requirement/Documentation

Type of change

How should this be tested?

Mandatory Tasks

Uh oh!

linear Bot commented Oct 1, 2023

Uh oh!

vercel Bot commented Oct 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

alannnc Oct 1, 2023

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Next.js Bundle Analysis for @calcom/web

Uh oh!

deploysentinel Bot commented Oct 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current Playwright Test Results Summary

⚠️ Flakes

Test Case Results

Test Case Results

Uh oh!

PeerRich commented Oct 2, 2023

Uh oh!

CarinaWolli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alannnc commented Oct 4, 2023

Uh oh!

CarinaWolli left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vercel Bot commented Oct 1, 2023 •

edited

Loading

github-actions Bot commented Oct 1, 2023 •

edited

Loading

github-actions Bot commented Oct 1, 2023 •

edited

Loading

deploysentinel Bot commented Oct 1, 2023 •

edited

Loading