Skip to content

feat: Google Drive Export Integration#5145

Open
Sentiaus wants to merge 20 commits into
apache:mainfrom
Sentiaus:gdrive-integration
Open

feat: Google Drive Export Integration#5145
Sentiaus wants to merge 20 commits into
apache:mainfrom
Sentiaus:gdrive-integration

Conversation

@Sentiaus
Copy link
Copy Markdown
Contributor

@Sentiaus Sentiaus commented May 21, 2026

PR Description

What changes were proposed in this PR?

In this PR, I've added integration with Google Drive, such that users can now export Datasets, Workflows, and Files to Google Drive. To complete integration, Google Picker needs to be enabled in the Texera GCP, as well as a apiKey and ClientSecret generated and stored in the production user-system.conf

There is also a page for explicitly connecting, reconnecting, and checking folders at /dashboard/user/google-drive, but there is no UI navigation to this.

Notably, there is currently no option to remove access, or change drives, and the drive the user connects can be separate from the gmail the user signs in with.

Schema Changes:

This change modifies the User Table, adding google_drive_refresh_token as a VARCHAR(512).

Backend Changes:

Created a new resource on the backend called GoogleDriveAuthResource with three GET endpoints, /token, /connect and /callback.

Created new values in user-system.conf and UserSystemConfig called clientSecret and apiKey.
These are required to set up Google Drive integration and must be configured on Texera GCP.

@RolesAllowed(Array("REGULAR", "ADMIN")) is applied to /token and /connect. The /callback endpoint is not role-gated because it is called via a browser redirect from Google's OAuth servers (no Authorization header), and authenticates the user instead via a short-lived JWT passed in the state query parameter.

GET Token:

Makes a call to Google APIs to get a short-lived in-memory access token for the user. This is not stored anywhere, and is fetched using the google_drive_refresh_token stored in the DB.

If the user does not have a refresh token, it returns a no_refresh_token status. If the refresh token has been revoked or expired, it returns invalid_grant. The frontend uses these statuses to update its connection state accordingly.

GET Connect:

Returns a Google OAuth authorization URL for the frontend to open in a popup. Accepts a reauth query parameter — when true, sets prompt=consent to force Google to issue a new refresh token. This is used when a previous token has returned invalid_grant.

GET Callback:

Called by Google's OAuth redirect. Authenticates the user via the JWT in the state parameter, exchanges the code parameter for a refresh token using Google's token endpoint, then stores the refresh token on the user record.

Frontend Changes:

DriveService — new injectable service (drive.service.ts) that:

  • getToken(): calls /token and returns the connection status and access token
  • connect(reauth?): opens a popup to the /connect URL and emits on onConnected() when the OAuth flow completes
  • onConnected(): observable that fires when the popup posts a gdrive-connected message
  • exportToDrive(blob, fileName): opens the Google Picker folder selector, then uploads the blob to the chosen folder via the Drive multipart upload API

Export dropdowns — the download button in three locations has been converted to a dropdown with "Download" and "Export to Drive / Connect to Drive" options:

  • Dashboard list items (list-item.component) — for workflows and datasets
  • Dataset detail page (dataset-detail.component) — separate dropdowns for per-version ZIP download and per-file download
  • Workflow editor menu (menu.component) — for the workflow JSON export button

All three locations show a success toast on connection and on successful export, and keep the dropdown button highlighted while the menu is open.

Screenshots of changes:
Screenshot 2026-05-21 at 4 10 32 PM
Screenshot 2026-05-21 at 4 10 47 PM
Screenshot 2026-05-21 at 4 11 02 PM
Screenshot 2026-05-21 at 4 12 15 PM
Screenshot 2026-05-21 at 4 12 44 PM

invalid_grant handling — when getToken() returns invalid_grant (expired/revoked refresh token), a driveNeedsReauth flag is set. Clicking "Connect to Drive" in this state calls connect(true), forcing Google to re-issue a refresh token via the consent screen.

Any related issues, documentation, discussions?

Google Documentation to enable Google Picker: https://developers.google.com/workspace/drive/picker/guides/overview

Closes #4240

How was this PR tested?

Unit tests were added for all Drive-related frontend changes:

  • list-item.component.spec.ts — added a Drive integration describe block covering: token status sets isDriveConnected correctly, onConnected() emission sets isDriveConnected = true and shows a toast, connect() is called when not connected, workflow export calls exportToDrive and shows a success toast, dataset export calls exportToDrive and shows a success toast.

  • menu.component.spec.ts — added a Drive integration describe block covering: token status, onConnected toast, connect() fallback when not connected, and onClickDriveExportWorkflow serializes workflow content and shows a success toast.

  • dataset-detail.component.spec.ts — new spec file covering: token status, onConnected toast, and both onClickDriveExportVersion and onClickDriveExportFile across connected/disconnected/missing-selection/success paths.

The backend /callback endpoint was tested manually via the full OAuth flow in a local dev environment.

Was this PR authored or co-authored using generative AI tooling?

Frontend changes and commit messages generated-by: Claude Sonnet 4.6

@github-actions github-actions Bot added engine ddl-change Changes to the TexeraDB DDL frontend Changes related to the frontend GUI common labels May 21, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 21, 2026

Codecov Report

❌ Patch coverage is 59.90991% with 89 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.93%. Comparing base (62883b8) to head (eec2822).

Files with missing lines Patch % Lines
...shboard/service/user/google-drive/drive.service.ts 43.47% 36 Missing and 3 partials ⚠️
...ser-dataset-explorer/dataset-detail.component.html 0.00% 22 Missing ⚠️
...c/app/workspace/component/menu/menu.component.html 0.00% 11 Missing ⚠️
.../component/user/list-item/list-item.component.html 0.00% 10 Missing ⚠️
...rd/component/user/list-item/list-item.component.ts 88.46% 0 Missing and 3 partials ⚠️
...le-drive-connect/google-drive-connect.component.ts 94.28% 2 Missing ⚠️
.../user-dataset-explorer/dataset-detail.component.ts 93.10% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5145      +/-   ##
============================================
- Coverage     43.35%   42.93%   -0.43%     
  Complexity     2214     2214              
============================================
  Files          1049     1060      +11     
  Lines         40561    41514     +953     
  Branches       4322     4461     +139     
============================================
+ Hits          17586    17823     +237     
- Misses        21887    22581     +694     
- Partials       1088     1110      +22     
Flag Coverage Δ *Carryforward flag
access-control-service 39.53% <ø> (ø) Carriedforward from 6e4ce86
agent-service 33.72% <ø> (ø) Carriedforward from 6e4ce86
amber 43.83% <ø> (-0.01%) ⬇️ Carriedforward from 6e4ce86
computing-unit-managing-service 0.00% <ø> (ø) Carriedforward from 6e4ce86
config-service 0.00% <ø> (ø) Carriedforward from 6e4ce86
file-service 32.18% <ø> (ø) Carriedforward from 6e4ce86
frontend 34.09% <59.90%> (-0.53%) ⬇️
python 90.50% <ø> (ø) Carriedforward from 6e4ce86
workflow-compiling-service 56.81% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Sentiaus Sentiaus changed the title Google Drive Export Integration feat: Google Drive Export Integration May 21, 2026
Sentiaus and others added 16 commits May 21, 2026 15:18
The clientSecret is required for auth code exchange on backend for the access and refresh tokens.
The apiKey is required for the Google Drive Picker component in the frontend.
Pass exception as second arg to logger.error so stack traces are
captured. Also fetch user from DB in /token instead of JWT, and
add LazyLogging trait.
- Add DriveService with OAuth connect flow, Google Picker integration, and multipart upload to Drive
- Add GoogleDriveConnectComponent for the OAuth callback page
- Add export dropdown to list-item (Download / Export to Drive) for workflows and datasets
- Register /dashboard/user/google-drive route for the callback
- Load Google API script in index.html

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Track dropdown visibility with exportMenuVisible
- Lock hover styles (background, button group, description) on the card via export-menu-open class
- Slow dropdown animation scoped to this dropdown via nzOverlayClassName

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both the per-file and per-version download buttons show a dropdown
with Download and Export to Drive options. Button stays highlighted
while the menu is open. Toasts on successful connection and export.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…valid_grant reauth

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Sentiaus Sentiaus force-pushed the gdrive-integration branch from ba86047 to e8487ff Compare May 21, 2026 22:22
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Sentiaus Sentiaus marked this pull request as draft May 21, 2026 22:40
Sentiaus and others added 3 commits May 21, 2026 15:40
npm install had replaced it with Yarn Classic format, breaking yarn --immutable in CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Sentiaus Sentiaus marked this pull request as ready for review May 21, 2026 23:22
@Sentiaus
Copy link
Copy Markdown
Contributor Author

@xuang7 @chenlica Could I get this reviewed please? Additionally, it requires a schema change, which is causing the build to fail for the services + amber.

@xuang7
Copy link
Copy Markdown
Contributor

xuang7 commented May 22, 2026

Thanks for the PR! I I haven't checked all the details yet, but I will take a closer look soon.
Since this PR includes changes across several different components, I suggest breaking it down into smaller PRs by component or functionality. That would make it easier to review and test each part.

@Sentiaus
Copy link
Copy Markdown
Contributor Author

Got it! @xuang7 I'll try breaking it down into the frontend and backend changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

common ddl-change Changes to the TexeraDB DDL engine frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export to external storage

3 participants