Skip to content

feat: extend S3 widget with multi-provider support#1760

Merged
Artuomka merged 1 commit into
mainfrom
s3-bucket-providers
May 7, 2026
Merged

feat: extend S3 widget with multi-provider support#1760
Artuomka merged 1 commit into
mainfrom
s3-bucket-providers

Conversation

@lyubov-voloshko
Copy link
Copy Markdown
Collaborator

@lyubov-voloshko lyubov-voloshko commented May 7, 2026

Adds support for AWS, DigitalOcean Spaces, Backblaze B2, Wasabi, and Cloudflare R2 in the S3 widget. The endpoint URL is derived on the backend per provider rules (region-based for most; account_id for R2) so users only pick a provider rather than supplying a raw endpoint. Renames widget DTOs/params to be provider-agnostic; widget_params JSON keys aws_access_key_id_secret_name/aws_secret_access_key_secret_name become access_key_id_secret_name/secret_access_key_secret_name.

Summary by CodeRabbit

  • New Features

    • Added support for multiple cloud storage providers: AWS S3, DigitalOcean Spaces, Backblaze B2, Wasabi, and Cloudflare R2.
  • Improvements

    • Simplified cloud storage configuration with unified, provider-agnostic credential parameters.
    • Enhanced S3 widgets with provider selection and account identification for improved flexibility and compatibility.

Adds support for AWS, DigitalOcean Spaces, Backblaze B2, Wasabi, and
Cloudflare R2 in the S3 widget. The endpoint URL is derived on the
backend per provider rules (region-based for most; account_id for R2)
so users only pick a provider rather than supplying a raw endpoint.
Renames widget DTOs/params to be provider-agnostic; widget_params JSON
keys aws_access_key_id_secret_name/aws_secret_access_key_secret_name
become access_key_id_secret_name/secret_access_key_secret_name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR refactors S3 widget support from AWS-only to multi-provider bucket storage. A new BucketProviderEnum defines supported providers (AWS, DigitalOcean, Backblaze, Wasabi, Cloudflare R2). Data structures are renamed for provider-agnostic naming. The S3Helper service now resolves provider-specific endpoints and regions. Frontend and backend components parse the new BucketWidgetParams interface and validate provider-specific requirements like Cloudflare R2's account_id. Comprehensive E2E tests validate the provider system.

Changes

Multi-Provider S3 Widget Refactoring

Layer / File(s) Summary
Provider Model
backend/src/entities/s3-widget/application/data-structures/bucket-provider.enum.ts
New BucketProviderEnum defines five supported bucket providers with string identifiers.
Data Structure Contracts
backend/src/entities/s3-widget/application/data-structures/s3-operation.ds.ts, backend/src/entities/s3-widget/application/data-structures/s3-widget-params.ds.ts
Request/response DTOs renamed to bucket-prefixed naming (GetBucketFileUrlDs, BucketFileUrlResponseDs, etc.). BucketWidgetParams replaces S3WidgetParams with optional provider and account_id, and renamed credential secret fields to provider-agnostic names.
Client Configuration & Provider Resolution
backend/src/entities/s3-widget/s3-helper.service.ts
createS3Client refactored to accept a BucketClientConfig object. New helper methods _resolveRegion and _resolveEndpoint resolve provider-specific region defaults ("auto" for Cloudflare R2, "us-east-1" for others) and S3-compatible endpoint URLs, with validation that Cloudflare R2 requires accountId.
Use Case Integration
backend/src/entities/s3-widget/use-cases/get-s3-file-url.use.case.ts, backend/src/entities/s3-widget/use-cases/get-s3-upload-url.use.case.ts
Use cases updated to parse BucketWidgetParams, retrieve credentials using new secret name fields, and pass structured config (with provider, region, accountId) to createS3Client.
Interface & Controller
backend/src/entities/s3-widget/use-cases/s3-use-cases.interface.ts, backend/src/entities/s3-widget/s3-widget.controller.ts
Interface and controller method signatures updated to reference new bucket-oriented data structures.
Backend Validation
backend/src/entities/widget/utils/validate-create-widgets-ds.ts
S3 widget validation requires non-AWS-prefixed secret name fields, validates provider against BucketProviderEnum, and conditionally requires account_id for Cloudflare R2.
Frontend Component Types
frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts, frontend/src/app/components/ui-components/record-view-fields/s3/s3.component.ts, frontend/src/app/components/ui-components/table-display-fields/s3/s3.component.ts
Component params property types updated to BucketWidgetParams with provider and account_id support.
Frontend Service
frontend/src/app/services/s3.service.ts
Response interfaces renamed to BucketFileUrlResponse and BucketUploadUrlResponse; method return types updated accordingly.
Frontend Test Fixtures
frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.spec.ts, frontend/src/app/components/ui-components/record-view-fields/s3/s3.component.spec.ts, frontend/src/app/components/ui-components/table-display-fields/s3/s3.component.spec.ts
Test fixtures updated to use new credential secret name fields in both object and JSON string formats.
Backend E2E Tests
backend/test/ava-tests/non-saas-tests/non-saas-s3-widget-e2e.test.ts
Test setup and assertions updated to new parameter names and bucket-agnostic error messages. New test suite validates widget creation across providers, including success for digitalocean, rejection for unsupported providers, and validation of cloudflare-r2 requiring account_id.
Frontend Defaults
frontend/src/app/components/dashboard/db-table-view/db-table-widgets/db-table-widgets.component.ts
Default S3 widget settings updated with provider field, generic secret name fields, and documentation for multi-provider configuration.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • gugu
  • Artuomka

Poem

🐰 From AWS alone we hop away,
To fields of buckets far and wide,
DigitalOcean, Wasabi play,
With Cloudflare R2 by their side.
Refactored types now generic stand,
Multi-provider magic, close at hand! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Security Check ⚠️ Warning Endpoint injection: region/accountId interpolated into URLs without validation. Unhandled JSON5 parsing exceptions. Provider validation bypassed by empty string. No input sanitization. Validate region/accountId format; wrap JSON5.parse in try-catch; fix provider check (widget_params.provider !== undefined); sanitize inputs before URL interpolation.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately captures the main objective: extending S3 widget functionality to support multiple providers, which is reflected throughout the changeset with new provider enum, renamed DTOs, and updated services.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch s3-bucket-providers

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Artuomka Artuomka enabled auto-merge May 7, 2026 13:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Extends the existing S3 widget to support multiple S3-compatible providers (AWS, DigitalOcean Spaces, Backblaze B2, Wasabi, Cloudflare R2) by introducing a provider field and deriving the correct S3 endpoint on the backend, while renaming widget params/DTOs to be provider-agnostic.

Changes:

  • Add provider (+ account_id for R2) support and backend endpoint resolution logic for S3-compatible providers.
  • Rename widget params keys to provider-agnostic names (access_key_id_secret_name, secret_access_key_secret_name) and update frontend/backend DTOs accordingly.
  • Expand backend E2E coverage to validate provider handling and required params.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
frontend/src/app/services/s3.service.ts Renames response interfaces to provider-agnostic “Bucket*” types while keeping API usage the same.
frontend/src/app/components/ui-components/table-display-fields/s3/s3.component.ts Updates widget params typing to include provider/account_id and new secret key names.
frontend/src/app/components/ui-components/table-display-fields/s3/s3.component.spec.ts Updates tests to use renamed secret key params.
frontend/src/app/components/ui-components/record-view-fields/s3/s3.component.ts Updates widget params typing to include provider/account_id and new secret key names.
frontend/src/app/components/ui-components/record-view-fields/s3/s3.component.spec.ts Updates tests to use renamed secret key params.
frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.ts Updates widget params typing / parsing to the new provider-agnostic shape.
frontend/src/app/components/ui-components/record-edit-fields/s3/s3.component.spec.ts Updates tests to include provider and renamed secret key params.
frontend/src/app/components/dashboard/db-table-view/db-table-widgets/db-table-widgets.component.ts Updates the S3 widget settings template/documentation for multi-provider usage.
backend/test/ava-tests/non-saas-tests/non-saas-s3-widget-e2e.test.ts Updates existing S3 widget tests for renamed params and adds provider validation tests.
backend/src/entities/widget/utils/validate-create-widgets-ds.ts Adds validation for new param names, supported providers, and R2 account_id requirement.
backend/src/entities/s3-widget/use-cases/s3-use-cases.interface.ts Renames DS types to provider-agnostic “Bucket*” naming.
backend/src/entities/s3-widget/use-cases/get-s3-upload-url.use.case.ts Uses provider-agnostic params and new S3 client factory signature.
backend/src/entities/s3-widget/use-cases/get-s3-file-url.use.case.ts Uses provider-agnostic params and new S3 client factory signature.
backend/src/entities/s3-widget/s3-widget.controller.ts Updates controller return types to renamed DS classes.
backend/src/entities/s3-widget/s3-helper.service.ts Implements provider-based region/endpoint resolution and config-driven S3 client creation.
backend/src/entities/s3-widget/application/data-structures/s3-widget-params.ds.ts Renames widget params interface and adds provider/account_id.
backend/src/entities/s3-widget/application/data-structures/s3-operation.ds.ts Renames request/response DS classes to “GetBucket*” / “Bucket*Response”.
backend/src/entities/s3-widget/application/data-structures/bucket-provider.enum.ts Introduces the canonical enum of supported bucket providers.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +43 to +47
const params: BucketWidgetParams =
typeof widget.widget_params === 'string' ? JSON5.parse(widget.widget_params) : widget.widget_params;

const accessKeySecret = await this._dbContext.userSecretRepository.findSecretBySlugAndCompanyId(
params.aws_access_key_id_secret_name,
params.access_key_id_secret_name,
Comment on lines 83 to 87
const accessKeySecret = await this._dbContext.userSecretRepository.findSecretBySlugAndCompanyId(
params.aws_access_key_id_secret_name,
params.access_key_id_secret_name,
user.company.id,
);

if (provider === BucketProviderEnum.CloudflareR2) {
return 'auto';
}
return 'us-east-1';
// bucket: bucket name (required)
// prefix: Optional path prefix for uploaded files
// region: AWS region (default: us-east-1)
// region: Region for the bucket (default: us-east-1; for cloudflare-r2 use "auto")
@Artuomka Artuomka merged commit b08fd0b into main May 7, 2026
21 of 23 checks passed
@Artuomka Artuomka deleted the s3-bucket-providers branch May 7, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants