Enhancement: Add bulk upload UI to MAAP Console

[sujen1412]

Feature Request

Add a bulk data upload interface to the MAAP Console that leverages the platform's AWS token generation API and existing bucket/prefix access controls.

Motivation

Users need a streamlined way to upload multiple files and datasets directly from the MAAP Console without leaving the platform or managing credentials externally.

Reference Implementation

VEDA has developed a prototype S3 browser uploader that could serve as a reference:

• Repo: https://github.com/developmentseed/s3-browser-uploader
• Review this for UI patterns, file handling, and progress tracking capabilities

MAAP Platform Integration Points

The Console can leverage existing MAAP capabilities:

1. Temporary AWS Credentials API (see Enhancement: Generate and copy temporary AWS tokens from MAAP Console #1309)

   • Generate short-lived tokens with appropriate S3 permissions
   • Handle token refresh/rotation for long uploads
   • Scope permissions to user's accessible buckets/prefixes

2. Bucket & Prefix Access Listing

   • Display available S3 locations where the user has write permissions
   • Use this to populate the upload destination selector
   • Prevent uploads to unauthorized locations

3. Console UI Integration

   • Add upload functionality as a new Console section or modal
   • Integrate with existing MAAP authentication/authorization

Proposed Features

Core Upload Functionality

• Drag-and-drop file upload interface
• Multi-file selection and batch processing
• Progress tracking for each file and overall upload progress
• Upload cancellation and retry capabilities
• Visual feedback on upload status (success, failed, in-progress)

S3 Integration

• Bucket/prefix selection from user's accessible locations
• Direct S3 upload using temporary credentials
• Chunked/multipart upload support for large files
• Automatic retry logic for failed uploads

UX Enhancements

• File size validation before upload
• Estimated time remaining
• Upload history/log
• Clear error messages with remediation steps
• Support for common file formats (data, metadata, configs)

Acceptance Criteria

• Users can select one or more files to upload
• Users can choose destination bucket/prefix from accessible locations
• Temporary AWS credentials are obtained from the API and used for upload
• Real-time progress is displayed during upload
• Upload completion/failure is communicated clearly
• Failed uploads can be retried
• Design integrates seamlessly with MAAP Console UI

Related Issues

• Enhancement: Generate and copy temporary AWS tokens from MAAP Console #1309 - Temporary AWS token generation API

Technical Considerations

• S3 multipart upload API for efficient large file handling
• Browser storage/IndexedDB for upload state persistence
• CORS configuration for direct S3 access from browser
• Security: ensure credentials are only used for authorized buckets/prefixes

[sujen1412]

CORS Configuration Requirement

What Happens Without CORS

If CORS (Cross-Origin Resource Sharing) is not properly configured on the S3 bucket:

1. Browser blocks the request: The browser will prevent direct communication from the Console to the S3 bucket. This is a security feature.

2. Upload fails with error: Users will see browser console errors indicating the request was blocked by CORS policy.

3. No direct browser-to-S3 uploads: The direct S3 upload flow breaks entirely. The temporary credentials become useless because the browser won't let them be used.

Solution

The S3 bucket(s) that MAAP users upload to must have CORS configured to allow requests from the MAAP Console origin.

Required CORS Configuration:

• AllowedOrigins: The MAAP Console domain
• AllowedMethods: GET, PUT, POST, DELETE, HEAD
• AllowedHeaders: Content-Type, Authorization
• ExposeHeaders: ETag, x-amz-version-id

Implementation Notes

• Document CORS setup requirement for MAAP bucket configuration
• Provide CORS configuration templates/scripts for bucket admins
• Consider detecting CORS misconfiguration and surfacing helpful error messages to users
• Add CORS validation to the bucket access listing API
• Test CORS configuration as part of bucket setup/validation flow