Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 29 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,18 @@ A React + TypeScript project demonstrating different approaches to file upload c

## Overview

This project explores various concurrency patterns for file uploads to S3, starting with sequential uploads and progressing to more sophisticated parallel upload strategies. It uses LocalStack to simulate AWS S3 locally for development and testing.
This project explores three distinct concurrency patterns for file uploads to S3:
1. **Sequential Upload** - complexity with blocking phases
2. **Async Batch Upload** - complexity with parallel processing within phases
3. **Streaming Upload** - complexity with real-time progress tracking and mutex protection

Each implementation showcases different levels of concurrency control, from basic sequential processing to advanced streaming with race condition management using async-mutex.

## Tech Stack

- **Frontend**: React 19 + TypeScript + Vite
- **AWS SDK**: @aws-sdk/client-s3 + @aws-sdk/s3-request-presigner
- **Concurrency**: async-mutex for race condition management
- **Local Infrastructure**: LocalStack + Docker + Nginx (CORS proxy)
- **Development**: ESLint + TypeScript strict mode

Expand All @@ -19,12 +25,17 @@ This project explores various concurrency patterns for file uploads to S3, start
src/
├── infra/
│ ├── S3Client.ts # LocalStack S3 client configuration
│ ├── S3UploadService.ts # Upload service implementations
│ └── crypto.ts # MD5 integrity checking utilities
├── useCases/
│ ├── useSequentialUpload.ts # Sequential upload implementation
│ └── useAsyncBatchUpload.ts # Batch async upload implementation
├── App.tsx # Main application component
│ ├── useAsyncBatchUpload.ts # Batch async upload implementation
│ └── useStreamingUpload.ts # Streaming upload with mutex protection
├── types/
│ └── uploadProgress.ts # Unified progress tracking interfaces
├── blog/
│ ├── PT-BR.md # Portuguese technical article
│ └── EN-US.md # English technical article
├── App.tsx # Main application with centralized file display
└── main.tsx # Application entry point
```

Expand Down Expand Up @@ -80,28 +91,21 @@ Nginx proxy handles CORS issues between the frontend and LocalStack:
- Nginx proxies to LocalStack with proper CORS headers
- Supports file uploads up to 100MB

## Features

### Upload Interface
- **Strategy Selection**: Radio buttons to switch between Sequential and Batch async
- **Performance Tracking**: Live results table showing run times and comparisons
- **File Management**: Multi-file selection with reset functionality
- **Upload States**: Loading indicators and disabled states during uploads

### Security Features

- Pre-signed URLs for secure uploads
- MD5 integrity checking
- MD5 integrity checking with Content-MD5 headers
- File validation and size limits

## Usage

1. **Start the application**: Follow the Development Setup steps above
2. **Select files**: Use the file input to choose multiple files
3. **Choose strategy**: Select either "Sequential Upload" or "Batch Upload (Async)"
3. **Choose strategy**: Select from "Sequential", "Batch (Async)", or "Streaming" upload
4. **Run test**: Click the upload button and observe the performance
5. **Compare results**: Switch strategies and run again to compare performance
6. **View metrics**: Check the results table for detailed timing comparisons
5. **Monitor progress**: For streaming uploads, watch real-time file-by-file progress
6. **Compare results**: Switch strategies and run again to compare performance
7. **View metrics**: Check the results table for detailed timing comparisons and percentage improvements

## Troubleshooting

Expand All @@ -124,9 +128,18 @@ Nginx proxy handles CORS issues between the frontend and LocalStack:
- LocalStack resource constraints in Docker
- Network saturation with many concurrent uploads

## Key Dependencies

- **async-mutex**: Race condition management for concurrent operations
- **crypto-js**: MD5 hash generation for file integrity
- **@aws-sdk/client-s3**: AWS S3 client for bucket operations
- **@aws-sdk/s3-request-presigner**: Pre-signed URL generation

## Resources

- [AWS S3 Upload Documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html)
- [Pre-signed URLs Guide](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html)
- [Object Integrity Checking](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)
- [LocalStack Documentation](https://docs.localstack.cloud/)
- [async-mutex Library](https://github.com/DirtyHairy/async-mutex)
- [XMLHttpRequest Progress Events](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/upload)
Loading