S3 Upload Test Server

A simple Express.js server with S3-compatible file upload functionality. Supports AWS S3 and S3-compatible storage services like MinIO, DigitalOcean Spaces, Cloudflare R2, and others.

Features

• ✅ Express.js server with file upload endpoint
• ✅ Web-based upload interface with drag & drop
• ✅ S3-compatible storage integration
• ✅ File validation and size limits
• ✅ Upload progress tracking
• ✅ Unique filename generation
• ✅ CORS support
• ✅ Environment-based configuration
• ✅ Health check endpoint
• ✅ Error handling

Prerequisites

• Node.js (v14 or higher)
• npm or yarn
• S3-compatible storage account and credentials

Quick Start

1. Clone and setup

   cd s3-upload-test
   npm install

2. Configure environment

   cp .env.example .env
   # Edit .env with your credentials

3. Start the server

   npm start
   # or for development
   npm run dev

4. Test the upload

   # Open browser and go to http://localhost:3000
   # Or test via curl
   curl -X POST -F "file=@your-file.jpg" http://localhost:3000/upload

Environment Configuration

Copy .env.example to .env and configure your credentials:

AWS S3 Configuration

AWS_ACCESS_KEY_ID=your_access_key
AWS_SECRET_ACCESS_KEY=your_secret_key
AWS_REGION=us-east-1
S3_BUCKET=your-bucket-name
# Leave S3_ENDPOINT empty for AWS S3
S3_ENDPOINT=
S3_FORCE_PATH_STYLE=false

MinIO Configuration

AWS_ACCESS_KEY_ID=minioadmin
AWS_SECRET_ACCESS_KEY=minioadmin
AWS_REGION=us-east-1
S3_BUCKET=test-bucket
S3_ENDPOINT=http://localhost:9000
S3_FORCE_PATH_STYLE=true

DigitalOcean Spaces Configuration

AWS_ACCESS_KEY_ID=your_spaces_key
AWS_SECRET_ACCESS_KEY=your_spaces_secret
AWS_REGION=nyc3
S3_BUCKET=your-space-name
S3_ENDPOINT=https://nyc3.digitaloceanspaces.com
S3_FORCE_PATH_STYLE=false

Cloudflare R2 Configuration

AWS_ACCESS_KEY_ID=your_r2_access_key
AWS_SECRET_ACCESS_KEY=your_r2_secret_key
AWS_REGION=auto
S3_BUCKET=your-r2-bucket
S3_ENDPOINT=https://your-account-id.r2.cloudflarestorage.com
S3_FORCE_PATH_STYLE=false

API Endpoints

GET /

Serves the web upload interface when accessed from a browser, or returns server information as JSON when accessed programmatically.

Browser Response: Returns the HTML upload interface with drag & drop functionality.

API Response:

{
  "message": "S3 File Upload Server",
  "endpoints": {
    "upload": "POST /upload - Upload a file to S3",
    "health": "GET /health - Health check",
    "ui": "GET / - Upload interface"
  }
}

GET /health

Health check endpoint.

Response:

{
  "status": "OK",
  "timestamp": "2023-10-06T12:00:00.000Z"
}

POST /upload

Upload a file to S3-compatible storage.

Request:

• Method: POST
• Content-Type: multipart/form-data
• Body: Form data with file field

Example with curl:

curl -X POST -F "file=@example.jpg" http://localhost:3000/upload

Example with JavaScript:

const formData = new FormData();
formData.append('file', fileInput.files[0]);

fetch('http://localhost:3000/upload', {
  method: 'POST',
  body: formData
})
.then(response => response.json())
.then(data => console.log(data));

Success Response:

{
  "message": "File uploaded successfully",
  "file": {
    "originalName": "example.jpg",
    "size": 123456,
    "mimeType": "image/jpeg",
    "s3Key": "1696608000000_a1b2c3d4_example.jpg",
    "location": "https://your-bucket.s3.amazonaws.com/1696608000000_a1b2c3d4_example.jpg"
  }
}

Error Response:

{
  "error": "No file provided"
}

File Handling

• Size Limit: 1GB (configurable in server.js)
• Storage: Files are stored in memory temporarily and then uploaded to S3
• Filename Generation: Automatic unique filename with timestamp and hash
• Metadata: Original filename, upload date, and file size are stored as S3 metadata

Testing with Different Tools

Using the Web Interface

1. Start the server: npm start
2. Open your browser and go to http://localhost:3000
3. Use the drag & drop interface or click to select a file
4. Click "Upload File" and watch the progress
5. View the upload results and file details

Using curl

# Upload a file
curl -X POST -F "file=@test.jpg" http://localhost:3000/upload

# Health check
curl http://localhost:3000/health

Using Postman

1. Create a new POST request to http://localhost:3000/upload
2. Go to Body → form-data
3. Add key file with type File
4. Select your file and send

Using HTML Form

<!DOCTYPE html>
<html>
<head>
    <title>File Upload Test</title>
</head>
<body>
    <form action="http://localhost:3000/upload" method="post" enctype="multipart/form-data">
        <input type="file" name="file" required>
        <button type="submit">Upload</button>
    </form>
</body>
</html>

Development

Scripts

• npm start - Start the production server
• npm run dev - Start with nodemon for development
• npm test - Run tests (not implemented yet)

Project Structure

s3-upload-test/
├── public/
│   └── index.html     # Web upload interface
├── server.js          # Main Express server
├── s3-client.js       # S3 client configuration and utilities
├── package.json       # Dependencies and scripts
├── .env.example       # Environment variables template
├── .gitignore         # Git ignore rules
└── README.md          # This file

Error Handling

The server handles various error scenarios:

• No file provided (400)
• File too large (400)
• S3 upload failures (500)
• Invalid endpoints (404)
• Server errors (500)

Security Considerations

• Files are validated for size limits
• CORS is enabled (configure as needed)
• Environment variables protect sensitive credentials
• Consider adding file type validation for production use
• Implement authentication/authorization as needed

Troubleshooting

Common Issues

1. "Access Denied" errors

   • Check your AWS credentials and permissions
   • Ensure the bucket exists and is accessible
   • Verify the bucket policy allows uploads

2. Connection timeouts

   • Check your S3_ENDPOINT configuration
   • Verify network connectivity to the S3 service
   • For local MinIO, ensure it's running

3. "Bucket not found" errors

   • Verify the bucket name in your .env file
   • Ensure the bucket exists in the specified region

4. CORS errors in browser

   • The server includes CORS headers
   • For production, configure CORS origins appropriately

Debug Mode

Add console logging to troubleshoot issues:

// Add to server.js for debugging
app.use((req, res, next) => {
  console.log(`${req.method} ${req.path}`, req.body);
  next();
});

Contributing

Feel free to submit issues and enhancement requests!

License

MIT License