CSYE6225 Cloud Computing - Web Application

FastAPI-based RESTful web application with PostgreSQL database (RDS) and S3 image storage, deployed on AWS using custom AMIs.

Overview

This repository contains a FastAPI application that provides:

• User registration and authentication (Basic Auth with BCrypt)
• Product management APIs (CRUD operations)
• Image upload to S3 with metadata tracking
• Health check endpoint for monitoring
• Database: External RDS PostgreSQL (no local database in AMI)
• Storage: S3 for product images

Architecture

• Application: FastAPI with Uvicorn
• Database: AWS RDS PostgreSQL 16.4 (external, managed by Terraform)
• Storage: AWS S3 (IAM role-based access, no hardcoded credentials)
• Deployment: Custom AMI built with Packer, launched via Terraform
• Runtime Config: Database and S3 credentials injected via Terraform user-data

Key Features

Assignment 6 Updates

• RDS Integration: Database runs on AWS RDS (not in AMI)
• S3 Image Uploads: Product images stored in S3 with hierarchical paths
• Lazy Initialization: S3 and database clients initialized only when needed
• Graceful Degradation: Returns 503 when database unavailable (no crashes)
• Immutable Infrastructure: AMI contains application only, no database
• IAM Role Security: S3 access via IAM roles (no access keys)

API Endpoints

Health Check

• GET /healthz - Returns 200 OK (no database dependency)

User Management

• POST /v1/user - Create new user

Product Management (requires authentication)

• POST /v1/product - Create product
• GET /v1/product/{id} - Get product (no auth)
• PUT /v1/product/{id} - Update product (full)
• PATCH /v1/product/{id} - Update product (partial)
• DELETE /v1/product/{id} - Delete product (cascades to images)

Image Management

• POST /v1/product/{id}/image - Upload image (requires auth)
• GET /v1/product/{id}/image - List images (no auth)
• GET /v1/product/{id}/image/{imageId} - Get image details (no auth)
• DELETE /v1/product/{id}/image/{imageId} - Delete image (requires auth)

Dependencies

fastapi==0.104.1
uvicorn==0.24.0
sqlalchemy==2.0.35
psycopg2-binary==2.9.10
python-dotenv==1.0.0
bcrypt==4.0.1
python-multipart==0.0.6
boto3==1.34.0
pytest==7.4.3
httpx==0.25.2

AMI Contents

The custom AMI built by Packer contains:

• Ubuntu 24.04 LTS
• Python 3.11 with virtual environment
• Application code in /opt/csye6225/
• libpq-dev (for psycopg2, PostgreSQL client NOT installed)
• Systemd service configured to auto-start
• Placeholder .env file (populated by Terraform at launch)
• NO database server (uses RDS)
• NO hardcoded credentials (injected at runtime)

Configuration

Application receives configuration at EC2 launch via Terraform user-data:

# Database (RDS)
DATABASE_HOST=<rds-endpoint>
DATABASE_PORT=5432
DATABASE_NAME=csye6225
DATABASE_USER=csye6225
DATABASE_PASSWORD=<secure-password>

# S3
S3_BUCKET_NAME=<unique-bucket-name>
AWS_REGION=us-east-1

# Application
APP_PORT=8000
APP_ENV=production

CI/CD Pipeline

Packer Validate (on PR)

1. Validates Packer configuration
2. Checks formatting
3. Runs integration tests

Packer Build (on merge to main)

1. Runs full test suite
2. Builds webapp.zip artifact
3. Creates custom AMI
4. Shares AMI with demo account
5. Tags AMI with build metadata

Local Development

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export DATABASE_HOST=localhost
export DATABASE_PORT=5432
export DATABASE_NAME=csye6225
export DATABASE_USER=postgres
export DATABASE_PASSWORD=password
export S3_BUCKET_NAME=dev-bucket
export AWS_REGION=us-east-1

# Run application
uvicorn main:app --host 0.0.0.0 --port 8000 --reload

# Run tests
pytest tests/ -v

Testing

# All tests
pytest tests/ -v

# Exclude database unavailable tests
pytest tests/ -v -m "not database_unavailable"

# Only database unavailable tests
pytest tests/ -v -m "database_unavailable"

Security Features

• No Hardcoded Credentials: All secrets injected at runtime
• IAM Role-Based S3 Access: No AWS access keys in code/AMI
• Lazy Initialization: Resources created only when needed
• Secure File Permissions: .env file has 600 permissions
• BCrypt Password Hashing: Industry-standard password security
• Basic Authentication: Required for sensitive operations
• Private S3 Buckets: All public access blocked
• Encrypted RDS: Storage encryption enabled
• Private Subnets: RDS not publicly accessible

Image Storage

Images stored in S3 with hierarchical structure:

s3://bucket-name/
└── users/{user_id}/
    └── products/{product_id}/
        ├── 1.jpg
        ├── 2.png
        └── 3.webp

Supported formats: JPG, JPEG, PNG, GIF, WEBP (max 10MB)

Deployment

1. Build AMI: Merge PR to main (triggers GitHub Actions)
2. Deploy Infrastructure: Use Terraform to create VPC, RDS, S3, EC2
3. Launch Instance: EC2 auto-starts with Terraform-injected config
4. Verify: Health check returns 200, API endpoints functional

Related Repositories

• Infrastructure (Terraform): VPC, RDS, S3, EC2 configuration
• Custom AMI (This repo): Application code and Packer build

Demo Verification

For Assignment 6 demo:

1. SSH to EC2 instance
2. Install PostgreSQL client: sudo apt-get update && sudo apt-get install -y postgresql-client
3. Test RDS connectivity: psql -h <rds-endpoint> -U csye6225 -d csye6225
4. Verify S3 bucket configuration via AWS CLI
5. Test data persistence after EC2 reboot
6. Verify graceful handling when RDS stopped

---

Course: CSYE6225 - Cloud Computing
Student: Malav
Assignment: 6 - RDS Integration and S3 Storage