Image Processing API

A NestJS backend application for uploading and processing images with PostgreSQL, BullMQ job queues, and Docker support.

Features

• POST /uploads - Upload a single image file (triggers background processing)
• GET /uploads - Get list of all uploaded images
• GET /uploads/:id - Get metadata of one image by ID
• GET /admin/jobs - View real-time job queue status
• Background Processing: Automatic image resizing (thumbnail, medium, HD)
• Job Queues: BullMQ for reliable background processing
• Notifications: Simulated email and push notifications on completion

Tech Stack

• Framework: NestJS with TypeScript
• Database: PostgreSQL with TypeORM
• Queue System: BullMQ with Redis
• Image Processing: Sharp
• File Storage: Local storage in uploads/ folder
• Documentation: Swagger/OpenAPI
• Containerization: Docker & Docker Compose

Project Structure

src/
├── entities/           # Database entities
│   └── image.entity.ts
├── uploads/            # Uploads module
│   ├── uploads.controller.ts
│   ├── uploads.service.ts
│   └── uploads.module.ts
├── queues/             # BullMQ queues
│   ├── queues.module.ts
│   └── processors/
│       ├── image.processor.ts
│       └── notification.processor.ts
├── admin/              # Admin dashboard
│   ├── admin.controller.ts
│   └── admin.module.ts
├── app.module.ts       # Main application module
└── main.ts            # Application entry point

Getting Started

Prerequisites

• Node.js 20+
• pnpm
• Docker and Docker Compose (optional)

Installation

1. Clone the repository
2. Install dependencies:

pnpm install

3. Copy environment configuration:

cp env.example .env

4. Start services with Docker:

docker-compose up -d

This will start:

• PostgreSQL on port 5432
• Redis on port 6379 (for BullMQ queues)
• NestJS API on port 3000

Running Locally (without Docker)

1. Make sure PostgreSQL is running
2. Update .env with your database credentials
3. Start the application:

pnpm start:dev

API Documentation

Once the application is running, visit:

• API: http://localhost:3000
• Swagger UI: http://localhost:3000/api
• Admin Dashboard: http://localhost:3000/admin/jobs (view job status)

Endpoints

Upload Image

POST /uploads
Content-Type: multipart/form-data

# Form field: file

Get All Images

GET /uploads

Get Image by ID

GET /uploads/:id

View Job Status (Admin Dashboard)

# Get all job statuses
GET /admin/jobs

# Get image processing jobs
GET /admin/jobs/image-processing

# Get notification jobs
GET /admin/jobs/notifications

# Get specific job by ID
GET /admin/jobs/image-processing/:id

Database Schema

Images Table

• id (uuid) - Primary key
• originalName (string) - Original filename
• storedName (string) - Stored filename
• path (string) - Full file path
• size (number) - File size in bytes
• status (enum: "uploaded" | "processing" | "processed" | "failed") - Current status
• thumbnailPath (string) - Path to thumbnail version (150x150)
• mediumPath (string) - Path to medium version (800x800)
• hdPath (string) - Path to HD version (1920x1080)
• errorMessage (string) - Error message if processing failed
• createdAt (timestamp) - Upload timestamp
• updatedAt (timestamp) - Last update timestamp

Environment Variables

Copy env.example to .env and configure:

• DB_HOST - Database host (default: localhost)
• DB_PORT - Database port (default: 5432)
• DB_USERNAME - Database username (default: postgres)
• DB_PASSWORD - Database password (default: postgres)
• DB_NAME - Database name (default: image_processing)
• REDIS_HOST - Redis host (default: localhost)
• REDIS_PORT - Redis port (default: 6379)
• PORT - Application port (default: 3000)
• NODE_ENV - Environment (development/production)

Development

# Development mode
pnpm start:dev

# Build
pnpm build

# Start production
pnpm start:prod

# Run tests
pnpm test

# Lint
pnpm lint

Load Testing

Stress test the file upload endpoints and see how Node.js handles concurrent uploads under pressure.

Quick Start

# Install k6 (if not installed)
# macOS: brew install k6
# Linux: see LOAD_TESTING.md

# Interactive test menu
pnpm run load:test

# Or run specific tests
pnpm run load:light   # 10 users
pnpm run load:medium  # 50 users
pnpm run load:heavy   # 100 users
pnpm run load:crash   # 500 users (might crash!)
pnpm run load:stress  # 1000+ users (WILL crash!)

See LOAD_TESTING.md for detailed documentation.

How It Works

Image Processing Flow

1. Upload: User uploads an image via POST /uploads
2. Store: Image metadata is saved to PostgreSQL, file saved to disk
3. Enqueue: A background job is added to the image-processing queue
4. Process: Worker processes the image:
   • Resizes to thumbnail (150x150)
   • Resizes to medium (800x800)
   • Resizes to HD (1920x1080)
5. Update: Database record is updated with paths to processed versions
6. Notify: A notification job is queued to simulate email/push notification

Job Queues

• image-processing: Handles image resizing and optimization
• notifications: Sends fake email and push notifications

Both queues are monitored via the admin dashboard at /admin/jobs.

Docker

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f app

# Stop services
docker-compose down

# Stop and remove volumes
docker-compose down -v

License

Private project