Lambda Image Processor - Event-Driven Pipeline

A fully serverless image processing pipeline built on AWS. When an image is uploaded to an S3 bucket, a Lambda function automatically triggers, resizes the image into multiple dimensions, saves the processed versions to an output bucket, and sends an email notification via SNS.

Architecture

                         S3 Event
┌──────────────┐      Notification      ┌──────────────┐
│  S3 Bucket   │ ─────────────────────▶ │    Lambda     │
│  (Input)     │                        │  (Python 3.12)│
│  uploads/    │                        └──────┬───┬────┘
└──────────────┘                               │   │
                                               │   │
                              ┌────────────────┘   └────────────────┐
                              ▼                                     ▼
                     ┌──────────────┐                      ┌──────────────┐
                     │  S3 Bucket   │                      │     SNS      │
                     │  (Output)    │                      │  (Email)     │
                     │  thumbnails/ │                      │              │
                     │  medium/     │                      └──────────────┘
                     │  large/      │
                     └──────────────┘

AWS Services Used

Service	Purpose
Amazon S3	Input bucket (receives uploads) and output bucket (stores processed images)
AWS Lambda	Serverless compute that runs the image processing code on each upload event
Amazon SNS	Sends email notifications when processing completes
AWS IAM	Role with least-privilege permissions for Lambda to access S3 and SNS
Amazon CloudWatch	Automatic logging of all Lambda executions for debugging

Key Concepts Demonstrated

Event-Driven Architecture

• S3 event notifications automatically trigger the Lambda function when a new object is created in the uploads/ prefix
• No polling, no cron jobs — the pipeline reacts instantly to uploads
• This pattern is fundamental to modern cloud architecture

Serverless Computing (Lambda)

• Zero server management — AWS handles provisioning, scaling, patching
• Pay-per-use — charged only for the milliseconds the function runs
• Automatic scaling — handles 1 upload or 1,000 concurrent uploads without configuration
• Configured with 256MB memory and 30-second timeout for image processing

IAM Least-Privilege Access

• Created a custom IAM role for the Lambda function with only the permissions it needs:
  • s3:GetObject on the input bucket
  • s3:PutObject on the output bucket
  • sns:Publish on the notification topic
  • logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents for CloudWatch
• This follows the security principle of least privilege — never give more access than required

S3 URL Encoding Fix

• S3 URL-encodes special characters in object keys when passing them through event notifications
• Spaces become +, commas become %2C, etc.
• Without decoding, Lambda fails with NoSuchKey errors
• Fixed by using urllib.parse.unquote_plus() to decode the key before accessing the object

Image Processing Output

For each uploaded image, Lambda creates three resized versions:

Size	Dimensions	Folder
Thumbnail	128x128	thumbnails/
Medium	512x512	medium/
Large	1024x1024	large/

Project Structure

lambda-image-processor/
├── lambda_function.py    # Lambda handler with image processing logic
└── README.md

How It Works

1. User uploads an image to s3://input-bucket/uploads/photo.jpg
2. S3 sends an event notification to the Lambda function
3. Lambda receives the event, extracts the bucket name and object key
4. Lambda downloads the image from the input bucket
5. Lambda resizes the image to 3 sizes using the Pillow library
6. Lambda uploads the resized images to the output bucket
7. Lambda publishes a success notification to the SNS topic
8. SNS delivers an email to all subscribers with the processing summary

How to Deploy

1. Create S3 Buckets

• Create an input bucket (e.g., my-image-input)
• Create an output bucket (e.g., my-image-output)

2. Create SNS Topic

• Create a topic (e.g., image-processing-notifications)
• Subscribe your email and confirm the subscription

3. Create IAM Role

• Create a role for Lambda with permissions for S3 (GetObject, PutObject), SNS (Publish), and CloudWatch Logs

4. Create Lambda Function

• Runtime: Python 3.12
• Handler: lambda_function.lambda_handler
• Add the Pillow layer for image processing
• Set environment variables: OUTPUT_BUCKET, SNS_TOPIC_ARN
• Set memory to 256MB and timeout to 30 seconds

5. Configure S3 Event Notification

• On the input bucket, add an event notification for s3:ObjectCreated:*
• Set prefix filter to uploads/
• Set destination to the Lambda function

6. Test

• Upload an image to s3://input-bucket/uploads/test.jpg
• Check the output bucket for resized versions
• Check your email for the SNS notification
• Check CloudWatch Logs for execution details

Troubleshooting

Issue	Cause	Fix
NoSuchKey error	S3 URL-encodes special characters in object keys	Use unquote_plus() to decode the key
Lambda timeout	Image too large or memory too low	Increase timeout and memory in Lambda config
No email received	SNS subscription not confirmed	Check email for confirmation link
Permission denied	IAM role missing permissions	Add required S3/SNS policies to the role

What I Learned

• How to design event-driven architectures using S3 triggers and Lambda
• IAM role creation with least-privilege policies
• Debugging serverless functions using CloudWatch Logs
• S3 object key URL-encoding behavior and the unquote_plus fix
• SNS topic creation, subscription management, and email notifications
• Lambda configuration: memory allocation, timeout, environment variables, and layers

Built With

• Python 3.12 (Pillow for image processing)
• AWS Lambda, S3, SNS, IAM, CloudWatch