SeamForge — Content-Aware Image Resizer

A full-stack web app implementing the 6-component seam carving pipeline:

1. Automatic Semantic Object Detection (SAM)
2. Adaptive Energy Map (gradient + saliency + semantic)
3. Wide-Area Seam Distribution
4. Super-Resolution Restoration (Real-ESRGAN)
5. Thin-Plate Spline Warping
6. Computational Optimisation Layer

---

Project Structure

seam-carving-app/
├── backend/
│   ├── app.py            ← Flask API server
│   ├── pipeline.py       ← Full 6-component pipeline
│   ├── requirements.txt
│   └── start_backend.sh
└── frontend/
    ├── package.json
    ├── public/index.html
    ├── src/
    │   ├── App.js / App.css
    │   ├── index.js / index.css
    │   └── components/
    │       ├── UploadZone.js/css
    │       ├── Controls.js/css
    │       ├── ResultPanel.js/css
    │       ├── LogPanel.js/css
    │       └── StatusBar.js/css
    └── start_frontend.sh

---

Setup

1. Backend

cd seam-carving-app/backend

# Create and activate virtual environment (recommended)
python3 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

# Install base requirements
pip install flask flask-cors numpy opencv-python scipy

# ── SAM (you already have this installed) ──────────────────────────────────
pip install git+https://github.com/facebookresearch/segment-anything.git
# Place your checkpoint files in the backend/ directory:
#   sam_vit_b_01ec64.pth   (ViT-B)
#   sam_vit_h_4b8939.pth   (ViT-H)

# ── Real-ESRGAN (optional) ─────────────────────────────────────────────────
pip install basicsr realesrgan
# Download RealESRGAN_x4plus.pth and place it in backend/
# https://github.com/xinntao/Real-ESRGAN/releases

# ── PyTorch (required for SAM and ESRGAN) ─────────────────────────────────
# CPU-only:
pip install torch torchvision
# GPU (CUDA 11.8):
# pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

Custom SAM checkpoint paths

If your SAM .pth files are in a different folder, set environment variables before starting:

export SAM_CKPT_B=/path/to/sam_vit_b_01ec64.pth
export SAM_CKPT_H=/path/to/sam_vit_h_4b8939.pth

2. Frontend

cd seam-carving-app/frontend
npm install

---

Running

Open two terminals:

Terminal 1 — Backend

cd seam-carving-app/backend
source venv/bin/activate
python app.py
# → Running on http://localhost:5000

Terminal 2 — Frontend

cd seam-carving-app/frontend
npm start
# → Opens http://localhost:3000

---

Usage

1. Open http://localhost:3000
2. Drop or click to upload any image (PNG/JPG/WEBP)
3. Set your target Width and Height in pixels
4. Toggle SAM and/or Real-ESRGAN on/off
5. Choose SAM variant: ViT-B (faster) or ViT-H (more accurate)
6. Click Run SeamForge
7. Watch the pipeline log, then compare Before / After
8. Click ↓ Save to download the result

---

How it works (pipeline summary)

Step	Component	What it does
1	SAM	Detects all objects automatically, builds semantic importance mask
2	Energy Map	E = gradient + saliency + semantic; objects get barrier-level energy
3	Seam Carving	Removes/inserts seams with clustering penalty + spatial smoothing
4	Real-ESRGAN	SR upscale then downscale to restore detail after resizing
5	TPS Warp	Smooths object-background boundary discontinuities
6	Proxy Map	All ops run on ≤800px proxy first for speed

---

Notes

• Without SAM/PyTorch installed, the system falls back to gradient+saliency energy (still works well)
• Without Real-ESRGAN, the SR step is skipped
• Large images (>2000px) may take 1–3 minutes — the proxy map helps significantly
• The backend stores uploads in backend/uploads/ and results in backend/outputs/