Pinterest Recreation AI

Pinterest Recreation AI is a full-stack image analysis tool that takes a reference image and turns it into a practical recreation blueprint. Instead of only describing the photo, it tries to answer the useful question: "How do I shoot something that looks close to this?"

The system analyzes the uploaded image, estimates scene and style characteristics, derives camera and lighting guidance, builds an editing recipe, assigns confidence levels, and returns a structured result that the frontend renders as a guided recreation dashboard.

What the project does

Given one reference image, the project generates:

A feasibility score and recreation difficulty
A camera blueprint with angle, distance, framing, crop, and focal-length suggestion
A lighting blueprint with direction, softness, color temperature, ratio, and dynamic range estimates
Composition guidance based on center of interest, symmetry, rule-of-thirds likelihood, and dutch angle
Pose and expression cues when detectable
Props detected from the image
A style summary and dominant palette
An editing recipe in Lightroom-like and Snapseed-like terms
A priority system that ranks what matters most for matching the reference
Beginner, pro, and ultimate guidance flows
Confidence blocks and warnings for uncertain analysis areas
Raw extracted data for debugging and iteration

Core idea

Most "image inspiration" workflows stop at inspiration. This project tries to convert inspiration into execution.

The backend uses classical image processing, lightweight CV heuristics, and optional computer vision models to estimate how the image was made. The frontend then turns those results into a readable blueprint that a creator can follow while shooting and editing.

Product flow

The user opens the landing page in the Next.js frontend.
The user uploads an image and can optionally provide location, intended capture time, and equipment notes.
The frontend sends a multipart/form-data request to the FastAPI backend.
The backend saves the upload, runs the analysis pipeline, stores raw and final JSON outputs, and returns a job id.
The frontend navigates to /result/[id].
The result page fetches the generated JSON and renders camera, lighting, edit, confidence, and advanced recreation intelligence panels.

Monorepo structure

.
|-- backend/
|   |-- app/
|   |   |-- api.py
|   |   |-- config.py
|   |   |-- main.py
|   |   |-- pipeline.py
|   |   |-- schemas.py
|   |   |-- extract/
|   |   |-- fusion/
|   |   |-- plan/
|   |   |-- utils/
|   |   |-- verify/
|   |   |-- eval/
|   |   |-- llm/
|   |   `-- storage/
|   |-- requirements.txt
|   `-- yolov8n.pt
|-- frontend/
|   |-- src/
|   |   |-- app/
|   |   |-- components/
|   |   `-- lib/
|   |-- package.json
|   `-- .env.local
`-- README.md

Tech stack

Frontend

Next.js 14
React 18
TypeScript
Tailwind CSS

Backend

Python 3.9+
FastAPI
Uvicorn
Pillow
NumPy
OpenCV
scikit-learn
MediaPipe, optional
Ultralytics YOLOv8, optional

Architecture overview

Frontend responsibilities

The frontend is responsible for:

Presenting the landing page and upload experience
Collecting optional context from the user
Sending image analysis requests
Fetching result JSON by job id
Rendering multiple views of the blueprint
Exposing a user layer, pro layer, ultimate layer, and debug layer
Showing a live scoring interface so users can simulate how closely they are matching the target

Backend responsibilities

The backend is responsible for:

Validating uploads
Saving uploaded files
Running the analysis pipeline synchronously
Deriving structured facts from low-level image analysis
Generating recreation plans and editing recipes
Persisting raw and final JSON results
Serving the upload directory as static files
Returning result payloads for the frontend

Backend request lifecycle

Entry point

backend/app/main.py creates the FastAPI app, enables CORS for the local frontend, mounts uploaded images as static files under /uploads, and registers the API router.

API endpoints

backend/app/api.py exposes two main endpoints:

POST /analyze
- Accepts an image file
- Accepts optional location, capture_time, and equipment
- Saves the file into backend/app/storage/uploads
- Runs the pipeline
- Updates the stored result JSON with upload_url
- Returns { "id": "<job_id>" }
GET /result/{job_id}
- Reads the generated JSON from backend/app/storage/results
- Returns the full blueprint payload

Accepted file types

The backend currently accepts:

.jpg
.jpeg
.png
.webp

Analysis pipeline

The full pipeline lives in backend/app/pipeline.py.

At a high level it performs:

Load and resize the uploaded image
Extract low-level signals from the image
Compute confidence blocks
Derive higher-level facts from raw extraction output
Build planning outputs such as tools, edits, feasibility, and step-by-step guidance
Generate an "ultimate intelligence" block for advanced guidance
Save both raw and final JSON

Stage 1: image loading and normalization

Utilities in backend/app/utils/image_io.py:

Open the image with Pillow
Convert it to RGB
Resize it to a maximum side length of 1280
Return width, height, and aspect ratio metadata

This keeps processing lighter and gives downstream modules a consistent image size.

Stage 2: extract raw signals

Raw extraction modules live in backend/app/extract/.

Palette extraction

extract/palette.py:

Downsamples large images for efficiency
Samples pixels
Uses KMeans clustering to estimate the dominant palette
Returns hex colors and a grayscale contrast estimate

Lighting extraction

extract/lighting.py:

Estimates brightness from grayscale mean
Estimates texture to classify light as soft or hard
Compares left/right and top/bottom brightness to infer dominant light direction
Estimates warm, cool, or neutral temperature
Builds a normalized 16-bin histogram
Builds a simple tone curve
Estimates dynamic range and key/fill ratio

Composition extraction

extract/composition.py:

Detects edges using OpenCV Canny when available, otherwise a fallback gradient method
Estimates edge density
Computes the center of interest from edge distribution
Detects likely rule-of-thirds placement
Estimates symmetry
Estimates dutch angle from detected lines when OpenCV is available

Scene extraction

extract/scene.py:

Uses simple heuristics to infer a scene label
Currently defaults toward broad labels such as studio or living room
Estimates background depth, blur strength, and rough subject-background separation

Pose extraction

extract/pose.py:

Uses MediaPipe pose if installed
Detects subject center, shoulder width, hip width, torso tilt, and subject scale
Returns a fallback note if MediaPipe is not installed

Object extraction

extract/objects.py:

Uses YOLOv8 if Ultralytics is installed
Loads yolov8n.pt
Detects object labels, confidences, and bounding boxes
Returns a fallback note if the dependency is unavailable

Expression extraction

extract/expression.py:

Uses MediaPipe Face Mesh if available
Estimates smile, neutral, or serious expression
Estimates eye openness and mouth-open ratio
Returns a fallback note if MediaPipe is unavailable

Stage 3: confidence scoring

backend/app/fusion/confidence.py turns raw extractor outputs into confidence blocks for:

Objects
Pose
Lighting
Scene
Expression
Depth

Each block includes:

score
warnings

These warnings are surfaced in the UI and are important because not every field is equally reliable.

Stage 4: fact derivation

backend/app/fusion/derive_facts.py converts raw extraction data into higher-level creative guidance.

This is the core translation layer between raw image measurements and usable production advice.

It derives:

Camera angle labels and angle degrees
Camera distance labels and distance in meters
Lens suggestions
Framing classification
Crop suggestions
Style mood and palette summary
Lighting summary with ratio, dynamic range, and color temperature
Composition summary
Background notes
Expression summary
Background depth summary
Priority ranking of what matters most
Beginner, pro, and ultimate guidance arrays
Real-time scoring weights and component scores

Stage 5: planning outputs

Planning modules live in backend/app/plan/.

Tool mapping

plan/tool_mapper.py recommends practical gear such as:

Phone camera
Natural window light
Diffusion alternatives
Free editing tools

Edit recipe generation

plan/edit_recipe.py builds an editing recipe based on:

Image histogram statistics
Palette saturation
Dynamic range
Lighting and style attributes

It outputs:

lightroom_like settings
snapseed_like settings
recipe_quality
calibration

This module is one of the more sophisticated parts of the project. It does a constrained optimization pass to produce more stable tone edits rather than only using fixed rules.

Feasibility scoring

plan/feasibility.py assigns:

A numerical feasibility score
Difficulty level: Easy, Medium, or Hard
recreatable boolean
Warnings
Short reasoning about achievability

Step builder

plan/steps.py produces a practical shooting sequence:

Background setup
Camera setup
Composition alignment
Lighting setup
Shooting loop
Edit pass
Verification order

Ultimate intelligence engine

plan/ultimate_engine.py generates advanced guidance designed for more deliberate recreation workflows.

It includes:

Adaptive acceptable ranges for distance, angle, lighting ratio, and white balance
Sensitivity analysis
Style stability
Natural light predictions
Equipment adaptation guidance
Overlay targets for face and alignment
Success prediction
Failure simulation
Scenario simulation
Correction strategy
Optimization loop
Accuracy envelope

This is effectively the "advanced operator mode" of the project.

Result payload shape

The final result saved to backend/app/storage/results/<job_id>.json includes fields such as:

{
  "id": "job_id",
  "input_image": "filename.jpg",
  "recreatable": true,
  "feasibility": {},
  "camera": {},
  "composition": {},
  "lighting": {},
  "style": {},
  "props": {},
  "pose": {},
  "expression": {},
  "background_depth": {},
  "lens_suggestion": {},
  "style_dna": {},
  "priority_system": [],
  "guidance": {
    "beginner": [],
    "pro": [],
    "ultimate": []
  },
  "realtime_scoring": {},
  "edit_recipe": {},
  "tools": {},
  "steps": [],
  "ultimate_mode": {},
  "confidence": {},
  "raw": {}
}

The raw extraction output is also stored separately in backend/app/storage/cache/<job_id>.raw.json.

Frontend walkthrough

Frontend source lives in frontend/src/.

Landing page

src/app/page.tsx renders the hero interface and opens the upload modal.

Upload modal

src/components/UploadModal.tsx:

Lets the user choose or drag-drop an image
Captures optional location, capture time, and equipment notes
Calls analyzeImage() from src/lib/api.ts
Redirects to the result route when the backend returns an id

Dropzone

src/components/Dropzone.tsx provides the file input and drag-and-drop UI.

Result route

src/app/result/[id]/page.tsx:

Reads the route parameter
Calls getResult(id)
Renders ResultView

Result dashboard

src/components/ResultView.tsx is the primary frontend renderer. It displays:

Uploaded reference image
Feasibility and live score
Beginner and pro mode toggles
User, pro, ultimate, and debug view toggles
Confidence bars
Guidance lists
Tools list
Camera blueprint
Lighting blueprint
Lighting precision widgets
Composition diagram
Style DNA
Expression and depth
Priority system
Ultimate-mode intelligence
Edit recipe cards
Debug metadata

API client

src/lib/api.ts contains:

analyzeImage(file, options)
getResult(id)

The frontend expects the backend base URL from:

NEXT_PUBLIC_BACKEND_URL=http://127.0.0.1:8000

Storage model

The backend creates these storage directories automatically:

backend/app/storage/uploads
backend/app/storage/cache
backend/app/storage/results

What each directory contains

uploads/
- Original uploaded image files
cache/
- Raw extractor output JSON
results/
- Final user-facing blueprint JSON

This means the app is stateful in local storage. It is simple and practical for local development, but not yet production-grade persistence.

Setup and installation

Prerequisites

Python 3.9 or later
Node.js 18 or later
pip
npm

Backend setup

cd backend
python -m venv .venv

Activate the virtual environment

Windows:

.venv\Scripts\activate

macOS/Linux:

source .venv/bin/activate

Install backend dependencies

pip install -r requirements.txt

Start the backend

uvicorn app.main:app --reload

The backend will run at:

http://127.0.0.1:8000

Frontend setup

cd frontend
npm install

Create frontend/.env.local:

NEXT_PUBLIC_BACKEND_URL=http://127.0.0.1:8000

Start the frontend:

npm run dev

The frontend will run at:

http://127.0.0.1:3000

Local development notes

CORS is configured for http://localhost:3000 and http://127.0.0.1:3000
Uploaded files are served by FastAPI under /uploads
The pipeline currently runs synchronously during the POST /analyze request
Large uploads or heavy optional models will directly affect request latency

Optional dependencies and fallback behavior

Some analysis capabilities are optional.

If `mediapipe` is missing

Pose extraction falls back to found: false
Expression extraction falls back to detected: false
Related confidence warnings become more important

If `ultralytics` is missing

Object detection returns an empty list
Prop suggestions become less reliable

If OpenCV is missing

Some image operations use lighter fallback logic

The app still runs without all optional dependencies, but output quality is reduced.

Current strengths

Clear end-to-end product flow
Fast local iteration
Good separation between extraction, fusion, and planning logic
Strong frontend presentation for technical output
Practical recreation focus rather than only descriptive AI output
Debug visibility through raw result payloads and confidence blocks

Current limitations

The project is functional, but several areas are still heuristic or early-stage:

Scene classification is simplistic and currently biased toward broad labels
The pipeline is synchronous and not queued
Result persistence is local JSON storage rather than a database
There is no authentication, rate limiting, or job management
There are no formal automated tests in the current repository
schemas.py is minimal and the API is not strongly typed end-to-end
llm/, verify/, and eval/ directories exist, but are not currently central to the live request flow
The repo currently includes generated storage artifacts and installed frontend dependencies, which are typically excluded from source control in production repositories

Recommended next improvements

If you want to evolve this project further, the highest-value next steps are:

Add a proper typed response schema for the complete blueprint payload.
Move analysis work into background jobs instead of processing inside the upload request.
Add tests for extractor outputs, planning logic, and API contract stability.
Improve scene classification with a more robust model or classifier.
Add result cleanup, retention policy, and .gitignore hygiene for uploads, cache, results, .next, and node_modules.
Add authentication and persistent storage if this will be deployed.
Add versioning to result payloads so frontend and backend changes remain compatible over time.

API quick reference

`POST /analyze`

Form fields:

file, required
location, optional
capture_time, optional
equipment, optional

Example response:

{
  "id": "2e8d3c3f2f3f4d07b2a3b4c5d6e7f890"
}

`GET /result/{job_id}`

Returns the full blueprint JSON generated for the uploaded image.

Example user experience

Upload a Pinterest-style portrait.
The system estimates that the image uses side light, a medium-close camera distance, a 50mm-equivalent look, and a soft warm tone.
The result page tells the user what to prioritize first, such as lighting direction and camera distance.
The user applies the shooting plan and edit recipe.
The live score and advanced guidance help tighten the match through iteration.

Who this project is for

This project is useful for:

Content creators
Photographers
Videographers matching still frames
Social media creators recreating aesthetic references
Developers interested in applied computer vision for creative tooling

Status

This repository is currently a solid prototype / local product build rather than a production-hardened system. The core value is already visible: turning a single reference image into actionable recreation guidance. The next phase is mainly about improving model quality, reliability, testing, and deployment architecture.

Name		Name	Last commit message	Last commit date
backend		backend
frontend		frontend
README.md		README.md

Latest commit

History

Folders and files

Repository files navigation