Oculus

Shared component of both the Petrarca Project and the Boccaccio Project

Oculus is a desktop document-scanning application (built with Electron and React) for turning photos of physical pages — taken with a webcam, a connected camera, or imported from files — into usable images. It is designed as the digitization front-end shared by both projects: its PDF/ZIP output feeds directly into the OCR and encoding stages of tools like Scriptorium's Copyist, wherever a physical/scanned source needs to become a digital one.

Oculus covers the whole capture-to-export pipeline for scanning documents: batch photo capture, automatic page detection and perspective correction (via OpenCV.js), manual crop/straighten/filter tools for the cases automation can't handle, double-page-spread splitting, and final export to PDF or ZIP.

---

Table of Contents

• Architecture Overview
• Capture
• Auto-Process (OpenCV pipeline)
• Manual Editor
• Export
• Cross-cutting Systems
• Tech Stack
• Project Structure

---

Architecture Overview

Oculus is a single-window Electron application with three top-level views managed directly in App.jsx (no router): Capture, Editor, and Export. State is centralized in a single Zustand store (useAppStore.js), which holds the photo batch, the currently selected photo, the multi-step editor's progress, export selection, and all the bookkeeping needed to resume the guided auto-processing flow after a manual edit.

• Main process (src/main/) — native file dialogs, image extraction from existing PDFs (lib/pdf-extract.js), PDF generation (lib/pdf-export.js, via pdfkit), and ZIP archive generation (lib/zip-export.js, via JSZip).
• Renderer process (src/renderer/src/) — the React 19 UI: camera capture, gallery, the OpenCV-powered auto-processing review flow, and the guided manual editor (filters → crop → straighten → confirm).
• Preload script (src/preload/) — exposes the safe window.api/IPC bridge for dialogs, PDF/ZIP export, and PDF image extraction.

All image processing that needs to be pixel-precise (perspective warp, cropping, splitting, filter baking) happens in the renderer using Three.js (for perspective-correct texture mapping) and OpenCV.js (for edge detection), operating on data: URLs / canvases; only the final export step touches the filesystem, via the main process.

---

Capture

The Capture view (CapturePage) is the entry point of the app: a live camera feed plus a swipeable gallery of the current photo batch.

• Live camera capture — a getUserMedia-based camera stream (useStreamCamera hook) works both on desktop and wherever a webcam is available; a dedicated camera selection modal lists all available video input devices (enumerateDevices) so the right one can be chosen — useful on desktop machines with multiple cameras (integrated, USB, virtual).
• Batch capture mode — a modal camera view for capturing many pages in a row without leaving the capture flow, with a quick flash feedback effect on each shot and a running photo count.
• File import — load existing image files (JPG/PNG/WebP) directly from disk via a native file dialog.
• PDF import — extract every embedded image from an existing PDF file and add them to the batch as individual photos, using a dependency-free, byte-level PDF scanner (no rasterization: JPEG streams are returned as-is, raw/deflated image data is inflated and rebuilt as a minimal valid PNG).
• Photo gallery — a swipeable/scrollable gallery of the current batch with pagination dots and prev/next controls (for both touch and desktop/mouse use), per-photo delete, a "clear all" action, and a "double-page" toggle (splitMode) that affects how the subsequent crop step behaves for photos of open books/spreads.
• Entry points to the rest of the app — tapping a photo opens it in the Editor; a dedicated action opens the Export view for the whole batch.

---

Auto-Process (OpenCV pipeline)

AutoProcessReview is a guided, semi-automatic review modal that runs each captured photo through a computer-vision pipeline built on OpenCV.js (@techstark/opencv-js), designed to produce OCR-ready pages with a single click while still leaving an escape hatch to manual editing.

• Automatic page-edge detection — locates the document's quadrilateral within the frame using a layered strategy: adaptive thresholding/Canny edge detection first, then increasingly permissive polygon approximation, then a background-subtraction fallback (sampling the border pixels as a stand-in for the background to isolate the document silhouette — effective even on non-orthogonal or already-cropped source photos), and finally a convex-hull fallback for contours broken up by shadows or reflections.
• Perspective correction (deskew) — warps the detected quadrilateral into a flat, rectangular page image.
• Double-page-spread detection & split — when a photo depicts an open book/document, the pipeline can detect the spine/split line and produce two separate, individually corrected page images instead of one.
• Guided review flow, per photo:
  1. Choose — preview of the (optionally pre-cropped) photo, a "double page" checkbox, and actions to discard the photo, pre-crop it, drop into manual editing, or run the automatic processing.
  2. Cropping (optional) — reuses the same crop tool as the manual editor to pre-isolate the document before automatic detection runs.
  3. Processing — runs the OpenCV pipeline (and the split step, if double-page was checked).
  4. Ready — a side-by-side comparison of the original against the result (one or two images), with the option to discard and retry or accept.
  5. Error — a clear error state with the option to go back and retry.
• Seamless handoff to manual editing — "Process manually" closes the review modal and opens the full guided Editor on the current photo, carrying over any pre-crop and the double-page flag; once the manual edit is saved or cancelled, the review modal reopens automatically on the next photo in the batch, correctly accounting for photos that were split into two.

---

Manual Editor

The Editor (EditorPage) is a sequential, guided 4-step workflow — Filters → Crop → Straighten → Confirm — for the cases the automatic pipeline can't handle, or for fine-tuning its results. Each photo in the batch can be paged through with previous/next navigation, and a step indicator/breadcrumb (with an auto-hiding header overlay) tracks progress.

• Step 1 — Filters (FilterPanel) — adjustable brightness, contrast, saturation, sepia, invert, hue rotation, blur, and free rotation (any angle, not just 90° multiples), applied live as CSS filters and then "baked" into the actual image pixels via an intermediate canvas before moving on — so later steps (crop, straighten) always operate on the filtered result. Rotation correctly recomputes canvas bounding-box dimensions for arbitrary angles so corners aren't clipped.
• Step 2 — Crop (CropTool) — drag-to-resize crop rectangle to trim unwanted borders, with pixel-accurate mapping between the displayed (letterboxed) image and the underlying natural-resolution pixels. In double-page mode, the crop tool instead returns a split line and produces two separate image halves to be straightened independently.
• Step 3 — Straighten (Straightener) — an 8-point (4 corners + 4 edge midpoints) perspective-correction tool built on Three.js: a 3×3-vertex plane mesh is deformed to match the user's points (with the center vertex interpolated from the four corners), and an orthographic camera reads the result back as a flattened, corrected image. A configurable output page-size preset (A4/A5/Letter, portrait or landscape, or free/auto) determines the target aspect ratio. In split mode, the two halves are straightened one after another, with the tool's internal point state cleanly reset between them.
• Step 4 — Confirm — a side-by-side comparison of the original against the processed result (or, in split mode, the original against both resulting pages), with actions to discard all changes or save — replacing the original photo with one (or two, in split mode) processed photos in the batch.
• Auto-process handoff awareness — when opened from the Auto-Process review flow rather than directly from the gallery, the Editor's exit logic transparently reports back whether the user saved or cancelled (and whether a split occurred), so the review modal can resume the batch from the correct position.

---

Export

ExportPanel turns the finished photo batch into a shareable file.

• Selective export — choose which photos from the batch to include, with select-all/deselect-all shortcuts.
• ZIP export — bundles the selected photos as sequentially numbered JPEG files (page_001.jpg, page_002.jpg, …) into a single ZIP archive, built with JSZip.
• PDF export — combines the selected photos into a single PDF (via pdfkit), one photo per page, with configurable page size (A4/Letter), page margin, and a "fit to page" option (scale the photo to fit within the margins while preserving aspect ratio) versus drawing it at natural size from the margin origin.
• Progress feedback — a progress indicator during export, and a post-export confirmation showing the resulting file name and save location.

---

Cross-cutting Systems

• Centralized app state (Zustand) — a single store holds the photo batch, the selected photo and editor step, double-page/split mode, export selection, and all the counters needed to resume the Auto-Process review flow correctly after a detour into manual editing (tracking how many original photos have been consumed and how many extra entries were added to the array by page splits).
• Dependency-free PDF image extraction — rather than pulling in a full PDF-parsing library for the common case of scanned-source PDFs, pdf-extract.js scans the raw PDF byte structure for image XObjects directly: DCTDecode (JPEG) streams are returned unmodified, while FlateDecode (raw pixel) streams are inflated with Node's built-in zlib and rebuilt into a minimal valid PNG using the dimensions and color space declared in the image's own dictionary.
• Perspective/geometry engine reuse — three-straighten.js centralizes all pixel-level geometry operations (filter baking, cropping around points, perspective straightening, page-size aspect-ratio math, splitting an image along a line) so both the manual Straightener/CropTool and any other consumer share exactly one implementation.
• External links — links opened via window.open() anywhere in the app are redirected to the OS's default browser rather than opening inside Electron.

---

Tech Stack

• Application shell: Electron, electron-vite, electron-builder (Windows/macOS/Linux builds)
• UI framework: React 19, Zustand (state management)
• Computer vision: OpenCV.js (@techstark/opencv-js) — page-edge detection, perspective warp, double-page split detection
• 3D/graphics engine: Three.js — 8-point perspective-correct straightening via deformed plane geometry and orthographic projection
• PDF handling: pdfkit (PDF export), a custom dependency-free byte-level extractor (PDF image import)
• Archiving: JSZip (ZIP export)

Project Structure

Oculus/
├── src/
│   ├── main/                       # Electron main process
│   │   ├── index.js                # App entry, window setup
│   │   ├── ipc-handlers.js         # Dialogs, export/import IPC endpoints
│   │   └── lib/
│   │       ├── pdf-extract.js      # Byte-level image extraction from PDFs
│   │       ├── pdf-export.js       # Photo batch → PDF (pdfkit)
│   │       └── zip-export.js       # Photo batch → ZIP (JSZip)
│   ├── preload/                    # IPC bridge (window.api)
│   └── renderer/src/
│       ├── App.jsx                 # View switcher: capture / editor / export
│       ├── main.jsx
│       ├── store/
│       │   └── useAppStore.js      # Zustand store: photos, editor step, export selection
│       ├── hooks/
│       │   ├── useStreamCamera.js  # getUserMedia camera stream management
│       │   ├── useFilters.js       # Filter state & CSS filter string building
│       │   └── useStraighten.js    # Straightener interaction state
│       ├── lib/
│       │   ├── three-straighten.js       # Filters baking, crop, perspective warp, split, page presets
│       │   └── opencv-auto-process.js    # OpenCV.js pipeline: quad detection, warp, double-page split
│       ├── pages/
│       │   ├── CapturePage.jsx     # Camera + gallery view
│       │   └── EditorPage.jsx      # Guided 4-step manual editing workflow
│       └── components/
│           ├── CameraCapture/      # Live camera UI, PDF/file import
│           ├── BatchCamera/        # Rapid-fire batch capture modal
│           ├── CameraSelect/       # Video input device picker
│           ├── PhotoGallery/       # Swipeable batch gallery
│           ├── AutoProcessReview/  # OpenCV auto-processing review modal
│           ├── FilterPanel/        # Step 1: image filters
│           ├── CropTool/           # Step 2: crop / double-page split
│           ├── Straightener/       # Step 3: 8-point perspective correction
│           └── ExportPanel/        # ZIP/PDF export UI
├── electron.vite.config.mjs
└── package.json