WebGrabber - AI-Powered Visual Website Editor

Project Overview

WebGrabber is a sophisticated visual website editor that allows users to "grab" any existing website, load it into a high-fidelity interactive canvas, modify it using AI or manual inputs, and export the result as a clean, production-ready HTML file.

It uses a "Figma-like" interface with a 3-pane layout, focusing on a premium, dark-mode specialized developer experience.

Core Features & Functionality

1. The Scraper Engine (/api/scrape)

The heart of WebGrabber is its ability to ingest any URL and render it faithfully in the editor.

• Technology: Uses Playwright (Chromium) to load the page in a headless browser.
• Smart Scrolling: Automatically scrolls to the bottom of the page and back to trigger lazy-loaded images and animations before capturing.
• Sanitization: Uses Cheerio to strip dangerous or messy elements:
  • Removes <script>, <iframe>, <noscript>, and CSP meta tags.
  • Converts all relative URLs (images, links) to Absolute URLs so assets load correctly on the user's domain.
• CSS Inlining: To bypass Cross-Origin Resource Sharing (CORS) blocks on stylesheets, the scraper creates a fetch request for every external CSS file, downloads the content, fixes relative url() paths within the CSS, and inlines it into <style> blocks. This ensures the site allows itself to be rendered in a cross-origin iframe.
• Element Tagging: Injects unique data-wg-id attributes (UUIDs) into all meaningful DOM elements (h1, p, sections, divs, etc.) to allow the Editor to track and manipulate them.

2. The Editor Interface

A professional 3-zone layout built with React, Tailwind CSS, and Lucide Icons.

Left Sidebar (Control Center)

• URL Input: Navigation bar to load new sites.
• Tools Menu:
  • Preview Mode: Interact with the site normally (links disabled).
  • Selection Mode: Highlight and click elements to edit them.
  • Design Review: automated checklist of design principles.
  • History: Toggle the timeline view.
• Export: Generates a clean HTML file of the current state.

Center Canvas (The Stage)

• Infinite Canvas Feel: A centered, fixed-size (1280x800) container that floats in the workspace.
• Scroll Locking: The parent container handles scrolling, while the iframe is locked (scrolling="no") to prevent double scrollbars.
• Loading States: A dedicated Loading Overlay with spinner appears during scraping operations.
• Isolation: The website lives inside a sandboxed <iframe> to prevent CSS bleeding into the editor UI.

Right Context Panel (Intelligence)

• Dynamic Content: content changes based on the user's focus.
  • Selection Mode: Shows the selected element's Tag, ID, and a text prompt area to ask AI for changes (e.g., "Make this button blue and round").
  • History Mode: Shows a chronological timeline of changes with Undo/Redo/Replay capabilities.
  • Design Review: Displays mock audit scores for Typography, Contrast, and Mobile responsiveness.

3. AI Modification Engine (/api/ai)

• Integration: Uses Cloudflare Workers AI (Llama-3-8b-instruct model).
• Workflow:
  1. User selects an element (e.g., a Header).
  2. User types a prompt: "Make it punchier and red".
  3. App sends the Element's current HTML, Styles, and User Prompt to the API.
  4. System Prompt enforces a strict JSON Output containing:
     • action: "update_style" or "update_text"
     • value: The new text content.
     • styles: A JSON object of CSS changes (e.g., { color: 'red', fontWeight: 'bold' }).
  5. The Editor applies these changes directly to the DOM via the iframe bridge.

4. Interactive Preview Frame (PreviewFrame.tsx)

• Script Injection: The React component injects a custom vanilla JS script into the iframe document.
• Event Hijacking:
  • mouseover: Adds a dashed blue outline to elements.
  • click: Prevents default links, selects the element, adds a solid blue outline, and sends a postMessage to the parent window with the element's Computed Styles and ID.
  • postMessage Listener: Listens for updates from parent (e.g., AI applied changes) and updates the DOM in real-time.

5. Export System

• Client-Side Generation: No backend needed for export.
• Sanitization:
  • Clones the current iframe DOM.
  • Removes all editor-injected scripts.
  • Strips data-wg-id attributes and editor-specific CSS (outlines, cursors).
• Download: Creates a Blob and triggers a download of webgrabber-export.html.

Environment Variables

The application requires Cloudflare credentials to power the AI features.

CLOUDFLARE_ACCOUNT_ID=your_account_id_here
CLOUDFLARE_API_TOKEN=your_api_token_here

State Management

Uses Zustand (useEditorStore) to manage:

• scrapedHtml: The raw HTML string.
• history / future: Stacks for Undo/Redo.
• selectedElement: Currently active DOM node data.
• currentMode: 'preview' | 'selection' | 'ai_edit'.

Project Structure

• /app/api/scrape: Backend route for Playwright/Cheerio logic.
• /app/api/ai: Backend route for LLM interaction.
• /app/editor: Main UI page.
• /components/PreviewFrame.tsx: The iframe wrapper and script injector.
• /store/useEditorStore.ts: Global state definitions.