Skip to content

Repository files navigation

🧩 Web Data Studio

A visual web data extraction & transformation platform for Chrome β€” pick, preview, clean, and export data from any page.

Detect tables, cards, lists and grids on any website, preview them in a live editable table, clean and reshape the data, and export it in the format you need β€” JSON, CSV, TSV, Markdown, HTML, XML, YAML or SQL. Or export the whole page in one click as Excel, CSV, PDF, Markdown, HTML or TXT, with a live visual preview and an editable text view before you save. Bulk-download every image, video and audio file on a page as a zip. All local, no server, no tracking.

Web Data Studio banner

Manifest V3 TypeScript Vite Version


🎬 Demo

Web Data Studio 10s demo

πŸŽ₯ Full feature walkthrough: Watch the video


πŸ“Έ Screenshots

Popup Explorer Preview & Cleaning
Popup Explorer Preview
Files / Resource Extractor Export Studio History
Files Export History

✨ Features

πŸ” Smart Detection

  • Scans the page and surfaces candidate data blocks automatically β€” native <table> elements, semantic lists, and repeated card/grid patterns built from plain <div>s.
  • Structural-similarity heuristic groups "same-shape" sibling elements to catch card grids, product listings, and search results that don't use semantic markup.
  • Recognizes popular grid/table libraries by their DOM fingerprints, including AG Grid, Tabulator, Handsontable, Kendo Grid, DataTables, Bootstrap Table, PrimeVue, Vuetify, Element Plus, Ant Design, and MUI.

🎯 Visual Element Picker

  • Chrome DevTools–style inspector overlay β€” hover to highlight, click to select any element directly on the page.
  • Works from the popup, the side panel, or the Ctrl+Shift+P shortcut.

πŸ“‹ Live Preview & Data Cleaning

  • Selected data opens in an editable preview table β€” click any cell to edit it, delete rows/columns inline.
  • One-click cleaning operations: trim whitespace, remove duplicate rows, UPPERCASE / lowercase / Title Case.
  • Automatic type inference across extracted columns.

πŸ–ΌοΈ File & Resource Extractor

  • Finds every image, video, audio, and downloadable resource on the page, grouped and previewable.
  • Select individual files or select-all, then zip and download the selection (or the entire set) using an in-browser ZIP build β€” no server round-trip.

πŸ“€ Export Studio

  • Export any extraction to JSON, JSON Lines, CSV, TSV, Markdown, HTML, XML, YAML, or SQL (INSERT statements).
  • Live export preview pane with one-click copy or download.

🌐 Web Tab

The Web tab operates on the entire current webpage, never the currently selected Explorer element. It's designed around five sub-tabs β€” Export, Capture, Analyze, AI, and Clean β€” of which Export is implemented today; the rest are on the roadmap.

πŸ“€ Export (available now)

  • Export the entire current page (not just a picked selection) as Excel (.xlsx), CSV, PDF, Markdown, HTML, or TXT β€” one click, no need to select anything first.
  • Every format auto-builds and auto-saves a real file immediately, then shows it in a live visual preview: a real spreadsheet-style grid for Excel/CSV (row numbers, column letters, sticky headers), the browser's native viewer for PDF, and sandboxed rendering for Markdown/HTML.
  • Toggle to a Text view for any format to see (and edit) the raw output β€” copy it or save your edits directly.
  • Excel detects every table on the page and exports one worksheet per table (falling back to full page content when no tables are found).
  • CSV lists every detected table individually for preview, plus a standing "Full page content" entry, with a Preview button per table and an Export all as .zip option covering everything at once.
  • PDF captures the full page β€” images, backgrounds, links, and tables β€” with an estimated page count and file size shown alongside the preview.
  • Markdown and HTML exports render through a sandboxed preview (Markdown via marked) so formatting is easy to check before saving.
  • TXT export preserves every link, image, and video as a reference instead of dropping them β€” and the Preview pane renders each one back as an inline image thumbnail, a playable video, or a clickable link/file chip, staying in sync as you edit the raw text. Markdown export also now embeds <video> elements directly, playable in the preview.
  • Markdown and TXT previews live-update as you type in the Text view β€” no need to save or reopen to see the rendered result, matching how HTML preview already worked.

πŸ•˜ History

  • Every extraction is saved locally so you can revisit or re-export past sessions without re-scanning the page.

⌨️ Keyboard Shortcuts & Commands

Shortcut Action
Ctrl+Shift+D Open Web Data Studio
Ctrl+Shift+P Toggle element picker
Ctrl+Shift+E Quick export last extraction

🧱 Architecture

  • Manifest V3 with a background service worker, content script, popup, and a dockable Chrome Side Panel UI.
  • Strict TypeScript across a modular core/ (detection, extractors, selectors, transform, exporters, storage) β€” no framework runtime, just Vite + vanilla TypeScript + ES modules.

πŸ—ΊοΈ Roadmap

Web Data Studio is under active development. Planned for upcoming releases:

Web tab β€” remaining sub-tabs (Export shipped in v1.5.0; these are next):

  • πŸ“· Capture β€” visible-viewport and full-page screenshots, automatically copied to the clipboard with a PNG download option.
  • πŸ“Š Analyze β€” full page stats (word/character/heading/link/image/table counts, estimated page size), metadata inspection (SEO tags, Open Graph, Twitter Cards, favicon, manifest), and sortable/exportable inventories of every link, image, and table on the page.
  • πŸ€– AI β€” bring-your-own-key support for OpenAI, Anthropic Claude, Google Gemini, OpenRouter, xAI Grok, DeepSeek, Mistral, Cohere, Ollama, and custom OpenAI-compatible endpoints, powering page summaries, key points, FAQ generation, and structured-data extraction (product, article, recipe, job listing, event) as JSON.
  • 🧹 Clean β€” a distraction-free Reader Mode, plus best-effort ad and popup/overlay removal for cleaner exports and captures.

Elsewhere:

  • Automation Studio (auto-refresh, DOM-change watching, scheduled/batch extraction across tabs)
  • Workspace manager β€” saved selectors, extraction templates, and transformation presets
  • Advanced Data Transformation Studio (calculated columns, regex transforms, custom JS pipelines)
  • JSON Studio β€” schema, TypeScript/Java/C#/Python/Dart model generation, OpenAPI export
  • Plugin system for custom extractors, exporters, and transformers
  • AI-assisted header detection, table recognition, and data cleaning (Explorer side)
  • ODS export, MongoDB / PostgreSQL / MySQL export targets

Contributions and ideas toward this roadmap are very welcome β€” see Contributing.


πŸ“¦ Installation (Developer Mode)

Stable builds are published as a ready-to-use dist.zip on the Releases page β€” no build step required.

  1. Go to the Releases page and download dist.zip from the latest tag.
  2. Unzip it β€” you'll get a folder containing manifest.json, popup.html, sidepanel.html, etc.
  3. Open chrome://extensions in Chrome.
  4. Enable Developer mode (top-right toggle).
  5. Click Load unpacked and select the unzipped folder.
  6. Pin the extension and click the icon, or press Ctrl+Shift+D, to open Web Data Studio.

πŸ› οΈ Development

Requires Node.js 18+.

# install dependencies
npm install

# start the dev build (watches src/ and rebuilds into dist/)
npm run dev

# type-check + production build into dist/
npm run build

# lint
npm run lint

# format
npm run format

After building, load the generated dist/ folder as an unpacked extension via chrome://extensions β†’ Load unpacked.

Project structure

src/
β”œβ”€β”€ background/     # MV3 service worker
β”œβ”€β”€ content/        # Content script, element picker overlay, resource detectors
β”œβ”€β”€ popup/          # Toolbar popup UI
β”œβ”€β”€ sidepanel/      # Main Studio UI (Explorer, Preview, Files, Web, Export, History)
└── core/
    β”œβ”€β”€ detection/   # Table/list/card/grid + library detection
    β”œβ”€β”€ extractors/  # Table & repeated-pattern extractors, type inference
    β”œβ”€β”€ selectors/   # CSS selector generation, element description
    β”œβ”€β”€ transform/   # Safe expression engine for transformations
    β”œβ”€β”€ exporters/   # JSON, CSV, TSV, Markdown, HTML, XML, YAML, SQL, Excel (.xlsx), multi-CSV zip
    β”œβ”€β”€ pdf/         # Full-page PDF capture
    β”œβ”€β”€ resources/   # Resource scanning + ZIP builder for the File Extractor
    β”œβ”€β”€ storage/     # Local extraction history
    └── utils/

🀝 Contributing

Issues and pull requests are welcome. If you're adding a new exporter, detector, or transformation, keep it dependency-free and in strict TypeScript, consistent with the existing core/ modules.

πŸ“„ License

MIT

About

A powerful open-source web data extraction platform that converts HTML tables, lists, and structured content into clean JSON, CSV, Excel, and SQL with visual mapping, AI-assisted detection, and no-code workflows.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages