@posty5/screenshoter

Crawl any website, discover all its pages, and take full-page screenshots — saved in a folder structure that mirrors the URL path. Built with Crawlee and Playwright.

---

🌟 What is @posty5/screenshoter?

@posty5/screenshoter is a TypeScript package that automates the process of capturing screenshots for every page on a website. It handles two tasks:

1. Crawl — Discover all URLs on a website using Crawlee's PlaywrightCrawler or sitemap parsing
2. Capture — Take a screenshot of each URL using Playwright and save it to disk
3. Single Page — Capture a screenshot of a single URL without crawling
4. Batch Capture — Capture screenshots of a list of URLs you provide

Screenshots are saved in a folder structure that mirrors the URL path:

https://posty5.com/en/social-media-publisher
  → screenshots/posty5.com/en/social-media-publisher/capture.webp

https://posty5.com/en/qr-code-generator
  → screenshots/posty5.com/en/qr-code-generator/capture.webp

https://posty5.com/
  → screenshots/posty5.com/capture.webp

Use Cases:

• 📸 Visual regression testing
• 🗂️ Website archiving and documentation
• 🖼️ Generating preview images for SEO / social sharing
• 📊 Auditing website pages at scale
• 🔍 Collecting URLs from a website for analysis

---

📦 Installation

npm install @posty5/screenshoter

After installing, make sure Playwright's Chromium browser is available:

npx playwright install chromium

---

🚀 Quick Start

Programmatic API

import { captureWebsite } from "@posty5/screenshoter";

const result = await captureWebsite({
  url: "https://posty5.com",
  outputDir: "./screenshots",
  format: "webp",
  excludePatterns: ["/api/**", "/user/*"],
});

console.log(`Captured ${result.captured} of ${result.totalUrls} pages`);

CLI

# Take screenshots of all pages
npx screenshoter capture https://posty5.com -o ./screenshots

# Collect URLs only (save to file)
npx screenshoter collect-urls https://posty5.com --file urls.txt

---

📚 API Reference

captureWebsite(config)

Crawl a website, discover all pages, and take a screenshot of each one.

Returns: Promise<CaptureResult>

import { captureWebsite } from "@posty5/screenshoter";

const result = await captureWebsite({
  url: "https://posty5.com",
  outputDir: "./screenshots",
  format: "webp",
  viewport: { width: 1440, height: 900 },
  maxPages: 50,
  maxDepth: 3,
  excludePatterns: ["/api/**", "/user/*", "*.pdf"],
  concurrency: 3,
  waitAfterLoad: 1000,
  headless: true,
});

console.log(result);
// {
//   totalUrls: 42,
//   captured: 40,
//   failed: 2,
//   skipped: 0,
//   pages: [
//     { url: 'https://posty5.com/en', filePath: './screenshots/posty5.com/en/capture.webp', status: 'ok' },
//     { url: 'https://posty5.com/en/about', filePath: './screenshots/posty5.com/en/about/capture.webp', status: 'ok' },
//     ...
//   ]
// }

---

collectUrls(config)

Crawl a website and save all discovered URLs to a text file — one URL per line.

Config type: CollectUrlsConfig — only URL collection options (no screenshot settings).

Returns: Promise<string[]>

import { collectUrls } from "@posty5/screenshoter";

const urls = await collectUrls({
  url: "https://posty5.com",
  outputFile: "./urls.txt",
  maxPages: 100,
  excludePatterns: ["/api/**"],
});

console.log(`Found ${urls.length} URLs`);
// urls.txt:
// https://posty5.com
// https://posty5.com/en
// https://posty5.com/en/social-media-publisher
// https://posty5.com/en/qr-code-generator
// ...

---

capturePage(config)

Capture a screenshot of a single page — no crawling involved.

Config type: CapturePageConfig — only screenshot options + url.

Returns: Promise<PageResult>

import { capturePage } from "@posty5/screenshoter";

const result = await capturePage({
  url: "https://posty5.com/en/about",
  outputDir: "./screenshots",
  format: "png",
  viewport: { width: 1440, height: 710 },
  waitAfterLoad: 1500,
});

console.log(result);
// {
//   url: 'https://posty5.com/en/about',
//   filePath: './screenshots/posty5.com/en/about/capture.png',
//   status: 'ok'
// }

---

capturePages(urls, config?)

Capture screenshots for a list of URLs. Reuses a single browser instance and supports concurrency.

Config type: CaptureConfig — only screenshot options (no url or crawl settings).

Returns: Promise<CaptureResult>

import { capturePages } from "@posty5/screenshoter";

const result = await capturePages(["https://posty5.com", "https://posty5.com/en/about", "https://posty5.com/en/social-media-publisher"], {
  outputDir: "./screenshots",
  format: "png",
  viewport: { width: 1440, height: 710 },
  concurrency: 2,
  waitAfterLoad: 1500,
});

console.log(`Captured ${result.captured} of ${result.totalUrls} pages`);

---

scrollToEnd(page)

A beforeScreenshot utility that smoothly scrolls to the bottom of the page and waits for scroll-triggered animations to settle before the screenshot is taken. Useful for pages with fade-in or reveal animations driven by IntersectionObserver.

Returns: Promise<void>

import { capturePages, scrollToEnd } from "@posty5/screenshoter";

const result = await capturePages(urls, {
  outputDir: "./screenshots",
  format: "png",
  fullPage: true,
  beforeScreenshot: scrollToEnd,
});

---

⚙️ Configuration

The configuration is split into two concerns:

URL Collection Options (CollectUrlsConfig)

Used by collectUrls() and captureWebsite().

Option	Type	Default	Description
url	string	(required)	Root URL to start crawling from
strategy	'crawl' | 'sitemap'	'crawl'	URL discovery strategy
sitemapUrl	string	<url>/sitemap.xml	Custom sitemap URL (sitemap strategy only)
maxPages	number	100	Maximum number of pages to crawl
maxDepth	number	5	Maximum crawl depth from the root URL
excludePatterns	string[]	[]	Glob patterns for URL paths to skip
includePatterns	string[]	[]	Glob patterns — only matching URLs are captured
sameDomainOnly	boolean	true	Only capture URLs on the same domain
headless	boolean	true	Run browser in headless mode (crawl strategy)
shouldCapture	(url: string) => boolean	—	Programmatic filter — return false to skip a URL

For collectUrls(), an additional option is available:

Option	Type	Default	Description
outputFile	string	'urls.txt'	File path to save the URL list

Screenshot Options (CaptureConfig)

Used by capturePage(), capturePages(), and captureWebsite().

Option	Type	Default	Description
outputDir	string	'./screenshots'	Directory to save screenshots
format	'webp' | 'png' | 'jpeg'	'webp'	Screenshot image format
viewport	{ width, height }	{ width: 1440, height: 900 }	Browser viewport size
fullPage	boolean	false	Capture the full scrollable page instead of just the viewport
waitAfterLoad	number	1000	Milliseconds to wait after page load before screenshot
concurrency	number	3	Number of parallel browser pages
headless	boolean	true	Run browser in headless mode
scrollToEnd	boolean	false	Smoothly scroll to the bottom before screenshot (triggers scroll animations). Auto-enabled when fullPage is true
beforeScreenshot	(page: Page) => Promise<void>	—	Callback to run before each screenshot

captureWebsite() accepts both sets of options combined (CaptureWebsiteConfig).

---

🚫 Filtering Dynamic Pages

Three mechanisms to control which pages get captured:

1. Exclude Patterns

Skip URLs matching glob patterns:

await captureWebsite({
  url: "https://example.com",
  excludePatterns: [
    "/api/**", // Skip all API routes
    "/user/*", // Skip user profile pages
    "/*/edit", // Skip edit pages
    "*.pdf", // Skip PDF links
    "/auth/**", // Skip auth pages
  ],
});

2. Include Patterns (Whitelist)

When set, only matching URLs are captured:

await captureWebsite({
  url: "https://example.com",
  includePatterns: [
    "/en/**", // Only capture English pages
    "/products/*", // Only capture product pages
  ],
});

3. Custom Filter Callback

For complex logic:

await captureWebsite({
  url: "https://example.com",
  shouldCapture: (url) => {
    const parsed = new URL(url);
    // Skip URLs with query parameters
    if (parsed.search) return false;
    // Skip paths with more than 4 segments
    if (parsed.pathname.split("/").filter(Boolean).length > 4) return false;
    return true;
  },
});

---

🖥️ CLI Reference

capture — Take Screenshots

npx screenshoter capture <url> [options]

Option	Description	Default
-o, --output <dir>	Output directory	./screenshots
-f, --format <format>	Image format: webp, png, jpeg	webp
--width <number>	Viewport width	1440
--height <number>	Viewport height	900
--max-pages <number>	Maximum pages to crawl	100
--max-depth <number>	Maximum crawl depth	5
--exclude <patterns...>	Glob patterns to exclude	—
--include <patterns...>	Glob patterns to include	—
--concurrency <number>	Parallel browser pages	3
--wait <number>	Wait ms after page load	1000
--no-headless	Show browser window	—
--urls-file <path>	Use URLs from a file instead of crawling	—

Examples:

# Basic usage
npx screenshoter capture https://posty5.com

# Custom output and format
npx screenshoter capture https://posty5.com -o ./my-screenshots -f png

# Exclude dynamic pages
npx screenshoter capture https://posty5.com --exclude "/api/**" "/user/*" "/auth/**"

# Only capture specific sections
npx screenshoter capture https://posty5.com --include "/en/**"

# Limit crawl depth and page count
npx screenshoter capture https://posty5.com --max-pages 20 --max-depth 2

# Use a pre-collected URLs file
npx screenshoter capture https://posty5.com --urls-file urls.txt

collect-urls — Collect URLs Only

npx screenshoter collect-urls <url> [options]

Option	Description	Default
--file <path>	Output file path	urls.txt
--max-pages <number>	Maximum pages to crawl	100
--max-depth <number>	Maximum crawl depth	5
--exclude <patterns...>	Glob patterns to exclude	—
--include <patterns...>	Glob patterns to include	—
--no-headless	Show browser window	—

Examples:

# Collect all URLs
npx screenshoter collect-urls https://posty5.com

# Save to custom file with filters
npx screenshoter collect-urls https://posty5.com --file sitemap.txt --exclude "/api/**"

---

🔧 Advanced Usage

Scroll Animations (Fade-in on Scroll)

Some pages reveal content as you scroll (using IntersectionObserver or CSS fade-in classes). A viewport-only screenshot would capture those elements as invisible/empty.

Use scrollToEnd: true or the built-in scrollToEnd utility to scroll through the page first:

import { capturePages, scrollToEnd } from "@posty5/screenshoter";

// Option 1 — config flag (auto-enabled when fullPage: true)
const result = await capturePages(urls, {
  fullPage: true, // scrollToEnd is automatically true
  waitAfterLoad: 1000,
});

// Option 2 — explicit flag
const result = await capturePages(urls, {
  scrollToEnd: true,
  waitAfterLoad: 1000,
});

// Option 3 — use the scrollToEnd utility as a beforeScreenshot hook
const result = await capturePages(urls, {
  fullPage: true,
  beforeScreenshot: scrollToEnd,
});

---

Before Screenshot Hook

Dismiss cookie banners, close popups, or interact with the page before taking the screenshot:

await captureWebsite({
  url: "https://example.com",
  beforeScreenshot: async (page) => {
    // Dismiss cookie consent
    const cookieBtn = page.locator('button:has-text("Accept")');
    if (await cookieBtn.isVisible()) {
      await cookieBtn.click();
      await page.waitForTimeout(500);
    }
  },
});

Two-Step Workflow: Collect → Edit → Capture

First collect URLs, manually edit the list, then capture only the URLs you want:

# Step 1: Collect all URLs
npx screenshoter collect-urls https://posty5.com --file urls.txt

# Step 2: Edit urls.txt — remove any pages you don't want

# Step 3: Capture only the remaining URLs
npx screenshoter capture https://posty5.com --urls-file urls.txt

Or programmatically with separated configs:

import { collectUrls, capturePages } from "@posty5/screenshoter";

// Step 1: Collect URLs (CollectUrlsConfig only)
const urls = await collectUrls({
  url: "https://posty5.com",
  strategy: "sitemap",
  maxPages: 500,
  excludePatterns: ["/api/**"],
  outputFile: "./urls.txt",
});

// Step 2: Filter programmatically
const staticPages = urls.filter((u) => !u.includes("/trends/"));

// Step 3: Capture with separate config (CaptureConfig only)
const result = await capturePages(staticPages, {
  outputDir: "./screenshots",
  format: "png",
  viewport: { width: 1440, height: 710 },
  concurrency: 3,
});

---

📂 Output Structure

The output folder mirrors the URL path structure:

screenshots/
├── posty5.com/
│   ├── capture.webp                              ← https://posty5.com
│   ├── en/
│   │   ├── capture.webp                          ← https://posty5.com/en
│   │   ├── social-media-publisher/
│   │   │   └── capture.webp                      ← https://posty5.com/en/social-media-publisher
│   │   ├── qr-code-generator/
│   │   │   └── capture.webp                      ← https://posty5.com/en/qr-code-generator
│   │   └── url-shortener/
│   │       └── capture.webp                      ← https://posty5.com/en/url-shortener
│   └── ar/
│       ├── capture.webp                          ← https://posty5.com/ar
│       └── social-media-publisher/
│           └── capture.webp                      ← https://posty5.com/ar/social-media-publisher

---

💻 Requirements

• Node.js: >= 18.0.0
• TypeScript: Full type definitions included
• Browser: Chromium (auto-installed via Playwright)

---

📖 Resources

• Website: https://posty5.com
• Dashboard: https://studio.posty5.com
• GitHub: https://github.com/Posty5/Screenshoter
• Support: https://posty5.com/contact-us

---

🆘 Support

• Contact Us: https://posty5.com/contact-us
• GitHub Issues: Report bugs or request features

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes
4. Commit your changes: git commit -m 'Add amazing feature'
5. Push to the branch: git push origin feature/amazing-feature
6. Submit a pull request

---

📄 License

MIT License - see LICENSE file for details.

---

Made with ❤️ by the Posty5 team