dynafetch

Fetch any website like a real browser. One function call.

dynafetch provides Chrome-level TLS fingerprinting, JavaScript execution, and full network interception. The response includes fully rendered HTML and all captured requests.

Install

npm i @grabbit-labs/dynafetch

Usage

import { dynafetch } from "@grabbit-labs/dynafetch";

const page = await dynafetch("https://example.com");
console.log(page.html);       // fully rendered HTML
console.log(page.framework);  // "nextjs", "inertia", "nuxt", ...
console.log(page.status);     // 200

Features

• Chrome TLS fingerprint: indistinguishable from a real browser at the network layer
• Full JS execution: SPAs, client-rendered content, and lazy-loaded data
• Request interception: captures every fetch(), XHR, and WebSocket call
• Framework detection: identifies Next.js, Nuxt, Inertia, Remix, Astro, SvelteKit, and more
• Performance: parallel module resolution and batch network calls; 700+ module Vite apps render in under 5 seconds

Hyper-Proxying

Route specific request types through your proxy while letting others connect directly. This reduces proxy bandwidth and latency for requests that don't need it.

// Proxy all requests
const page = await dynafetch({
  url: "https://example.com",
  proxy: "http://user:pass@ip:port",
});

// Only proxy the page fetch and API calls, not static assets
const page = await dynafetch({
  url: "https://example.com",
  proxy: {
    url: "http://user:pass@ip:port",
    only: ["page", "api"],
  },
});

Scope	Covers
"page"	Initial HTML document fetch
"api"	fetch() and XHR calls from page scripts
"assets"	JS scripts, ES modules, static resources

Headers and cookies

Chrome 146 headers are included by default. Custom headers merge on top; your values override the defaults, everything else is preserved.

const page = await dynafetch({
  url: "https://example.com",
  headers: { "Accept-Language": "fr-FR" },
  cookies: { session: "abc123" },
});

AI SDK tool

Use dynafetch as a tool in the Vercel AI SDK:

import { z } from "zod";
import { generateText, tool } from "ai";
import { dynafetch } from "@grabbit-labs/dynafetch";

const result = await generateText({
  model: yourModel,
  tools: {
    fetchPage: tool({
      description: "Fetch a web page with full browser emulation and return the rendered HTML",
      parameters: z.object({
        url: z.string().url().describe("The URL to fetch"),
      }),
      execute: async ({ url }) => {
        const page = await dynafetch(url);
        return { html: page.html, status: page.status, framework: page.framework };
      },
    }),
  },
  prompt: "Get the homepage of example.com and summarize it",
});

Quiescence tuning

dynafetch waits for async network activity to complete before returning. These options control that behavior:

// Return quickly
const page = await dynafetch({
  url: "https://example.com",
  maxWaitMs: 1000,
  idleWaitMs: 50,
});

// Wait longer for slow endpoints
const page = await dynafetch({
  url: "https://example.com",
  maxWaitMs: 5000,
  idleWaitMs: 200,
});

Option	Default	Description
minWaitMs	75	Minimum ms before checking idle state
idleWaitMs	100	Ms of zero pending requests to consider settled
maxWaitMs	3000	Hard cap on wait time
moduleWaitMs	6000	Max wait for ES module bundling
timeoutMs	none	Overall operation timeout

Architecture

1. Harvest: fetches the HTML document through a TLS client matching Chrome's handshake. Parses scripts, modulepreloads, and SSR state.

2. Module graph resolution: recursively discovers and batch-fetches the full JS dependency tree in parallel. 700+ modules resolve in approximately 5 batch rounds.

3. Execute: runs scripts in a sandboxed environment with browser API shims. All network calls are intercepted and routed through the TLS proxy.

4. Settle: waits for async activity to complete, then returns rendered HTML, framework metadata, and timing breakdown.

Requirements

• Node.js 18+

The TLS proxy ships as precompiled binaries for macOS (arm64, x64), Linux (arm64, x64), and Windows (x64). No additional toolchain required.

License

MIT