Skip to content

Recipes

Mark Rogers edited this page Oct 16, 2026 · 2 revisions

Outputs come from fetchwright 0.1.0 against a stand-in for Google's API, with sample content (see Home).

Save a thread as JSON and read it in Python

python fetchwright.py read https://www.reddit.com/r/examplesub/comments/1abc234/ --json --out thread.json
wrote thread.json (1169 characters)

--out is never cut, unlike stdout. Then:

import json

with open("thread.json", encoding="utf-8") as f:
    thread = json.load(f)
print(thread["title"])
print(len(thread["comments"]), "comments kept, dropped:", thread["dropped"])

prints:

I made a small upscaler node
3 comments kept, dropped: {'placeholder': 1, 'lone_token': 1, 'empty': 0}

Call fetchwright from another program

Run it as a subprocess, read JSON from stdout with --max-chars 0 so it is never cut, and act on the exit code:

import json
import os
import subprocess
import sys

CLI = os.environ.get("FETCHWRIGHT_CLI", "skills/fetchwright/scripts/fetchwright.py")
url = "https://www.reddit.com/r/examplesub/comments/1abc234/"
proc = subprocess.run([sys.executable, CLI, "read", url, "--json", "--max-chars", "0"],
                      capture_output=True, text=True, encoding="utf-8")
if proc.returncode == 3:
    raise SystemExit("rate limited: stop for now")
if proc.returncode == 4:
    raise SystemExit("no key, or the key was refused")
result = json.loads(proc.stdout)
print(result["read_ok"], result["model"], len(result["comments"]))

prints:

True gemini-3.8-flash 3

Exit 1 (not read) and 5 (every model down) still print a record, with read_ok false and the reason in why_not. Exit codes are listed on Commands. Do not retry a 429 in a loop: wait, and check the quota in AI Studio.

Read several URLs

python fetchwright.py read https://www.reddit.com/r/examplesub/comments/1abc234/ https://example.org/article --gap 6

Each URL gets its own request, --gap seconds apart (5 by default), and its own record: in text, one after the other; with --json, a list. A 429 on one model's quota moves to the next model; any other 429, or every model's quota spent, stops the run and prints the records read so far.

Spend fewer tokens on long threads

Most of a thread's cost is output tokens: the instruction asks for every comment word for word. Ask for fewer:

python fetchwright.py read https://www.reddit.com/r/examplesub/comments/1abc234/ --max-comments 30

On a paid project the fetched page is billed as input tokens at the model's price, and URL context has no separate charge (pricing page, 2026-10-15).

Use one model only

FETCHWRIGHT_MODELS=gemini-3.5-flash-lite python fetchwright.py read https://www.reddit.com/r/examplesub/comments/1abc234/

The same works with --models gemini-3.5-flash-lite. With one model there is no fallback, so an overloaded model gives exit 5. The inline form above is for POSIX shells; in PowerShell set $env:FETCHWRIGHT_MODELS first (not tested).

Keep the raw responses

--save-raw DIR writes each raw API response as JSON, named after the URL. It is useful for checking what Google returned, or for building test fixtures, after replacing the text with invented text. The files hold page text: never commit them.

Clone this wiki locally