Marlin v1.0 - Personal domain search index

First public release of Marlin: a personal, single-user search index you fork and run yourself.

Ingest domains → fetch homepages → summarize with an OpenAI-compatible LM (LM Studio or rented vLLM) → search summaries with category/tag/geo filters. Discovery is list ingest plus link-following after catalog (optional BFS spider; not required). Fetch and LM stay separate so the GPU isn’t idle on HTTP.

What’s in v1

• Pipeline: ingest, fetcher, LM worker, steward (spiral apex auto-block), API + web UI
• Catalog: one structured LM call per host (Gemma 4 E4B defaults); skip LM for empty / bot-check / parked pages
• Staging in Postgres: page text + outbound hosts wiped on done (not a permanent corpus)
• Forkable policy under data/: TLD whitelist, crawl/language priorities, LM + worker profiles, catalog + steward policies, denylists, label aliases, seed lists
• Ops UI: search, ignore lists, Dashboard, Workers, Analyze (labels / platforms / steward)
• Scale knobs that mattered in practice: category-weighted outbound enqueue, subdomain caps, blocked-apex denylist, soft-start LM concurrency for vLLM
• Docs: Getting Started, Vast templates, agent invariants (AGENTS.md)

Explicit non-goals

No crawl/link graph, no full HTML corpus, no IP/TLS scanning, no Redis, no multi-tenant auth, no recrawl scheduler.

As-is

MIT, provided without support expectation — fork and own it. Story of the ~560k weekend run: The Birth. Operator guide: docs/GETTING_STARTED.md.