Skip to content

v1.0.0

Choose a tag to compare

View all tags
@vignesh-nagarajan-vn vignesh-nagarajan-vn released this 06 Aug 04:58
· 9 commits to main since this release
v1.0.0
9a65373

Quarry-LDR turns a research topic into a cited report, now as markdown plus a branded PDF. The design still rests on one idea: the local GPU compresses roughly 750K tokens of raw scraped web text into roughly 60K tokens of deduplicated, reranked evidence. What changed in v1 is who reasons over it: by default the GPU is also the brain. Qwen3-8B plans the research and writes every section, Qwen3-4B triages evidence and audits coverage, and a cross-encoder verifies every cited sentence against its cited chunks before render. A default run makes zero API calls and needs no API key.

Engine What happens Cost per report
Naive (no Quarry) ~750K raw tokens through Opus $10 to $15
premium local GPU compresses; Claude plans, audits, writes $1.36 to $2.88, measured
assisted local plan and draft; Haiku 4.5 gap audits and one polish pass $0.02 to $0.12, measured
local (default) everything runs on your GPU $0.00, enforced by the ledger

The measured numbers come from live validation runs on one laptop card, an NVIDIA RTX 5060 Mobile with 8 GB of VRAM, where the 8B writer generates at 35.3 tokens per second. Four unedited sample reports covering all three engine tiers ship in pdf-reports/ with the numbers behind each, from the $0.00 local run to the $2.55 Claude-written premium report.

Highlights

  • Fifteen checkpointed stages from plan to render, now including VERIFY: every cited sentence is scored against its cited chunks by the cross-encoder, and what the evidence does not support is rewritten on the 4B model or dropped. Validated live in both directions, including organic drops at the production floor.
  • The $0.00 default is enforced, not asserted: local calls are metered into the same cost ledger as API calls, with real token counts from llama-server's usage block at zero price.
  • Three engine tiers behind one config key (engine.mode), sharing one pipeline, one ledger, and one verification gate. The premium path keeps v0's prompt-cached corpus economics, re-confirmed under v1 at $2.55 with one 60K-token cache write and nine reads.
  • The local models run as Q4_K_M GGUF quantizations via llama.cpp (the 8B writer at 16K context with flash attention and a q8_0 KV cache); the bge-m3 embedder and bge-reranker-v2-m3 cross-encoder run at fp16. Four models share one 8 GB card under a VRAM arbiter with a hard 6.5 GB budget.
  • Reports ship as markdown plus a branded Typst PDF: cover page, contents, run charts (compression funnel, source mix, coverage, claim verification), references resolving to URL plus chunk offsets, and a run-facts appendix. The PDF path is fail-soft; the markdown never depends on it.
  • The assisted polish is guarded: if Haiku's polish changes the citation-marker multiset, the polish is discarded and the local draft stands. Both branches proven live.
  • An interrupted run resumes from its last completed stage; SearXNG metasearch and a polite fetcher (robots.txt respected, rate limited, cached); Windows, Linux, and WSL2 all work.

Getting Started

An NVIDIA GPU (8 GB VRAM or more) and Docker for SearXNG are required. An Anthropic API key is needed only for the assisted and premium engines; the default engine runs without one.

git clone https://github.com/vignesh-nagarajan-vn/Quarry-LDR
cd Quarry-LDR
make bootstrap                # fresh Windows without GNU make: powershell -ExecutionPolicy Bypass -File scripts/bootstrap.ps1
make searxng                  # starts local search in Docker
uv run python scripts/download_models.py   # fetches llama-server and both GGUFs
uv run quarry verify          # preflight check with remediation hints
uv run quarry research "your topic"

The report and its PDF land in data/reports/ with a cost ledger and a run manifest. make smoke-local runs a $0 end-to-end rehearsal if you want proof before a real topic. To buy Claude reasoning instead, add ANTHROPIC_API_KEY to .env and pass --engine assisted or --engine premium. Every flag, engine, config knob, and operational caution: the Run Guide.

From the Beta

v0.9.0-beta was the hybrid architecture: the GPU compressed, Claude reasoned, every run cost dollars. v1.0.0 keeps that behavior intact as the premium engine and makes it optional. New since the beta: the engine tiers, the local Qwen3 writer with per-section synthesis, the VERIFY entailment gate in every mode, the branded PDF with run charts, request-timeout and schema hardening from live validation, and a documentation set that includes a full run guide. The hybrid remains preserved on archive/v0-hybrid-api and under its own tag.

Documentation

  • Run Guide: setup, all three engines with commands and measured costs, configuration, resume, rehearsals.
  • Architecture: pipeline diagram, engine routing, VRAM arbiter rules, full configuration reference.
  • Sample Reports: four real pipeline outputs across all three engines, $0.00 to $2.55.
  • First Test: the v0 validation record; cost anatomy and the $2.88 premium example report.
  • Troubleshooting: symptoms, causes, exact fixes.

Known Limitations

  • NVIDIA CUDA only; developed and tested on 8 GB cards (RTX 5060 Mobile, desktop RTX 4060). More VRAM is fine, smaller cards are untested.
  • Local-engine prose is an 8B model's: verified and cited, but flatter than Claude's. The quality ceiling is a config swap away; that is the point of the engine tiers.
  • Upstream search engines throttle IPs that burst many queries in a short window. The fetch cache and checkpointed resume soften it; the Run Guide documents probing and pacing. The premium sample ran during such a window and stands on fewer sources than premium normally gathers; it says so in its own prose.

License: MIT.

Assets 2
Loading