-
Notifications
You must be signed in to change notification settings - Fork 0
ralph autoresearch rubrics difference
Nicolas Cravino edited this page Jul 29, 2026
·
1 revision
id: ralph-autoresearch-rubrics-difference title: Ralph, Autoresearch, and Rubrics: What Is Actually Different? tags: [llm] category: agentic-frameworks published: 2026-07-29 created: 2026-07-29 updated: 2026-07-29 freshness: fresh date_source: git-commit source_file: loops-Ralph-autoresearch-rubrics-what-is-actually-the-difference.md
Ralph loops, Karpathy-style autoresearch, and rubric-judged loops are iterations of one pattern — agent loop, external state, human sets the goal once — and the real axis of difference is who holds the evaluator: a deterministic check or another LLM.
- Ralph: fresh context per iteration, one unit of work at a time until the backlog is empty — best when a spec/PRD decomposes cleanly; fresh context reduces drift and rot. [^src:articles-published §ralph-autoresearch-rubrics-difference]
- Autoresearch: keep-and-roll-back against a fixed measurable metric (latency, accuracy, cost) that acts as the safety rail, discovering gains not specified in advance. [^src:articles-published §ralph-autoresearch-rubrics-difference]
- Rubric loops: LLM-as-a-judge with per-criterion feedback for fuzzy quality — powerful but drift-prone, so keep them bounded and in-session. [^src:articles-published §ralph-autoresearch-rubrics-difference]
- They compose (Ralph with a rubric; autoresearch inside a Ralph backlog), with the default hybrid: the LLM proposes, the deterministic check disposes. If the check can be coded, make it code — that is the version safe to run overnight. [^src:articles-published §ralph-autoresearch-rubrics-difference]