autore runs one model-directed agent loop. The model decides whether to answer directly, use registered tools, recover from failures, request input, inspect evidence, or stop. The harness enforces permissions, budgets, persistence, validation, and deterministic evaluation.
python3 -m pip install -e .
autore "Explain this concept in plain language" --llm-provider ollama --llm-model ollama/qwen3.5:latestThe default model is openai/gpt-5.6-terra through OpenRouter. Put OPENROUTER_API_KEY in .env.local or export it in your shell; never store the key in source code. You can alternatively set OPENAI_API_KEY, ANTHROPIC_API_KEY, or run a compatible Ollama model for native tool calling. A live provider is required for tool-using runs; the harness will not fabricate a tool trajectory offline.
Isolated research workers are enabled by default with openai/gpt-5-mini. Set RESEARCH_HARNESS_WORKER_MODELS or pass --worker-models with comma-separated cheaper OpenRouter model ids to change the pool; use an empty value to disable delegation. Per-worker prompt-plus-completion budget defaults to 50,000 tokens and can be changed with --worker-max-tokens. Terra remains the sole manager and final decision-maker. In its normal model-directed loop, it decides whether to delegate, how many workers to call within configured budgets, their assignments, and how to use their observations. Each worker gets a launch-time read-only snapshot of the parent objective, task constraints, current authoritative baseline, and validated learnings. Once a measured champion exists, worker search lanes use a repeating 3:1 allocation: three champion mutations that preserve the exact incumbent and isolate distinct bounded changes, then one independent exploration branch. Workers return artifacts with concise evidence, failures, limitations, and follow-up ideas from isolated workspaces; nothing is automatically merged, evaluated, promoted, or adopted. Only completed workers with at least one usable artifact count as covered research. Failed, partial, budget-exhausted, and artifact-free assignments remain explicit unresolved coverage.
autore "Research a current technical question" --retriever web --max-iterations 8
autore "Find source figures for this claim" --max-tool-calls 48 --max-runtime-seconds 300
autore "Analyze this repository's public architecture" --retriever local
autore "Optimize the prediction-market challenge" --grader --grader-loops 8
autore "Optimize from this exact strategy" --grader --baseline-file strategy.py
autore --resume-run 001_run_interrupted-research
autore eval
autore eval --run-id 001_run_completed --eval factual-accuracy --eval completeness
autore eval --run-id 001_run_completed --all
autore --helpLive runs use the resize-aware Autore terminal view with restrained NeoFetch-style accents, cyan interaction focus, and semantic success, warning, and failure colors. Arrow keys preserve selection across redraws, Enter opens details, and Escape collapses without exiting. See the TUI guide for color fallbacks, debug logging, controls, and visual regression captures.
All production tasks start the same lead ResearchAgent; tools are optional capabilities selected by that controller. delegate_task is one optional capability, not a workflow phase or routing policy. Multiple calls requested in one model turn may run concurrently, subject only to infrastructure limits. Worker handoffs return to the lead as observations, while every official evaluation, comparison, budget decision, frontier validation, promotion, and final synthesis stays on the lead trajectory. With --grader, the controller can submit complete candidates, preserve evidence-backed learnings and negative results, and validate a frozen frontier on reserved ranges. No score is fabricated and no candidate is promoted without an eligible official measurement. New runs do not ingest prior outputs.
The guided CLI does not ask the user to choose an evidence backend. Paper discovery is exposed to the model as one search_research_papers tool; it queries configured arXiv, OpenAlex, and Semantic Scholar adapters concurrently, merges duplicates and complementary metadata, ranks the consolidated set, preserves provider provenance, and reports isolated provider failures compactly. --retriever remains an advanced capability restriction for offline runs, reproducible tests, or environments where only a specific backend should be enabled internally.
flowchart TD
goal["Goal"] --> lead["Lead agent"]
lead --> decision{"Model action"}
decision -->|tool call| registry["ToolRegistry\nregistered capability"]
registry -->|ToolResult observation| lead
decision -->|final answer| done["Lead synthesizes result"]
The public loop is singular: Goal → lead agent → model action → ToolRegistry → observation returned to lead → repeat, reserved validation, or final synthesis.
Grader runs separate the full audit transcript from model working context. Every iteration receives a deterministic checkpoint containing the strategy ledger, exact champion/latest code, official edge metrics, fetched literature extracts, and only the newest unresolved tool exchange. Older tool chatter remains in agent_messages.json and agent_events.jsonl but is not replayed to the model. Fetched documents are keyed by canonical URL and served from the artifact cache on repeat requests.
Prediction-market strategies use three closed, single-purpose contracts: register_prediction_market_candidate requires code; register_prediction_market_candidate_path requires candidate_path; and evaluate_prediction_market_candidate requires candidate_id and accepts neither source selector. Registration returns a stable candidate ID, artifact path, and SHA-256 without spending an official evaluation. The evaluator verifies the registered bytes, performs duplicate detection, and then runs the official scorer. After an eligible champion exists, normal evaluated candidates must name and structurally retain that champion; independent rewrites are rejected without spending evaluator budget unless the deterministic plateau policy has issued a one-candidate architectural-reset authorization. Canonical trial records store the measured champion trial as the parent, so candidate-graph edges resolve to real nodes. Before the first optimization model turn, the exact objective baseline, --baseline-file, or adapter seed is materialized and hash-verified; exact user-supplied Python is syntax-checked before registration, and invalid ingestion stops without spending a model turn or official evaluation. Multiline TUI pastes preserve source newlines. Evaluation budgets, trials, comparison, validation, and promotion remain lead-only.
Document download bytes and model-facing extracted characters are separate limits, so ordinary HTML pages larger than the extract budget are parsed and compacted instead of rejected. Transient provider timeouts are retried once at the agent-loop boundary with the same working state after transport retries are exhausted; malformed or otherwise deterministic provider errors still terminate immediately. Kimi uses a 120-second default request timeout, configurable with RESEARCH_HARNESS_LLM_TIMEOUT_SECONDS.
Each run writes outputs/<run>/:
final_report.md— validated answer or explicitly labelled partial result.run_state.json— actual model turns, tool calls, observations, budgets, and termination state.checkpoint.json— atomic restart state preserving messages, events, counters, sources, worker handoffs, and optimization history.side_effects.json— write-ahead ledger that prevents completed or uncertain grader calls, mutations, and promotions from being replayed.agent_messages.json— provider-neutral transcript preserving tool-call IDs and results.agent_events.jsonl— the single canonical, append-only full trace with stable IDs and ordering across resume.agent_timeline.png/agent_timeline.svg— readable overview and complete event timeline with semantic operation colors and independent status styling.failures.json— canonical, append-only, secret-safe ledger for unsuccessful, interrupted, skipped, partial, and not-executed operations, including delivery and recovery metadata.failed_paths.json— backward-compatible projection of legacy failure records.cost.json— model usage and estimated cost.learnings.mdandlearnings.jsonl— created when the controller records grader-backed findings or dead ends.datasets/<id>.json— complete normalized table/numeric extraction plus source, section/table, and row provenance.extractions/figures-<id>.json— persisted figure inspection results, captions, image URLs, aspect ratios, and source metadata.document_analyses/<id>.json— LLM document analysis with source-stated findings separated from model inferences.charts/<id>.svgandcharts/<id>.json— deterministic SVG chart and reproducible dataset/config provenance.candidate_graph.json,.svg, and.png— immutable optimization candidates and typed lineage edges.experiment_ledger.json— hypotheses, mechanism classes, parent/reset branches, configurations, outcomes, and negative results.search_diagnostics.json— novelty, similarity, repetition, category coverage, plateau, reset, and validation-gap metrics.optimization_decisions.json— deterministic continue/reset/stop decisions.validation_frontier.json— reserved-range measurements and the final promotion decision.candidate_comparisons/*.json— deterministic, non-promoting comparisons between any two persisted official evaluations, including scores, component metrics, evaluator compatibility, runtimes, failures, and output-code diffs.champion_history.json— ordered promotion decisions, separate from lineage. Deprecatedchampion_tree.*compatibility files may also be emitted during migration.
For example, after fetch_document returns a verified source ID, the model may call extract_structured_data with that ID, then call generate_svg_chart using the returned dataset ID and column names. It can call analyze_research_document when a paper needs a grounded methodology/results reading. These calls are optional; they do not create a paper-processing pipeline. Chart generation rejects missing datasets, nonnumeric values, and incompatible selected units rather than guessing conversions.
Workspace reads are deny-by-default for sensitive files such as .env and .git. Document retrieval validates every DNS-resolved host and redirect against private, loopback, link-local, and reserved addresses. HTML documents may be rendered into compact Markdown through curl.md after the target URL passes those checks; direct fetch remains available as the fallback.
Model turns and wall-clock runtime are unbounded when their CLI limits are omitted. Explicit --max-iterations and --max-runtime-seconds values remain hard limits. The guided CLI defaults to eight official candidate evaluations for grader runs. Fetching an arXiv /abs/ URL resolves to its PDF and extracts bounded page text with PDF-page locators; the abstract page is not treated as the paper body.
When a configured model-iteration limit ends an optimization run after at least one eligible official measurement, deterministic finalization still emits the complete best-so-far output set. final_report.md includes the exact winning candidate and accumulated learnings; optimal_code.py is the one canonical Python output, while immutable candidate source is retained as content-addressed text and measurements remain in JSON history. optimization_result.json points to that winner. The run retains the truthful budget_exhausted termination reason, and optimization_result.json labels the outcome best_at_iteration_limit rather than implying unconstrained convergence.
Failed, unmeasured, forged, or ineligible grader attempts never create or replace optimal_code.py. If no provenance-backed eligible official measurement exists, final_report.md embeds exact hash-verified fallback code under “Best available unvalidated code,” best_available_code.py retains those bytes, and no optimization result or canonical optimal output is published. Every optimization report includes a Python block plus termination, status, failure counts, unresolved work, recovery guidance, and artifact paths.
autore eval opens a resize-aware multi-select picker for completed-run evaluations. The non-interactive --eval and --all forms are suitable for CI. Deterministic citation and tool-efficiency checks and structured LLM judges receive the original task, final answer, retained excerpts, citation mapping, and tool statistics. Results are written under eval_results/, never into or over the source run.
Search tools must return relevant records or an explicit error. A DuckDuckGo bot challenge is surfaced as a tool failure, not bypassed. The agent can recover with registered primary-source APIs or direct public URLs through fetch_document, figure inspection, or the bounded terminal tool; arXiv exact IDs are fetched directly and unrelated papers are rejected before persistence. The harness compacts source records before returning them to the model while retaining complete source metadata in the artifact store.
env PYTHONPYCACHEPREFIX=/private/tmp/research-harness-pycache python3 -m unittest tests.test_research_agent
env PYTHONPYCACHEPREFIX=/private/tmp/research-harness-pycache python3 -m unittest discover -s tests
autore --helpRun the Bump version workflow from the GitHub Actions tab on main. It increments the patch component by default (0.1.0 → 0.1.1); choose minor or major in the workflow form when needed. The workflow tests the repository, synchronizes setup.cfg and research_harness/__init__.py, commits the change, and creates and pushes the matching vX.Y.Z tag.