Skip to content

Releases: bhanneke/E2ER-project

v0.8.1

Choose a tag to compare

@github-actions github-actions released this 29 May 23:29
dba74a0

Stability + corpus extensions. Bug fixes from a full code review —
safety (Allium guardrails no longer bypassed without a
data_dictionary.json; SQLite Allium-approval workflow works; SSRF
hostname resolution), Lane-A robustness (strategist JSON guards,
mechanism-gate, resume-status, single-order cascade), Lane B/C wins
(storage citations; OpenAlex/S2 null crash; e2er run --acknowledge- unproven; FileToolHandler sandbox; OpenRouter content=""). Plus
user-driven additions: structured GitHub issue templates for data-source
and literature-provider requests, and LOCAL_DATA_DIR extensions
(comma-separated roots, recursive walk, PDFs staged into
workspace/literature/ with read_reference(path=…)).

Cross-lane

  • Structured GitHub issue templates for the most common asks:
    data_source_request (provider, auth, coverage, example RQ) and
    literature_provider_request (capability, gap, auth). Both routed by
    lane-* / provider-request labels. Generic feature requests still go
    via feature_request.md.

Lane A — Pipeline

  • Fix: malformed strategist JSON no longer crashes the paper.
    ceiling_check and run_self_attack did a bare json.loads on LLM
    output — truncated/invalid JSON raised and failed the whole run. They now
    use the tolerant extract_json and skip malformed WorkOrder/finding
    items instead of raising.
  • Fix: a missing mechanism-reviewer score can no longer be silently
    accepted.
    The Rule-1 mechanism gate no-op'd when the mechanism score was
    absent, letting a paper ACCEPT on the other reviewers' average. A missing
    (but expected) mechanism score now forces MAJOR_REVISION.
  • Fix: resume tolerates a bad/legacy persisted status. PaperStatus( state.last_status) could raise ValueError and wedge a completed paper
    into FAILED on resume; it's now coerced with a safe fallback.
  • Fix: tier-0 context builder handles explicit-null manifest fields
    (datasets: null / research_question: null) instead of TypeError.

Lane B — Literature

  • Fix: store_paper persists citation counts. citations was in the
    ON CONFLICT DO UPDATE clause but missing from the INSERT column list, so
    inserts dropped the count and conflict-updates zeroed it. Added to the
    insert.
  • LOCAL_DATA_DIR extensions. Accepts a comma-separated list of
    roots, an opt-in LOCAL_DATA_DIR_RECURSIVE=true to walk
    subdirectories (paths under workspace/data/ are preserved), and now
    also stages *.pdf into workspace/literature/. The bib-relevant
    specialists' reference summary lists those local PDFs so they can be
    read via the new read_reference(path=...). New
    src/modules/local_corpus.py consolidates parsing/walking;
    LocalBibLibrary uses it for .bib discovery across multiple roots.
  • read_reference accepts a new path argument (workspace-
    relative) for the staged local PDFs — no download, no auth, sandboxed
    under the workspace root.

Lane C — Data

  • Fix (safety): guardrails no longer fully bypassed without a data
    dictionary.
    _query_allium only ran validate_all when a
    data_dictionary.json was present, so a production query with no
    dictionary ran with ZERO validation. Now the structural rules (no
    SELECT *, time-bound) and feasibility-first/approval gate always fire;
    only the field-whitelist (Rule 2) is dictionary-gated (skipped with a
    warning).
  • Fix: audit inserts generate app-side UUIDs. log_query /
    create_approval_request relied on a DB id default; SQLite has none, so
    id was NULL and the approval-request join silently never surfaced
    pending production queries on the default SQLite DB. Now both generate a
    uuid4() client-side — the Allium approval workflow works on SQLite.

Cross-lane

  • Fix: cost-estimate labeling for the codex/gemini backends. app.py
    checked codex_cli/gemini_cli, but the real backend literals are
    codex/gemini, so synthetic cost figures were mislabeled as real.
  • Fix: literature_kb_enabled honors DATABASE_URL. It keyed off
    legacy postgres_url/db_password, leaving the pgvector KB silently off
    for the documented DATABASE_URL=postgresql://… path. Now derived from
    the resolved DB URL.
  • Fix (security): SSRF guard resolves hostnames. _check_url only
    blocked literal private IPs; a hostname (e.g. metadata.google.internal
    → 169.254.x, or localhost) slipped past. It now resolves the host and
    blocks if any resolved address is private/loopback/link-local.
  • Fix: e2er run --acknowledge-unproven flag. The CLI hardcoded
    acknowledge_unproven_tuple=True, silently disabling the $1 first-run
    floor (and the README documented a flag that didn't exist). The flag now
    exists (default off → floor enforced for metered backends); the $0
    flat-rate CLI backends (claude_code/codex/gemini) auto-acknowledge.
  • Fix: single-order dispatch gets the cascade guard. The missing-
    canonical-artifact check ran only in execute_parallel; a lone specialist
    could "succeed" without its artifact and starve downstream work. Extracted
    assert_artifacts_written, now applied to both paths.
  • Fix: FileToolHandler sandbox uses path containment, not a string
    prefix
    (a sibling workspace with a prefix name could escape).
  • Fix: OpenRouter tool-only turns send content="" instead of null
    (some OpenAI-compatible servers reject null content + tool_calls).

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 29 May 00:57
dd29c8f

Pluggable data & literature providers. Specialists now discover data
sources in light of the research question — FRED and yfinance reach the
tool loop via list_data_sources + a unified fetch_data, and Allium sits
behind a Warehouse capability (its 5 guardrails unchanged). They also
pull the researcher's own reference library (local .bib, LOCAL_DATA_DIR,
and Zotero via the Web API) and read full-text PDFs (read_reference).
Both lanes are now registry-pluggable, so new providers are drop-in.

Cross-lane

  • scripts/live_check.py — live smoke harness. Exercises the real
    data/literature provider paths (yfinance, FRED, Allium connectivity,
    OpenAlex search, read_reference on an OA PDF, Zotero library) against
    live services, auto-skipping providers without credentials. No LLM calls
    (free). Complements make smoke (offline/mocked) and make smoke-paid
    (full LLM run). Run: python scripts/live_check.py.

Lane C — Data

  • Allium folded behind a Warehouse capability (M3b of
    docs/MODULARIZATION_PLAN.md).
    Allium is now a first-class registered
    provider: AlliumWarehouse owns its card(), tools() (→ ALLIUM_TOOLS)
    and handler() (→ DeferredAlliumToolHandler); _run_pipeline assembles
    it by iterating warehouses(settings) instead of hardcoding, and the
    catalog builds its card from the warehouse. Pure refactor — same condition
    (Allium key present), same tools, the 5 QueryValidator guardrails and
    approval flow are untouched
    , and has_allium/data_module_enabled are
    unchanged. Completes the Lane-C registry (series + warehouse).
  • Series data in the agent loop + RQ-aware discovery (M3a of
    docs/MODULARIZATION_PLAN.md).
    FRED and yfinance are no longer
    CLI-only — specialists reach them in the tool loop. New SeriesFetcher
    capability + data registry (providers.py, registry.py) mirror the
    Lane-B pattern. Two new tools: list_data_sources (serves the registry
    catalog so the agent picks the right source for the research question)
    and a unified fetch_data(provider, method, params). Allium is unchanged
    — it keeps its guarded query_allium tool and is advertised in the
    catalog (the 5 guardrails are untouched). Series tools are always on
    (yfinance needs no key); budgeted (_MAX_FETCHES=20). M3b will fold
    Allium behind a Warehouse capability into the same registry.

Lane B — Literature

  • Fix: literature search crashed on OpenAlex/S2 explicit nulls. A live
    search returned 0 papers because openalex._parse raised
    'NoneType' object has no attribute 'get' on a result whose
    primary_location.source (or open_access / authorships) was an
    explicit null.get(k, default) doesn't apply the default for a
    present-but-null value. Both parsers now guard with or {} / or [].
    Regression tests added (the mocked payloads previously only used
    well-formed fields, so the bug only surfaced live).
  • Full-text read_reference tool (M2.5 of docs/MODULARIZATION_PLAN.md).
    Specialists can now read a reference's PDF in full to deepen the lit
    review, not just its abstract. New read_reference literature tool takes
    a pdf_url (surfaced in search/fetch results and on [PDF]-marked
    reference-list entries, incl. Zotero attachments) or a doi (resolves an
    open-access PDF). Downloads (auth'd for Zotero hrefs, /file/view
    /file), extracts text via pypdf (pdf.py), and returns it
    truncated to ~20K chars. Tightly budgeted (_MAX_READS=6 + per-read char
    cap) given the prior 522K-token literature blowup. fetch_bytes gained a
    max_bytes override (PDFs exceed the 2 MB default). New pypdf dep. New
    ZoteroLibrary ReferenceLibrary reads the researcher's Zotero library
    via the Web API's native JSON (zotero.py), maps items to
    PaperMetadata, and captures each item's primary PDF attachment href
    (for the planned on-demand read_reference tool, M2.5). Config:
    ZOTERO_API_KEY + one of ZOTERO_USER_ID / ZOTERO_GROUP_ID; merged
    into the reference summary after local .bib, deduped by (title, year).
    Unset → no-op. Sync fetch_text_sync helper added for the (sync)
    reference-library path. Degrades to [] on any Zotero error — can't
    break paper creation.
  • Provider interface + registry (M1 of docs/MODULARIZATION_PLAN.md).
    Formalized the de-facto interface the source modules already shared into
    capability sub-types — SearchSource (web discovery; OpenAlex, arXiv,
    Semantic Scholar) and ReferenceLibrary (the researcher's own corpus;
    LocalBibLibrary over LITERATURE_BIBTEX_FILE + LOCAL_DATA_DIR) — in
    new providers.py / registry.py. LiteratureToolHandler and
    _load_reference_summary now iterate the registry instead of hardcoding
    provider names. Pure refactor: the search (OpenAlex→arXiv) and DOI-fetch
    (OpenAlex→S2) fallback chains are reproduced exactly; +13 tests, no
    behaviour change. This is the seam Zotero (M2) and Citavi (M4) plug into.

v0.7.3

Choose a tag to compare

@github-actions github-actions released this 26 May 21:00
8348daf

Fix the patch_revisor section-target resolution bug surfaced by
the v0.7.2 live re-validation on paper 7f4f2363. The drafter
got a paper all the way through to the revision phase (v0.7.0's
verify_numbers parser fix worked), but the patch_revisor emitted
edits targeting canonical section names (section:results,
section:mechanism) that didn't exist in the actual draft. The
merger reported "target region not found" with no hint and the
paper REJECTED on parser bugs, not real hallucinations — for the
second release in a row.

Lane A — Pipeline

  • Merger emits "did you mean..." suggestions on section/table
    not-found.
    When apply_edit can't resolve a section: or
    table: target, the error message now appends the list of
    available section titles or labelled tables in the document.
    Example before/after:
    • Before: target region 'section:results' not found in document
    • After: target region 'section:results' not found in document (available sections: 'Introduction', 'Identification Strategy', 'Empirical Strategy', 'Discussion')
      Two new public helpers: list_section_titles(text) and
      list_table_labels(text). Suggestions are suppressed when the
      list is empty (avoids the misleading
      (available sections: ) suffix on minimal LaTeX skeletons).
      Universal targets (paper:full / abstract / references)
      don't get suggestions.
  • writing/scoped-revision.md skill update. New section
    ("Before you compose any edits — list the draft's actual
    targets") instructs the patch_revisor to grep the draft for
    \section{...} and \label{tab:...} lines before composing
    patches. Explains the case-insensitive substring matching the
    merger uses, the common failure mode (canonical-name vs
    actual-heading mismatch), and the paper:full fallback for
    findings that don't have a dedicated section.

Test counts

  • Mocked suite: 598 passed (was 590 in v0.7.2; +8 here).
  • 8 new tests in tests/pipeline/test_patch_merger.py:
    • 4 for list_section_titles and list_table_labels helpers.
    • 4 for the extended error: section suggestions, table
      suggestions, no suggestion for non-section/table targets, no
      misleading suffix when the list is empty.

v0.7.2

Choose a tag to compare

@github-actions github-actions released this 26 May 15:52
2802329

Closes the v0.7.1-noted follow-up: a CLI command to resume
paused / failed / zombie papers. Completes the status / cancel /
resume trio so the operator never has to drop down to curl.

Cross-lane

  • e2er resume <paper_id> — restart a paused or failed
    paper from the terminal. Optional --max-cost N raises the
    cap atomically with the resume (sent through to the v0.5+
    ResumeRequest body). Surfaces the paper's title + previous
    status + cap delta + last_error before issuing the POST, so
    the operator knows what they're restarting. Unlike status
    and cancel, this command DOES auto-start uvicorn — the user
    is asking the paper to start running again, so the server
    needs to be up.
    • 200 → prints the new transient status (resuming) +
      dashboard URL, optionally tails to terminal via --tail
    • 400 → surfaces the validation detail (e.g. non-positive
      cap) directly so the user can fix and retry
    • 409 → "already running" with a hint to e2er cancel first
    • 404 → "paper not found"
  • 9 new regression tests in tests/test_cli_status.py
    covering: no-cap-change happy path, cap-raise happy path,
    completed-paper short-circuit, 400 / 409 / 503 / 404 error
    paths, --tail integration, the API-unreachable branch.

Test counts

  • Mocked suite: 590 passed (was 581 in v0.7.1; +9 here).

v0.7.1

Choose a tag to compare

@github-actions github-actions released this 26 May 14:02
dbd88e7

Two new lightweight CLI commands surfaced by the v0.7.0
fresh-install UX test: when e2er run's tailer times out (or
the user ^C's it), there was no scripted way to re-attach,
inspect the current state, or cancel a runaway paper without
opening the dashboard.

Cross-lane

  • e2er status <paper_id> — one-shot snapshot of a paper:
    status, mode/methodology, cost meter (with the
    cost_is_estimate marker on CLI backends), specialist call
    count, token total, workspace path, dashboard URL. Shows
    last_error verbatim when present so the user can diagnose
    REJECTED / PAUSED / FAILED without parsing the events log.
    With --tail, re-uses the same polling loop e2er run uses
    so the user can re-attach after ^C. Short-circuits on already-
    terminal status (no wasted polls). Hits the local API by
    default; respects E2ER_API_URL for remote inspection.
  • e2er cancel <paper_id> — POSTs the /cancel endpoint
    with a confirmation prompt (skippable via --yes). Surfaces
    the title + current status + spend-so-far before the user
    confirms so they don't cancel by accident. Terminal-status
    short-circuit. Treats post-cancel 404 as success (the paper
    finished while we were asking; that's what the user wanted).
    Brief post-cancel poll so the user sees the CANCELLED
    transition land before the shell returns.
  • Cost output now formats with two decimals. Pre-fix
    e2er status showed $8.462921999999999; now $8.46. Float
    noise was reaching the user-facing string when the API
    returned high-precision cost totals.
  • _poll_status now treats rejected as terminal. Pre-fix
    the e2er run tailer kept polling forever on REJECTED papers
    (a v0.5+ status it didn't know about). Observed during fresh-
    install testing on paper 2ca473aa.

Test counts

  • Mocked suite: 581 passed (was 554 in v0.7.0; +27 cli_status).
  • 27 new tests in tests/test_cli_status.py covering
    formatters, exit codes, the unreachable-API branch, the
    confirmation prompt, and the post-cancel-404 race handling.

Known follow-up (v0.7.2 candidate)

  • e2er resume <paper_id> — natural complement to cancel.
    PAUSED papers can be resumed via curl POST /resume today;
    a CLI command would close the same UX gap that status and
    cancel close. Out of scope for v0.7.1.

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 24 May 21:07
f296dc6

Better onboarding + a verify_numbers parser fix, bundled.
Surfaced by direct user feedback ("pip install e2er and then
what?") and by the v0.6.1 live run on paper f79b7cd9 that hit
two false-positive critical mismatches caused by parser bugs.

Cross-lane

  • New e2er init command — guided first-paper setup wizard.
    Closes the post-pip install e2er onboarding gap. Walks the
    user through 4 steps (LLM backend pick + prereq check, data
    module on/off, optional BibTeX path, optional Postgres
    DATABASE_URL), an optional GitHub-integration prompt, then
    writes ./.env (with confirm-overwrite), runs e2er install-skills, and prints three concrete example research
    questions to copy. Hand-rolled stdin wizard — no new
    dependencies (no click / prompt_toolkit). TTY-detected so
    non-interactive invocations exit with a helpful one-line guide
    instead of blocking on input(). Secrets discipline: GitHub
    PATs and API keys collected during the wizard are written to
    .env as comments, never as live env vars. 24 new unit tests
    in tests/test_cli_init.py. README quickstart updated to lead
    with e2er init.

Lane A — Pipeline

  • Fix two verify_numbers false-positives: ISO date strings
    in column headers (2021-03-01) were being parsed as the bare
    year 2021, false-positive-mismatching against unrelated
    source values; and LaTeX brace-protected thousands separators
    (1{,}573.89 — the form that survives math mode) were being
    split into two bogus numbers (1 and 573.89). Both surfaced
    on the v0.6.1 live-validation paper f79b7cd9, which was
    REJECTED entirely on parser bugs rather than real
    hallucinations. New _normalize_cell(cell) helper runs
    before _NUMBER_RE on each tabular cell: normalizes {,}
    , so the existing thousands branch picks the value up
    intact, then strips ISO / slash / US date patterns so years
    inside dates don't leak as numeric claims. Bare years outside
    date context (e.g. Sample size & 2021) still extract — the
    fix is targeted at dates, not all four-digit numbers. 5 new
    regression tests in tests/pipeline/test_verify_numbers.py.

Test counts

  • Mocked suite: 554 passed (was 525 in v0.6.1; +24 wizard +5
    verify_numbers fix).

v0.6.1

Choose a tag to compare

@github-actions github-actions released this 23 May 14:54
71604ee

Hot-fix on v0.6.0 closing the known follow-up surfaced by the
v0.6.0 live run on paper 3bc58e8d.

Lane A — Pipeline

  • Iterative-phase guard extended to drop the legacy revisor
    on iterations 2+, alongside paper_drafter. Both specialists
    rewrite paper_draft.tex from scratch every time they run, so
    the same drift argument that motivated step 6's
    paper_drafter guard applies to revisor. v0.6.0's live run
    showed the strategist dispatching revisor during iterative
    phase even though paper_drafter was correctly skipped — the
    guard only filtered one. v0.6.1 closes the same door for both.
  • Strategist system prompt updated to name revisor
    explicitly alongside paper_drafter in the iterative-phase
    rule, and to point at patch_revisor (dispatched automatically
    by the runner's revision phase) as the legitimate path for
    scoped revisions. Removes the v0.6.0 ambiguity where the prompt
    said "use revisor only when upstream artifacts are updated"
    but the runner now expects no revisor calls in iterative
    phase at all.
  • test_section_writer_not_dropped_on_iteration_2 renamed to
    test_legitimate_specialists_not_dropped_on_iteration_2 and
    updated to reflect the v0.6.1 contract (was asserting revisor
    survives the guard, now asserts only the legitimate specialists
    do).
  • 4 new regression tests in test_iterative_phase_guard.py
    pinning the extended-guard contract.

Full mocked suite: 525 passed (was 521 in v0.6.0; +4 here).

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 23 May 10:07
9ad0fa1

Targeted-revision discipline. Closes the three drift sources
identified in docs/V0.6_PLAN.md: full-rewrite revisor on
MAJOR_REVISION, parallel-revisor write race in self-attack, and
unconstrained paper_drafter re-dispatch in the iterative phase.
Validated end-to-end on paper 3bc58e8d (2026-05-22, 38 min,
$12.36 est., Sonnet via Claude Code CLI).

Lane A — Pipeline

  • New patch_revisor specialist + deterministic merger.
    Replaces the pre-v0.6 revisor in every dispatch site. Writes
    structured edits to paper_draft.tex.edits.json; the merger
    (src/core/strategist/patch_merger.py) validates each edit's
    target against the work order's Finding list, applies
    in-scope edits to paper_draft.tex, and emits
    paper_draft.tex.applied.diff as a unified-diff audit
    artifact. One edit type supported in v0.6: replace_text
    with find / replace / find_must_be_unique. Target
    schema: section:<name> / table:<label> / references /
    abstract / paper:full. Edits whose target isn't in the
    findings are rejected before any text is touched.
  • Structured Finding dataclass + three collectors. New
    src/core/strategist/findings.py introduces the
    Finding(source, source_detail, target, severity, problem, suggested_fix) frozen dataclass that every revision source
    emits: collect_self_attack_findings,
    collect_verify_numbers_findings, collect_review_findings.
    combine_findings sorts severity-desc with source priority
    (verify_numbers > self_attack > review on ties — numerical
    mismatches are the most mechanical to fix).
  • MAJOR_REVISION wired through patch_revisor. Replaces the
    pre-v0.6 free-text-rationale path. Combines review findings +
    (when present) verify_numbers findings, serialises them as a
    JSON block in the work order's focus, dispatches
    patch_revisor, calls merge_patch_file. fully_applied →
    COMPLETED; missing patch file or failed edits → REJECTED with
    the first 3 failures named in last_error. Edge case:
    MAJOR_REVISION with no actionable findings short-circuits to
    COMPLETED without dispatching (avoids wasted spend).
  • Self-attack critical findings wired through patch_revisor.
    Eliminates the pre-v0.6 parallel-revisor write race. Top-3
    critical findings are batched into ONE patch_revisor call.
    Patch failures at this phase are advisory (logged, do NOT
    REJECT) — the downstream review phase catches what remains.
  • verify_numbers auto-patch loop (proactive gate). Pre-v0.6
    the gate was defensive: critical mismatch → REJECTED. v0.6
    closes the detect → patch → re-detect loop: critical mismatch →
    dispatch patch_revisor with the mismatch findings → re-run
    verify_numbers on the patched draft → REJECTED only if the
    second pass still has criticals. Bounded by
    _VERIFY_NUMBERS_AUTO_PATCH_BUDGET = 1 (single attempt) so a
    drafter that consistently disagrees with the source JSON
    doesn't loop. The persisted number_verification.json
    reflects the post-patch state.
  • Iterative-phase guard against paper_drafter re-dispatch.
    Two-layer defence:
    • Soft: strategist's system prompt instructs it to use
      section_writer (scoped to a section:<name> focus) on
      iterations 2+, never paper_drafter. Validated on the live
      run — strategist used section_writer 3× in iter 2.
    • Hard: _dispatch drops paper_drafter work orders when
      self._iteration >= 2, logging a warning. Catches the
      strategist if it ignores the soft instruction. iteration 0
      (initial) and iteration 1 (first iterative) still allow
      paper_drafter legitimately.
  • patch_revisor loads three skills. writing/scoped-revision
    (new — defines the patch-file shape with worked examples for
    verify_numbers and self_attack findings),
    writing/cite-numbers-by-source (v0.5 — same discipline as
    the drafter), writing/personal-style, reasoning/anti-slop.
  • Five architecture invariants pinned. Each step has a
    primary regression test; tests/pipeline/integration/test_v0_6_invariants.py
    documents all five in one place and adds cross-step
    assertions (legacy revisor never dispatched by v0.6 runner
    paths; both source types reach patch_revisor's focus when
    review + verify_numbers both have findings; merger
    scope-enforcement holds across dispatch sites).

Test counts

  • Mocked suite: 521 passed (was 422 in v0.5.0; +99 in v0.6).
  • New test modules: test_findings.py, test_patch_merger.py,
    test_patch_revision_wiring.py, test_self_attack_patch_wiring.py,
    test_verify_numbers_auto_patch.py, test_iterative_phase_guard.py,
    test_v0_6_invariants.py.

Known follow-ups (deferred to v0.6.1)

  • The legacy revisor specialist is no longer dispatched by v0.6
    runner code paths, but the strategist may still freely dispatch
    it from _run_iterative_phase. Surfaced by the 2026-05-22 live
    run (one revisor call in iterative phase). Candidate fix:
    extend the iterative-phase guard to also drop revisor on
    iterations 2+, OR update the strategist prompt to discourage
    it explicitly.

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 21 May 11:54
b4e5b55

Anti-hallucination & methodology-aware pipeline. Full design
record at docs/V0.5_PLAN.md. Motivated by v0.4.5 live tests on
papers a6182f08, cbe8048f, eea5379b, and validated end-to-end
against fresh live runs on 2026-05-20 (234a11ea, fd6bf64d) and
2026-05-21 (525fa03c) — see docs/V0.5_LIVE_VALIDATION.md.

Lane A — Pipeline

  • Programmatic anti-hallucination gate before review (new file
    src/core/pipeline/verify_numbers.py, 357 lines). Scans every number
    in \begin{tabular} blocks of paper_draft.tex and matches each
    against the flat numeric values from summary_statistics.json,
    estimation_results.json, robustness_results.json, and
    figure_spec.json. Tolerance 0.5% relative; integers ≥10 must be
    exact; signs must match. Critical mismatches (relative error >10%
    vs the closest source value) → status REJECTED and reviewers
    never spawn. Persists number_verification.json at workspace root
    on every run. Live-test paper a6182f08's "log realized variance
    falls by 0.41 ($t=-3.9$)" hallucination was caught by
    technical_reviewer only after 6 reviewers had run; this gate
    catches it deterministically, at $0, before any reviewer spends a
    token. Graceful skip when no source JSON files are present (warn +
    pass), so papers from before the analyst contract was tightened
    don't regress.
  • Methodology-aware phase routing. PipelineRunner.__init__ now
    accepts methodology: str = "empirical", propagated from
    papers.methodology through _run_pipeline and resume_paper in
    the API. For methodology == "theoretical",
    _reviewers_for_methodology() drops data_reviewer from the
    6-reviewer panel and _run_replication_phase() early-returns.
    Live-test paper cbe8048f burned ~$0.34 on a data_reviewer stub
    over an empty contract plus ~$0.43 on a replication packager with
    no replication artifacts — both wasted, both gone in v0.5.
  • New status PaperStatus.REJECTED, distinct from FAILED.
    FAILED is reserved for crashes; REJECTED means the pipeline
    ran successfully and the quality gate (verify_numbers,
    HARD_REJECT, MECHANISM_FAIL) returned a negative verdict.
    Resumable: transitions back to IDEA / IN_PROGRESS / REVIEW /
    REVISION / CANCELLED. _run_revision_phase's HARD_REJECT and
    MECHANISM_FAIL branches updated to emit REJECTED instead of
    FAILED. New IN_PROGRESS → REJECTED transition for the
    verify_numbers gate path.
  • BudgetExceededErrorPAUSED, resumable. New except BudgetExceededError branch in PipelineRunner.run(), alongside
    the existing CircuitBreakerError handler. Persists state, logs a
    paused_budget event with {spent, cap}, returns a structured
    {status: "paused", reason: "budget_exhausted", ...} payload.
    The operator raises --max-cost and POSTs
    /api/papers/{id}/resume; existing resume-from-disk logic picks
    up at the first incomplete phase. Previously a budget exhaustion
    was indistinguishable from a crash.
  • PAUSED and REJECTED rows now persist last_error on the
    papers table. Pre-v0.5, only FAILED and CANCELLED rows carried
    the error/reason; PAUSED and REJECTED dropped it at the SQL layer,
    leaving the dashboard with last_error=NULL and no way to render
    the budget breakdown, circuit-breaker specialist, or review-gate
    rationale. _update_status now treats PAUSED and REJECTED the
    same way as FAILED and CANCELLED for error preservation.
    Discovered while writing the v0.5 budget-pause regression test.
  • POST /api/papers/{id}/resume accepts max_cost_usd in the
    request body.
    Pre-v0.5 the endpoint silently ignored the body and
    read the cap from the DB row, so raising the cap on a budget-paused
    paper required a manual UPDATE papers SET max_cost_usd = ...
    beforehand (the workaround surfaced during the 2026-05-20 live
    validation). The endpoint now accepts an optional ResumeRequest
    body; a positive max_cost_usd is validated and persisted on the
    row atomically with the status reset, then passed to the runner.
    Zero or negative values 400. Calls without a body preserve the
    pre-v0.5 behaviour (use the existing row value).
  • paper_drafter, section_writer, abstract_writer, and
    revisor load a new writing/cite-numbers-by-source skill
    that
    teaches the cite-by-JSON-key discipline: every numeric value in
    the paper must trace to a value in summary_statistics.json,
    estimation_results.json, robustness_results.json, or
    figure_spec.json. HTML-comment markers (<!-- src: file#key -->)
    let verify_numbers mismatches name the exact source path the
    drafter should have used. Reduces hallucination rate in the first
    place; complements the post-hoc gate. Includes the "empty sidecar
    → no quantitative claims" rule so the design-without-estimates
    pathway is explicit.
  • Test-mock fix: MockLLMBackend._detect_specialist now matches
    on the canonical You are the <Name> specialist role line in the
    system prompt rather than searching for any specialist name
    substring. The old heuristic silently misrouted calls whenever a
    skill referenced another specialist by name (e.g. the new
    writing/cite-numbers-by-source mentions "econometrics
    specialist" → paper_drafter calls were routed to the econometrics
    output → paper_draft.tex was never produced). Now matches one
    occurrence per prompt with no skill-content interference.
  • Machine-readable JSON sidecar contract for verify_numbers.
    Pre-v0.5 every specialist was told to write EXACTLY ONE file, so
    even when a skill described a JSON sidecar (e.g. data/figure-spec),
    the system prompt overrode it and the JSON never appeared. The
    2026-05-20 live runs confirmed this empirically: both papers wrote
    number_verification.json with skipped_reason="no source JSON files found" — the gate was effectively a no-op. v0.5 adds a
    SPECIALIST_SIDECAR_ARTIFACTS registry, a sidecar_artifacts field
    on WorkOrder (auto-populated by _inject_context), and a
    multi-file "Required Output" prompt block that lists every required
    file with its role + JSON validity rules. data_analyst now emits
    summary_statistics.json and figure_spec.json;
    econometrics_specialist now emits estimation_results.json
    (with optional robustness_results.json). Two new schema skill
    files (data/summary-statistics-schema,
    econometrics/estimation-results-schema) teach the JSON shapes and
    the "write {} instead of omitting when data was unavailable"
    rule that distinguishes "honest empty" from "missing" for the gate.

v0.4.5

Choose a tag to compare

@github-actions github-actions released this 19 May 23:05
0937e65

Bug pack rolling up findings from the v0.4.4 live test (paper eea5379b)
that completed end-to-end on a fresh pip install e2er. The pipeline
itself works; these are correctness + clarity fixes around it.

Lane C — Data

  • Fix nested workspace path on --save-to (Lane C, replication
    correctness). The data_analyst subprocess runs with cwd at the paper's
    workspace dir; _resolve_workspace then resolved the relative default
    workspace_root="workspaces" against THAT cwd, so the CSV landed at
    workspaces/<id>/workspaces/<id>/data/. The model worked around this
    by emitting a _candidate_csv_paths fallback in estimation.py — a
    prompt-engineered band-aid for a pipeline bug. Fix: _resolve_workspace
    now prefers $E2ER_WORKSPACE_ROOT (claude_code injects the absolute
    path) over the relative settings default.
  • Inject absolute workspace_root into the claude_code subprocess env
    (E2ER_WORKSPACE_ROOT) and use an absolute path as the subprocess cwd.
    Without both, the relative workspaces string can re-resolve at any
    nested call site.

Lane A — Pipeline

  • Accept pipeline_mode as an alias for mode in CreatePaperRequest.
    e2er run --mode single_pass reached the API as pipeline_mode,
    which Pydantic silently dropped → server fell back to the default
    "iterative" → first-run log line falsely reported the wrong mode.
    Also fix src/cli_run.py to send the canonical mode field.
  • Reword the first-run cap log line. "override=True" read like the
    server overrode the user's cap; it actually meant the user acknowledged
    the unproven (model, methodology, mode) tuple so the $1 floor was
    lifted to their requested cap. New format spells it out:
    cap=$20.00 (user_ack_unproven=True, first_run_floor=$1.00).

Cross-lane

  • Label CLI-backend costs as estimates. Anyone running on
    claude_code / codex_cli / gemini_cli sees Sonnet-rate synthetic
    dollars even though the Max plan absorbs the actual cost. Startup log
    now warns once when a flat-rate backend is selected; the /api/papers/<id>
    usage payload carries a cost_is_estimate flag so dashboards can
    render the number with the right hedge.
  • e2er migrate works on pip-installed wheels. Old code pointed at
    scripts/migrate.py which is excluded from the wheel. Moved to
    src/db/migrate.py (importable, ships in the wheel), reads SQL files
    via importlib.resources("sql") with a dev-checkout fallback.
  • Drop the stale _SCRIPTS_DIR PATH entry on pip installs. Guarded
    with .exists() so the resolved PATH doesn't carry a non-existent
    site-packages/scripts/ directory that confused which-style probes
    inside the claude_code sandbox.