References

References

Full bibliography backing DeepDelve's architecture decisions. Was README.md's "References" section.

Why local models are hard for this task, briefly: DeepDelve's own model bake-off (10 candidates, 9 disqualified, gpt-oss:20b the only full pass) is not an idiosyncratic gap. A published capacity- floor study found qwen2.5:14b as the "minimum viable production" threshold for reliable tool invocation, with sub-8B models failing at 40-85%+ rates on a narrower, more controlled task than DeepDelve's own open-ended research (Huang, Malwe, Wang, arXiv:2601.16280). Independently, two studies found small/mid open-weight models specifically fail at structured tool-call serialization (schema-valid output, wrong content) in a way a 6,000-sample fine-tuning run could not fix, because it happens downstream of anything fine-tuning touches (Li, Zhang, Lv, arXiv:2606.25605; Ray, arXiv:2605.26128). MAST's 14-mode failure taxonomy (Cemri et al., arXiv:2503.13657, NeurIPS 2025) maps closely onto DeepDelve's own documented bug catalog (see ROADMAP.md's "narrate instead of write," over-research, and exclusion-enforcement entries) — evidence these are known, published agent-failure patterns, not DeepDelve-specific quirks. Full literature review, corrections, and still-open leads in RESEARCH.md.

• Huang, Malwe, Wang. When Agents Fail to Act: A Diagnostic Framework for Tool Invocation Reliability in Multi-Agent LLM Systems. arXiv:2601.16280. Capacity-floor evidence for the local-model-selection constraint above (14B as the tested "minimum viable production" threshold for tool invocation), motivating why gpt-oss:20b rather than a smaller default.
• Li, Zhang, Lv. Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraints. arXiv:2606.25605. Confirms DeepDelve's own code never combines a JSON-schema response_format with tool availability in the same call (the exact condition that causes 0% tool-invocation-rate on every open-weight model tested, including GPT-OSS-20B); its proven Two-Pass Execution mitigation independently matches DeepDelve's own Planner→Builder/FindingsWriter split.
• Ray. The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models. arXiv:2605.26128. Originating paper for the constraint-tax concept above; its "reason free, constrain late" finding independently confirms the same design choice as the Two-Pass Execution paper.
• Cemri, Pan, Yang, et al. Why Do Multi-Agent LLM Systems Fail? (MAST). arXiv:2503.13657, NeurIPS 2025 Track on Datasets and Benchmarks. 14-mode failure taxonomy mapping closely onto DeepDelve's own documented bug catalog (see ROADMAP.md).
• Cemri, Cojocaru, Pan, et al. Fantastic Adaptive Taxonomies and How to Use Them (ATLAS/AdaMAST). ICML 2026 Workshop on Failure Modes in Agentic AI. multi-agent-systems-failure-taxonomy/ATLAS. Induces a domain-specific failure taxonomy directly from a target system's own execution traces rather than MAST's fixed vocabulary; a ROADMAP candidate for DeepDelve's own _run_state.json history.
• Liu, Lin, Hewitt, et al. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172, TACL 2024. A candidate mechanism (distinct from context-size truncation) for DeepDelve's own recurring "real content silently vanishes during final synthesis" pattern.
• Li, Zhang, Zhang, Chaturvedi. Coverage-based Fairness in Multi-document Summarization. arXiv:2412.08795, NAACL 2025. Formalizes per-cluster/per-document representation checks ("Equal Coverage"/"Coverage Parity") as the standard way to catch a summarizer that silently favors one source-group over another; validates the per-task binary-coverage design behind check_report_underuses_evidence (Builder-stage report-writing check, src/engine/completion.py) as a legitimate, cheap proxy for their more expensive NLI-based metric — closes a live-confirmed gap where a report can clear a flat citation-ratio threshold while dropping an entire query facet, because the surviving facet simply had more raw sources.
• Huang, Ma, Tahir, Zhang, Xiao, Xiao. Regression Accumulation in Multi-Turn LLM Programming Conversations. arXiv:2607.01855. Finds 55.7% of multi-turn LLM correction regressions are a later fix breaking an earlier, already-satisfied requirement through incompatibility, not forgetting; their validated "Verification Gate" mitigation (full re-verification every turn, every failing constraint made visible) directly informed the fix for a real check-priority-shadowing bug (utils/grounding.py::cheap_grounding_problems, src/engine/completion.py::_with_other_grounding_addendum) where real_grounding_problem's own first-hit priority chain could silently shadow a second, simultaneously-true grounding problem for an entire run's retry budget.
• Kamoi, Zhang, Zhang, Han, Zhang. When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs. arXiv:2406.01297, TACL. Corrected 2026-08-17: previously (mis)cited here as the source of the "~64.5%... across 14 open models" self-correction-blind-spot figure — read in full and confirmed that number does not appear anywhere in this paper; it belongs solely to Tsui's Self-Correction Bench below, which this survey does not cite (checked its full reference list directly). This paper is instead a methodology critique: it finds many published self-correction studies don't clearly define their research question or design controlled experiments, and provides a checklist (§8) for what a valid self-correction study needs (a fair baseline, a genuine "own mistake" condition, appropriate metrics) — real prior art for treating any single self-correction number cautiously, including the ones cited in this project's own README, but not itself an empirical source for the blind-spot figure.
• Tsui. Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models. arXiv:2507.02778, COLM 2026. Read in full (not just the WebSearch summary a first pass this session mistakenly relied on) — the ORIGINAL and, per the correction above, the ONLY primary source found for the ~64.5% figure (testing 14 open-source non-reasoning models). Appendix C's per-model breakdown (read 2026-08-17) shows the average hides a huge spread relevant to this project's own model choices: Qwen3-14B/Qwen3-32B/Qwen3-30B-A3B, tested in non-thinking mode (the same regime config_template.yaml runs in), score 0.004-0.108 — near-total blind spot, far worse than the 64.5% average implies. Isolates the blind spot as a genuine activation failure (models correct the identical error when it's presented externally, ruling out "doesn't know better") and finds it is NOT solved by closed-source frontier models either (Claude 3.5 Haiku 52.5%, Claude Sonnet 4 41.4% blind spot) — relevant confirmation that a frontier model failing to self-correct its own prior output is a known, measured phenomenon, not an anomaly. Its most actionable finding for this project: appending the single word "Wait" after a model's own erroneous output, with no fine-tuning at all, reduces the blind spot by 89.3% and nearly matches that same model's full reasoning-mode accuracy — a candidate addition to _dispatch_writer_review_fix's retry-instructions prepend (src/engine/completion.py, commit 4dc19bc) not yet implemented.
• Song Tae-Eun. Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessions. arXiv:2603.12123. Single-author, non-academic preprint (Daejeon Jungang Cheonggua Co.), no confirmed peer-review venue — read in full 2026-08-17. Session-separation between production and review as a mitigation for the self-correction blind spot above; motivated routing the two checks' fix directive through edit_workspace_file (a narrowly-scoped addition) explicitly rather than an implicit full write_workspace_file regeneration from the model's own prior draft. Its own controlled ablation (4 review conditions, F1 on 150 injected errors) is a direct, positive validation of PEER_REVIEWER_INSTRUCTIONS's existing shape, not just a general motivation: the paper's best condition, Cross-Context Review (artifact only, no generation prompt, F1 28.6%), beat its "Subagent Review" condition (fresh session but WITH the original generation prompt, F1 23.8%, p=0.004) — knowing the original task actively hurt review quality. DeepDelve's own PeerReviewer dispatch (_dispatch_writer_review_fix, src/engine/completion.py) already sends only "Review '{artifact}' for accuracy and coherence..." — no query, no production context — matching the paper's winning artifact-only condition, not the weaker prompt-aware one, independently arrived at rather than copied from this paper.
• Xu, Zhu, Wang, Wang, Athiwaratkun, Wang, Zou, Zhang. When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework. arXiv:2506.16411, ICLR 2026. Read in full 2026-08-17. Names three distinct long-context failure modes (task noise/cross-chunk dependence, model noise/context-length confusion, and aggregator noise — individual facts correct but a merge/synthesis step drops whole clusters); the observed multi-facet-abandonment pattern above is a clean match for aggregator noise specifically, scoping which fix family (hierarchical/divide-and-conquer decomposition) is the natural next escalation if the edit_workspace_file directive fix above doesn't move the needle (see ROADMAP.md Pending). **Its formal "Regime 2: the Silo Effect" (task noise dominates — a decomposition schema loses global reasoning "regardless of model quality," Proposition 3.1/§3.6) is a THIRD independent paper this project has now read that converges on the same structural conclusion as RESEARCH.md §18f's MARL paper and the Illusion-of-Multi-Agent-Advantage paper: DeepDelve's per-facet research dispatch is the well-behaved case (task decomposes cleanly), and the consolidation/aggregation stage (FindingsWriter, Builder) is structurally the one place error concentrates, independent of which theoretical lens (PAC-learning sample complexity, causal MAS ablation, or this paper's fidelity-loss decomposition) is used to derive it.
• Khanal, Tao, Zhou. Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents. arXiv:2603.29231, Northern Kentucky University, 2026-04-01. Formalizes the pass@1-vs-reliability gap this project's own fixes were validated against with n=1 live-run anecdotes until 2026-08-17: pass@k (succeeds at least once across k trials) vs. pass^k (succeeds EVERY time) are different questions, and τ-bench's own measurement (GPT-4o: 61% pass@1 but only 25% pass@8 on retail agent tasks) shows how far apart they can be. One of the paper's three benchmark domains is literally "Agentic Web Research" (multi-step search/fetch/synthesis) — directly on-domain, not just analogous — and its own methodology uses k=3 repeats, the same floor eval/evaluate.py's new compute_reliability_summary (pass@k/pass^k reporting over the existing --runs flag) is built around.
• Mustahsan, Lim, Anand, Jain, McCann. Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation. arXiv:2512.06710, AAAI 2026. Read in full including all appendices. Proposes ICC (variance decomposed into task-difficulty vs. agent-inconsistency components) as a more rigorous reliability metric than a bare pass@k/pass^k rate. Its most load-bearing finding for this project: on the benchmark task shape closest to DeepDelve's own (GAIA Level 3, "hard open-ended reasoning"), even GPT-5 only reaches ICC=0.629 ("moderate," not "good" reliability) and GPT-4o's ICC=0.304 means 70% of observed variance is trial-to-trial randomness, not task difficulty — "single-run results are essentially unreliable." The paper's own convergence analysis found stable ICC estimates need n≈32 trials for this task shape (vs. n≈8-16 for simpler structured tasks), and its own closest analog to a DeepDelve-shaped agent (o4-mini deep research) only managed n=8 trials for cost reasons, with its own Limitations section stating this isn't enough for generalizable conclusions. Directly informs eval/evaluate.py's new --runs/reliability-summary feature: k=3 is a practical floor chosen for this project's own per-run cost (20-70 minutes on local hardware), not a claim that k=3 achieves statistically converged reliability for a task this hard.
• Jwalapuram, Lin, Li, Jiao, Wang, Ming, Ke, Qin, Carenini, Joty. The Illusion of Multi-Agent Advantage. arXiv:2606.13003, Salesforce Research/HKUST/UBC/NTU, 2026. Read in full. Its critique — automated multi-agent frameworks that dynamically decide their OWN coordination structure per query (DyLAN, ADAS, AFlow, etc.) mostly collapse into simple single-agent self-consistency at 10x the cost — does NOT indict DeepDelve's architecture, which uses a fixed, hand-designed pipeline, not per-query dynamic routing. The paper's own contrast case, "Expert-MAS" (deterministic, explicit role decomposition, Python-orchestrated control flow — structurally the closest match to DeepDelve's own shape), is the one architecture in the paper that wins decisively (GPT-5: 57.0% → 96.5% over single-agent). Its real contribution for this project is the audit methodology, not a verdict against decomposition: complexity is "architectural bloat" specifically when added without verified causal contribution (their own causal ablations, e.g. reducing decision-making from consensus to single-agent authority raising success +9.4%) — a standard this project's own completion-check mechanisms (src/engine/completion.py) have never been held to, since each was validated only by "did today's specific symptom stop recurring," never by a controlled with/without ablation.
• Su, Wu. When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic Systems. arXiv:2602.08272, University of Hong Kong, 2026-02-10. Read in full (main body, Sections 1-5; theorem proofs in the appendices not read line-by-line). A PAC-learning theoretical analysis of multi-agent RL training, empirically validated on real GSM8K data, not an inference-time deployment study — but its central mechanism generalizes: decomposing into genuinely INDEPENDENT subtasks scales cost down (complexity dominated by the single hardest subtask), while DEPENDENT subtasks introduce error propagation with a quadratic worst-case penalty in agent count, confirmed on GSM8K ("SARL consistently outperforms MARL due to error propagation" in the dependent-subtask setting). Maps precisely onto where DeepDelve's own bugs concentrate: the Planner's per-facet research dispatch is the genuinely-independent case this paper's theory favors, but FindingsWriter/Builder's consolidation stage is the dependent, error-propagating junction — and every bug in today's 7-fix arc (evidence-crowding, marker leaks, the self-correction loop) occurred there, not during independent per-facet research.
• Jiang, Yang, Cui, et al. Deep Research in Physical Sciences: A Multi-Agent Framework and Comprehensive Benchmark (DelveAgent / PhySciBench). arXiv:2606.18648. Primary architecture source (Adaptive Planning Loop, Dual-Granularity Memory, Hierarchical Reflection).
• Huang, Chen, Zhang, et al. Deep Research Agents: A Systematic Examination and Roadmap. arXiv:2506.18096
• Xu, Peng. A Comprehensive Survey of Deep Research: Systems, Methodologies, and Applications. arXiv:2506.12594
• Xi, Lin, Xiao, et al. A Survey of LLM-based Deep Search Agents. arXiv:2508.05668
• Del Rosario, Krawiecka, Schroeder de Witt. Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations. arXiv:2509.08646 (SAP / ACM / University of Oxford; correction 2026-08-17 — previously cited generically as anonymous "Plan-and-Execute agentic architectures survey work"; read in full, it's actually a practitioner security-hardening guide, not a survey, though its P-t-E architectural description is accurate and matches what's cited it for). The established pattern the Builder + Build to Review to Fix loop maps onto: decouple planning (decompose, can use a cheaper/more strategic pass) from execution (carries out + retries mechanically), re-planning only on genuine failure rather than every step — its own §7.1 describes this exact re-planner-node pattern, fed the original objective, the original plan, and prior step outcomes, matching DeepDelve's own adaptive-planning-loop shape. Directly informed by the observation that DeepDelve's Planner conversation grows unboundedly across a run with no compaction (a documented "context poisoning" risk) and that the existing delegate_tasks mechanism already gives every dispatched sub-agent a genuinely fresh context. The fix is routing report-writing retries through that mechanism instead of the Planner's own conversation.
• Min, Krishna, Lyu, et al. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. arXiv:2305.14251, EMNLP 2023. Read in full 2026-08-17. Decompose-then-verify pattern (break a generation into atomic checkable facts, score each independently) that the grounding layer's line-scoped checks already followed in spirit; cited as prior art motivating the NLI entailment check (nli_unsupported_problem) below. Its own stated limitation is directly relevant to this project's check design: FActScore measures factual PRECISION only — it "does not penalize a model that abstains from responding too frequently or generates fewer facts," an inherent precision/recall trade-off the paper's own Limitations section names explicitly. DeepDelve's completion-check pipeline already pairs a precision-side check (nli_unsupported_problem/grounding) with separate recall-side checks (check_thin_coverage, check_report_underuses_evidence) that catch a report gaming precision by omitting risky claims — closing exactly the gap FActScore's own authors flag as unaddressed by precision-only metrics.
• Goswami, Kurra. HALT-RAG: A Task-Adaptable Framework for Hallucination Detection with Calibrated NLI Ensembles and Abstention. arXiv:2509.07475 (author names corrected 2026-08-17 from a placeholder "Anonymous" credit — no institutional affiliation stated in the paper itself, no confirmed peer-review venue, read in full). Source of the "layer NLI entailment on top of lexical/term-overlap checks, don't replace them" design choice for nli_unsupported_problem: HALT-RAG's own ablation (Table 2) quantifies this directly — removing the entailment signal costs 4.5 F1 points, removing the contradiction signal costs 2.1, removing lexical features costs 1.3 — confirming NLI and lexical signals are complementary, not redundant, which is why the entailment check only runs on claim lines that already passed the existing term-overlap check rather than gating independently. Honestly-scoped limitation, relevant to DeepDelve's own use: evaluated only on HaluEval's clean, relevant source documents — the paper's own Discussion names robustness to noisy/irrelevant RETRIEVED context (DeepDelve's actual situation — real fetched web pages, not a curated benchmark) as "an important and unevaluated area for future work."
• Rasheed, Banerjee, Mukherjee, Hazra. From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agents. arXiv:2602.13855 (title corrected 2026-08-17 — previously cited under a fabricated title, "Claim-Evidence Provenance in Grounded Generation"; the author and substance were right, the title wasn't, caught on a full read). Perspective/position paper, no confirmed peer-review venue — proposes the "Auditable Autonomous Research" (AAR) standard, formalizing claim nodes with typed edges to source nodes so a claim's provenance can be checked independently of its neighbors on the same line. That claim-node framing, together with FActScore above, motivated decompose_claim_segments's atomic-claim decomposition: splitting a line into per-citation segments so each claim is checked only against its OWN bound citation, closing a gap where two distinct claims sharing one line could pass on a shared generic term.
• Thorne, Vlachos, Christodoulopoulos, Mittal. FEVER: a large-scale dataset for Fact Extraction and VERification. NAACL 2018, fever.ai. FEVER-style claim/evidence framing behind find_cross_source_contradictions: cross-references a report's claims against ALL of its own fetched sources (not just the one cited), catching a report that silently picks one side of a real disagreement between two of its own sources without flagging the conflict.
• Santos, Peng, Macdonald, Ounis. Explicit Search Result Diversification through Sub-Queries. ECIR 2010. Source of the greedy marginal-aspect-coverage reranking pattern behind tools/web.py::_diversity_rerank: after the engine's own top result, each subsequent pick is whichever remaining result adds the most new aspect terms, rather than raw engine rank.
• Anthropic. How we built our multi-agent research system. anthropic.com/engineering/multi-agent-research-system. Independent confirmation that a multi-agent research architecture (lead agent delegating to parallel subagents with their own fresh context) beat a single agent by 90.2% on Anthropic's own internal research eval, validating DeepDelve's existing Planner-Searchers-Analyzers shape and directly informing the Builder + Build to Review to Fix loop's decision to route report-writing retries through a fresh-context sub-agent dispatch rather than the Planner's own growing conversation.
• kyuz0/deep-research-agent: base architecture this was forked from.
• CYC2002tommy/Deep-Research-Agent: source of the "full-text reading is mandatory" and content-level claim-grounding ideas.
• nashsu/llm_wiki: source of the findings.md to final_report.md two-pass pattern and the structured run-state idea.
• dzhng/deep-research: source of the schema-forced FOLLOW-UP DIRECTIONS idea (Searchers must return next-round research leads for the Planner) and the information-density rule for findings (entities, exact metrics, dates). Its structural iterative-deepening loop (learnings-conditioned query generation with geometric narrowing) is a ROADMAP candidate.
• Alibaba-NLP/DeepResearch (Tongyi DeepResearch): source of the heavy search mode (test-time scaling, credited in tools/web.py) and the DocumentAnalyzer verbatim-evidence rule (its visit-tool extractor separates verbatim evidence from summary). Its context-budget endgame and the Tongyi-DeepResearch-30B-A3B model itself are ROADMAP candidates.
• imbad0202/academic-research-skills: reviewed for its literature-review paper structure and Anti-Leakage Protocol ("Knowledge Isolation Directive": prefer session materials over parametric memory, flag [MATERIAL GAP] instead of fabricating). Both are ROADMAP candidates for the academic output-mode work. Its bibliographic-API citation verification (Semantic Scholar/OpenAlex/Crossref/arXiv) was reviewed but not adopted, see ROADMAP "Rejected".
• SkyworkAI/DeepResearchAgent: reviewed (self-evolution agent runtime: RSPL/SEPL protocol layers, RL-based prompt/solution optimizers, versioned tracing). Not adopted, see ROADMAP "Rejected".
• froggeric/Qwen-Fixed-Chat-Templates: community-patched Qwen 3.5/3.6-family chat template fixing an "empty <think> block poisons the model into tool-looping" defect in the stock template, plus KV-cache-invalidating whitespace churn across turns. Applied directly (2026-07-28) to an Ornith-1.0-9B GGUF's tokenizer.chat_template metadata via gguf_new_metadata.py --chat-template-file (no tensor rewrite) during a live bake-off — measurably reduced task-splitting/looping regressions. See RESEARCH.md §13/§14 for the full before/after evidence.
• deepreinforce-ai/Ornith-1 GitHub issues #4 and #16: the model author's own issue tracker, read directly as primary corroboration that the tool-looping pattern hit during this project's own Ornith-1.0-9B bake-off reproduces independently across unrelated harnesses (VSCode/Continue, Claude Code) — confirms it as a real, model-family-wide trait rather than a DeepDelve-specific bug, and names the same root cause (froggeric's template fix, HF discussion #42 on the 35B GGUF) this project independently arrived at the same day.
• microsoft/agent-framework's own packages/devui reference app (frontend/src/components/features/workflow/execution-timeline.tsx): source of src/static/index.html's per-agent-run timeline (one collapsible entry per sub-agent dispatch with a running/done status icon, replacing an earlier flat interleaved log) — checked directly against the actual framework DeepDelve is built on, not improvised, after a live UX complaint about the flat log reading as noise once more than one or two sub-agents had run.
• assafelovic/gpt-researcher's Next.js frontend: two ideas checked directly and adapted, not copied verbatim — components/research/ResearchContent.tsx's disabled={loading} pattern (every input locks during a run, not just the submit button) confirmed a live UX bug in DeepDelve's own web UI; components/ResearchBlocks/elements/SourceCard.tsx is the source of the report viewer's source-cards grid, with its external Google-favicon-service call deliberately dropped (a per-citation outbound request conflicts with this being an otherwise fully local-first tool) in favor of a plain domain-initial avatar.