Codebase Review

DeepDelve Codebase Review & Critique

A strict review of DeepDelve's architecture, state management, grounding systems, and specific bug patterns.

Executive summary

Main strengths: per attempt quota top up (fixes retry starvation), auto fetch fusion (search and fetch combined so the LLM can't shortcut to snippets), artifact quarantine (renames flawed drafts aside so the model can't re condition on its own mistakes), and deterministic URL grounding (tracking what was actually fetched, not trusting the model's narration). A strict audit still turned up several latent design flaws, thread safety violations, and broken fallbacks.

The salvage fallback defect

_salvage_narrated_report is meant to catch a report the model narrates as chat text instead of writing via write_workspace_file. It's flawed because the text accumulator resets every loop iteration in CLI mode, and state["current_msg"] clears to None on any tool call in TUI mode:

while has_requests:
    has_requests = False
    user_input_requests = []
    turn_text = ""  # <-- RESET HERE

In a real traced incident, the model narrated a full, accurate report on attempt 3 but never called the write tool, then on the final attempt produced nothing at all. Salvage only ever checks the immediate last turn's text, so the good narration from attempt 3 was thrown away in favor of attempt 4's empty string. In TUI mode this is structurally worse: any tool call at all, even think_tool, wipes current_msg before the completion check ever runs, so salvage almost never fires there.

Fix, scan backward through the session log for the last substantial Planner text block instead of trusting only the immediate last turn:

def _find_last_planner_text(events: list) -> str:
    for event in reversed(events):
        if (event.get("source") == "Agent" and event.get("type") == "text"
                and event.get("depth", 0) == 0):
            text = event.get("data", {}).get("text", "")
            text_clean = re.sub(r'System \(\d/\d\):.*', '', text).strip()
            text_clean = re.sub(r'System \(final\):.*', '', text_clean).strip()
            if len(text_clean) >= 200:
                return text_clean
    return ""

Other flaws found

Specialist separation is purely prompt based. WebSearcher/AcademicSearcher and DocumentAnalyzer/DataAnalyzer share identical tool lists in app.py, nothing but the prompt enforces the split, so a misrouted task or model drift has no system level guardrail to catch it. Fix: give each specialist its own distinct tools (an academic API tool, structured extraction tools).

Thread safety violations in tui.py. Session state (_session_events, _current_call_by_source, etc.) is module level globals, fine for single user CLI but a real bug in web mode, where multiple users would share and corrupt the same lists. Fix: move into App instance attributes or task isolated contextvars.

RunState.add_finding() is dead code. Defined but never called anywhere, so the structured findings cache stays empty and the "reuse extracted facts across queries" goal is only half implemented. Fix: have delegate_tasks actually log each finding.

A stopword bug in _extract_salient_terms. A multi word proper noun starting with a stopword ("The Python Programming Language") gets discarded entirely instead of just stripping the leading stopword, causing false negatives in grounding checks. Fix:

words = phrase.split()
while words and words[0] in _PROPER_NOUN_STOPWORDS:
    words.pop(0)
if words:
    terms.add(" ".join(words))

DuckDuckGo client thread safety. A shared, lazily initialized DDGS() singleton gets called concurrently from asyncio.gather, and the library isn't documented as thread safe for that. Fix: instantiate DDGS() fresh inside each search call instead of reusing a global instance.

A live CLI test case

Query: "Who eliminated Colombia from the 2026 FIFA World Cup?" (Switzerland, round of 16, 0-0 then 4-3 on penalties, outside any local model's training data). The agent burned its first two attempts on memory refusal and planning without delegating, finally researched correctly on attempt 3 but forgot to write the report, then on attempt 4 wrote a fully correct, well synthesized report, but cited a FIFA match URL it never actually fetched, only a Sporting News page it had really fetched was real. The grounding check correctly caught the fabrication and refused to verify the report, exactly as designed. Net result: fact accuracy was 100%, but the agent fabricated a more authoritative sounding citation for content it already had a real source for, and had already burned its retry budget on the earlier delegation stalling.