Completed II

Completed, Part II

Part 2 of the Completed list of finished ROADMAP items.

The three tier domain specialized architecture: Planner delegates to WebSearcher, AcademicSearcher, PeerReviewer, Builder, and FindingsWriter, and the two Searcher roles in turn delegate to DocumentAnalyzer and DataAnalyzer. Five Planner-tier delegates in total.

The Planner now only plans and delegates, it cannot write any file (2026-07-14), a user driven design decision: "the planner should only plan and delegate... giving the planner the job of writing the findings will poison context." The Planner previously wrote findings.md itself, and a retry there grew its own conversation the same way Builder was invented to prevent for the report, confirmed live the same day when a benchmark hit 4 consecutive retries and exhausted its budget with nothing written. Fixed with a new FindingsWriter Planner-tier delegate, dispatched only by the Write/Review/Fix loop, built entirely from RunState's structured findings data with zero shared conversation history; the Planner's own write_workspace_file tool was removed entirely.

Live verified end to end: FindingsWriter and Builder both dispatched correctly via independent loops, and PeerReviewer caught real issues in both, confirming the layers were genuinely independent, not redundant, since a report can cite something findings.md itself correctly recorded as unsupported. A new gap surfaced by the same testing: when Builder repeatedly re committed an unfixed mistake, the Planner had no signal a Write/Review/Fix cycle had just run and kept delegating more research instead, wasting 25 minutes and 35 fetched URLs before the system's own safety net still produced an honest report. Fixed by having run_completion_check continue straight into the next iteration on a successful Builder/FindingsWriter dispatch instead of returning control to the Planner. A related budget blind spot: run_stream_chars only counted the Planner's own streamed text, so a completion check nudge appended outside that loop was invisible to the budget guard; fixed by adding the nudge's length to the running total.

A wave of structural reliability fixes shipped in this period: per-attempt quota top-up, artifact quarantine before nudging, structured _run_state.json logging, real URL-presence plus content-level grounding checks, history-scanning salvage for a narrated-but-never-written report, and upstream grounding checks on each specialist's summary before it reaches the Planner. HTML boilerplate stripping got fixed on the primary fetch path (previously only the fallback path stripped nav/footer/script). The DDGS search client moved from a shared singleton to a per-call instance to avoid concurrent search corruption. extract_structured_data gave DataAnalyzer and DocumentAnalyzer a real tool-level distinction, not just a prompt-driven one. Heavy search mode now searches deeper and auto-fetches more real results instead of fabricating query variant strings. A human-in-the-loop gate, an MCP tool loader, readable run folder names, and several TUI clipboard fixes (a clipboard-first copy with an OSC52 fallback, multi-line paste flattening, paste debounce) also shipped here.

Several Planner-role models (devstral:24b, hermes3:8b, qwen2.5-coder:14b-instruct, llama3-groq-tool-use:8b, mistral:7b-instruct-v0.3-q5_K_M) were tried and rejected against the real nested delegate_tasks schema; Devstral made zero real tool calls across a full run despite roughly double the parameter count, narrating perfectly formatted JSON instead, confirming bigger isn't automatically better on this schema. Superseded on 2026-07-11 when gpt-oss became the default.

A per-task fetch tracking race condition got fixed: a before/after length check on a shared, run-wide fetched-URL list raced under concurrent delegate_tasks dispatch, confirmed live when a 3-task run produced 9 cross-attributed findings; fixed with a per-task scoped contextvar, verified at 10-task concurrency with zero duplication. This enabled a delegation scope relevance check flagging a specialist's summary when nothing it fetched actually mentions the required entity. delegate_tasks also started rejecting unresolved placeholder tasks and same-batch cross-task dependencies before dispatching.

A large live test battery (15+ runs) around 2026-07-11/12 surfaced real fabrication and infrastructure gaps. A code audit found the eval harness's find_latest_session filtering on a stale folder prefix, silently scoring raw stdout instead of the real report for every eval run since a folder rename. A placeholder detector false positive was root caused to a live run where a well formed 12-task batch got rejected wholesale because one task's name used an ordinary numbered label; facing rejection, the model gave up and fabricated all of findings.md from memory. Fixed by checking only the instructions field. A headless/headed browser fetch fallback got added for pages that bot-wall a plain HTTP GET (Springer, ScienceDirect, MDPI each had a distinct bot-wall signature); confirmed live that a headed browser beats headless (MDPI's block fires at the network level before JS loads). A boilerplate-strip regex bug that deleted a real 221k character article body because its CSS class happened to contain "sidebar" got fixed with a size guard. ScienceDirect's Cloudflare Turnstile stayed unfixable and was deliberately not pursued further, the same reasoning as declining to add CAPTCHA solving.

Three structural fabrication gaps closed in one pass: a findings.md wholesale fabrication gate (zero cited URLs, or none matching a real fetch, quarantines and forces re-delegation); structural exclusion enforcement (skipping matching tasks individually rather than rejecting the whole batch, since wholesale rejection was what caused the placeholder incident's fabrication); and unresolved referent rejection (a task instruction leaning on a bare pronoun with no anchor gets rejected with guidance to restate the subject).

The Windows dual-boot migration shared the Ollama model store via an explicit path, fixed a cp1252 encoding error in headless mode, and pinned doc extras since markitdown[all] silently downgrades on Windows with newer Python.

A production batch on 2026-07-11, validated live during a 13-run benchmark day: a findings.md existence gate before the report is accepted, --resume-run, a TUI intake clarifier, a run time budget, --depth presets, a repeatable --seed-url, a regulation_id_check, and quarantined draft restore at final verdict (the real draft wins over meta narration, loudly labeled).

A completion check refactor replaced a ~250 line if/elif chain (which had shipped the same swallowed elif bug twice) with a data-driven check list, pinned by a verdict matrix in the test suite. A Windows workspace escape bug in _get_safe_path (joining a base path with a drive-qualified filename discards the base entirely) got fixed with a containment check.

A context budget endgame guard shipped, since local models silently truncate from the top on overflow, indistinguishable from model collapse; settings.context_budget_chars counts text/args/ results per stream and forces one wrap-up turn on overshoot, verified live producing an honest "budget exhausted" report instead of silent truncation. A grounding hardening batch the same evening added stub-fetch detection, a charset decoding fix, citation format enforcement, and a URL prefix boundary fix.

Repo governance and CI landed the same day (an external audit's one genuinely real finding: no license): MIT LICENSE, a CI workflow running install/lint/tests on every push, and dependency pins. The rest of that audit's "critical" findings were checked directly and found false or already solved.

Academic/literature review output mode (2026-07-12), triggered by a live query where DeepSeek produced a properly structured literature review while the local model collapsed on the same query. A new settings.report_style rewrites the report structure to Abstract/Introduction/thematic sections/synthesis/References, with author-year citations. A grounding function, parse_academic_references, maps author-year keys to their References entry's URL. A real bug caught during test coverage: a line with two citations only checked the first, since the regex used .search() instead of .finditer(). A more severe bug found in a follow-up audit before any live run: the citation detector required every token before the comma to start with an ASCII capital, silently failing to detect "et al.," ampersand, or accented surname citations, exactly the forms the feature's own prompt tells the model to use. Both fixed; a live validation run produced the correct shape with zero false positives.

Several smaller TUI/engine fixes landed the same period: a checkmark-on-error bug (ToolCallWidget always showed a green checkmark regardless of result text); a fuzzy filename fallback for read/grep tools after 16% of workspace read calls in one audit used a garbled filename reconstructed from memory, cascading into abort exceptions; expanded _run_state.json logging; and a /resume-run slash command finally added to the TUI after it had been CLI-only for a full session unnoticed, prompting the standing TUI/CLI parity rule. An answer mode (short, no headings, inline citations) also shipped, live validated through one quarantine-and-retry cycle to a clean pass. A TUI ProcessingWidget timer leak (never stopping when a final turn streams zero content) got fixed with unconditional cleanup once the stream is guaranteed exhausted.

A cross-model benchmark against hosted NVIDIA NIM models (DeepSeek V4 Pro, Nemotron Super 49B, gpt-oss-20b) found none beat the local gpt-oss:20b: DeepSeek crashed on an uncaught 429, Nemotron made zero real delegation calls and fabricated citations, and gpt-oss-20b was the only clean pass, though thin. This motivated two "specialized non-LLM component instead of another LLM call" tracks: NLI-based grounding verification (a cross-encoder entailment classifier catching a citation with the right source and shared terms but a wrong specific detail, verified 0.023 vs 0.997 on an irrelevant vs. relevant pair against the real checkpoint; switched to a ~200MB CPU-only wheel after the naive install pulled a 6GB CUDA build for no GPU use), and fetch-time metadata extraction (pulling title/author/date from the existing boilerplate-stripper parse, eliminating a redundant sub-agent dispatch pattern that had fired 13 times identically in one day's logs).

An uncaught crash on malformed tool call retry exhaustion got fixed: a huge write argument truncated mid-JSON hit a bare raise on its third occurrence, killing a run after 18 sources fetched and 5 report attempts already on disk; the CLI now degrades to the standard budget-exhaustion verdict path instead of crashing, and the TUI gained the same retry-then-degrade logic it previously had none of. A validation error visibility gap (a generic "Argument parsing failed" was the most common error signature of the day, its real cause silently stripped because the framework's detailed error config was never enabled) got fixed, also helping the model self-correct on retry.

The Builder sub agent and Build/Review/Fix loop (2026-07-13)

Direct fix for context growth risk: the Planner's own conversation only ever grows (no compaction), so every completion check retry historically meant re-showing the model its own rejected drafts, a documented context poisoning pattern. The fix reuses delegate_tasks's existing fresh-context mechanism instead of inventing a new one. A new Builder role writes and rewrites final_report.md; completion check problems get classified as Builder-fixable (rewrite from existing findings.md) or Planner-escalated (genuinely needs more research). For Builder-fixable problems, the system dispatches Builder then a fresh PeerReviewer directly, with a required "REVIEW: CLEAN"/"REVIEW: ISSUES FOUND:" opening line, none of it touching the Planner's own conversation state.

Two live validation runs confirmed this end to end, including a hard three-way-AND benchmark where the loop fired correctly 3 times on real unsupported problems. A new finding: on later attempts, Builder itself hit the same narrate-instead-of-write failure the Planner used to be prone to, because Builder shares the run's single write quota pool with the Planner and prior Builder dispatches; the pre-existing quarantine restore fallback caught this correctly, restoring the best surviving draft with a loud unresolved-check banner rather than a silent failure. The mechanism works as designed but surfaced a real quota-sharing constraint under heavy retry load.

Six-phase grounding upgrade plan (2026-07-13 through 2026-07-14)

Phase 1, claim-level grounding: the prior check compared a whole line's terms against every source cited anywhere on that line, letting a shared generic term between two distinct claims mark both "supported" even when one citation didn't actually back its claim. Fixed with decompose_claim_segments, splitting a line into atomic segments at citation boundaries (mechanical regex, no NLP dependency), checking each segment only against its own bound citation.

check_excluded_topic: closed a gap where delegate_tasks already skipped an excluded task but did nothing to stop that topic appearing in the report anyway via a sibling task's tangential findings; the check reuses the same exclusion parser, scoped to report headings, not whole-document substring matching.

Phase 2, cross-source contradiction detection (FEVER style), depends on Phase 1's segmentation. find_cross_source_contradictions checks whether an uncited fetched source reports a numerically different figure for the same subject the report's claim covers, catching the report silently picking a side of a real disagreement. A pre-ship bug (pairing every subject with every number on a line regardless of kind) got caught and fixed in development. A second bug found live during Phase 6's TUI smoke test: bibliographic-only lines (image captions, reference entries) got treated as claim subjects, firing a structurally unfixable, non-converging phantom contradiction after every Builder rewrite, caught only because the user pushed back rather than assuming a stream bug. Fixed by treating a line as bibliographic when fewer than 8 real letters remain after stripping markdown links and leading markers; a re-run converged in 1 cycle and ~5 minutes versus 5+ non-converging cycles before.

Phase 3, xQuAD style search diversity reranking: near-duplicate results commonly dominate DDGS's own relevance ranking. _diversity_rerank greedily reorders by marginal new aspect-term coverage (the top result always stays first), pure reranking, no LLM call, no new dependency.

Phase 4, a topical relevance cross-encoder reranker: a third grounding layer asking whether the cited source is actually about the same subject, not just lexically overlapping and non-contradicting, catching acronym collision cases neither upstream layer catches. Verified against the real checkpoint (0.023 irrelevant vs. 0.997 relevant, either side of the 0.1 threshold) before the first live run. A bug caught during this pass: the new check's config gate was missing from the test suite's mocking guards, so the first full suite run silently loaded the real unmocked reranker model.

Phase 5, coverage accounting (ResearchMap): distinct from every other check, this asks whether the Planner's own delegated plan actually paid off, since a report can be perfectly grounded yet thin because most delegated angles came back empty and got silently dropped. Built entirely from already reliable structural data (RunState.coverage()), not a new Planner-authored schema, after explicitly ruling out the free-text todos file as a source of truth. The new check_thin_coverage isn't Builder/FindingsWriter fixable, since fixing it needs new delegation, so it falls through to the Planner path by design.

Live verification found 2 more real bugs. First, an earlier sub-agent timeout fix existed only in the config template, never backfilled into a real user config, silently disabled the whole time, meaning every earlier "live verified" confirmation of that fix had only been true because the key was temporarily test-added and reverted, prompting the standing rule that new settings keys must be grepped in the live config, not just the template. Second, a corrective Fix dispatch had no evidence base of its own, since it's a fresh sub-agent with zero memory of the first dispatch; fatal for FindingsWriter, whose source material only ever existed as a prompt string, confirmed live when a corrective dispatch burned its turn hunting for guessed nonexistent filenames. Fixed by re-appending the original write instructions. Third, check_thin_coverage false-positived on the project's own Builder/FindingsWriter/PeerReviewer dispatches, structurally indistinguishable by depth alone from a real top-level task; fixed by naming those three roles explicitly and skipping them.

Phase 6 (item B4): unify run_cli/run_agent's stream iteration and retry logic. The two genuinely duplicated pieces (a deadline-racing async generator, and the malformed tool call retry decision, previously copy-pasted and once missing entirely from the TUI) got extracted into shared helpers, closing out the six-phase plan.

Later fixes (2026-07-14 through 2026-07-21)

A claim_grounding_problem false positive on citation-only sub-bullets got fixed: this project's own Builder output puts a claim on one line and its citation on a separate "Source:" sub-bullet, and that bare sub-bullet was being processed as its own claim segment, failing because the writer's own paraphrase in the link text doesn't appear verbatim in the source. A helper built earlier the same day for the contradiction check just hadn't been applied here yet; live re-verification showed zero false positives, converging in under 500 seconds versus 1000+ before.

Immediate narration salvage inside the writer dispatch loop (2026-07-18): the existing salvage function only ran as a last resort for a missing report, not a missing findings file, which was the case burning a candidate model's full retry budget. Now checked immediately after every write dispatch: 200+ characters of real text with no artifact gets persisted right away as an auto-recovered draft. A live re-test against the motivating case found a more precise root cause: that model's dispatches returned genuinely empty responses, not narrated content, so salvage correctly declined to act rather than fabricate from nothing, and a full re-run confirmed a second, independent instance of the runaway-generation failure class already fixed elsewhere.

Shared quota pool starvation (2026-07-18): ring-fences a task's remaining quota once it's shown real fetch activity, granting one small one-time top-up instead of hard-blocking, bounded to once per tool per run, directly targeting a documented case where a task had fetched 2 real sources then hit a "Quota reached" wall before ever synthesizing them.

A Brave Search MCP country parameter that rejected real countries, including Colombia, got fixed: the server's enum turned out to be a fixed 37-code list missing Colombia entirely; a wrapper strips an out-of-enum value and falls back to an unscoped search.

The completion check remediation loop was found live (2026-07-20) to exhaust read_workspace_file's quota before the final Builder pass could read what it needed, since each remediation cycle burns the same shared pool; fixed with a quota top-up helper mirroring the existing writer-side one.

Two TUI quality-of-life fixes: AgentMessageWidget click-to-copy on the agent's own answers, and right-click paste using system clipboard tools with no OSC52 fallback (that escape sequence is write-only). Live verified round trip; surfaced that clipboard tools weren't actually installed on the dev machine, meaning both features had been silently non-functional, worth checking on any fresh setup.