Changelog Recent 2

Changelog: Recent, Part II (2026-07-26)

Part 2 of Recent in the Changelog, continuing the same very busy day, 2026-07-26.

v2 combined LoRA live benchmarked end to end, disqualified

This closed the "not yet done" caveat from the fine tuning entry in Part I. The deployment pipeline merged the adapter on CPU, converted it to a GGUF, tagged it in Ollama with a reused, proven tool call template, passed a direct tool call smoke test cleanly, then got pointed at by the live config and run against the standing sales forecasting and heuristic algorithms benchmark, the same query every prior model candidate had been run against.

The result: disqualified, final_report.md never written, retry budget exhausted. The task verification check fired twice against the same 3 tasks for fabricated or unverified citations, and the model's own first synthesis attempt narrated report content in prose instead of calling the write tool at all ("Since I cannot write or edit files directly, I will describe the content..."). After 2 failed grounding verification retries it re dispatched a duplicate, untracked task, tripping a hard stop before any report was produced. This directly answered an open "live citation grounding retest" question from the held out eval, still broken live, consistent with that eval never reaching ceiling.

Worth noting for the record: confirmed via a raw API test, on both this candidate and plain qwen3:4b, that the run executed under the already known, already accepted Qwen3 think mode passthrough Ollama bug, a pre existing, model family wide confound, not something specific to this LoRA. Named for the record, it doesn't change the verdict, since base qwen3:4b carries the identical confound and this candidate still fabricated citations and never produced a report.

Cleanup followed the project's disqualified candidate hygiene rule: the Ollama tag got removed, the merged and GGUF intermediate files got deleted, about 11.6GB reclaimed, and the LoRA adapter itself got kept as the training artifact. The config got restored to the gpt-oss baseline.

v2 combined LoRA re tested clean through the native Ollama backend, disqualified again

Done on 2026-08-19, this closed the "clean re test, not yet done" caveat the entry above had carried since 2026-07-28. The adapter got re merged and redeployed, and a direct curl against the live tag confirmed that think: true, not false, correctly isolates reasoning into its own field on the native endpoint, while think: false reproduces the identical known bug. One clean trial ran via the eval harness against the standing visa and rent ablation benchmark.

One real process mistake happened along the way, worth recording: the first attempt at this re test edited the live config file and launched the eval harness without explicitly passing a config path, but the harness never reads the live config at all, it always builds its own from the template unless a config is passed explicitly. That entire roughly 47 minute run silently re benchmarked the gpt-oss baseline again, not the candidate, a wasted run, caught only by checking the run's own materialized config after the fact. Corrected by building a dedicated config file and passing it explicitly, confirmed correct by re checking the new run's own materialized config before trusting the result.

The result: a score of 0.25, worse than the earlier confounded run's 0.5, finishing on its own well under the timeout ceiling. final_report.md was the deterministic salvage banner, the model narrated in chat instead of ever calling the write tool across its full writer retry budget. findings.md repeatedly failed grounding on the same fabricated URLs across consecutive rebuild attempts, the same citation fabrication shape as the original disqualification. This confirmed the verdict cleanly: the fine tune's targeted objectives are real, held out gains hold, but citation fabrication and writer dispatch convergence, dimensions the combined reward never targeted, are still broken at 4B scale even with reasoning cleanly isolated. Not a serving layer artifact this time, a real capability ceiling for this base size on these specific failure modes.

The same day, a deeper root cause got found for why retries weren't recovering, and it turned out not to be a model capability problem at all, it was a structural bug. The known Qwen3 think passthrough bug inflated every turn's token count 2 to 3 times, blowing through the context budget in just 2 of the 8 configured completion check attempts. The TUI's own context budget force final path then jumped straight to the terminal branch, where the starvation guard's own window happened to be active and swapped the real, still retriable problem, citation fabrication, out for a check whose own docstring explicitly promises it "will NOT block this run from finishing." The forced final path had no awareness of that contract, so a low severity hygiene nudge got reported as the run's terminal blocker instead of the real correctness problem, which never actually got its intended 8 attempt budget to resolve. This got fixed with a new parameter marking that one check as never allowed to become the final blocker once a cycle is already headed to the final branch regardless.

The consequence for the disqualified verdict above: it stands as recorded, but is now understood to be confounded, the model never got a real chance to resolve the citation fabrication problem before being cut off. Re benchmarking with this fix in place would be needed for a truly clean verdict, not done that session per explicit scope, fix the bug, don't re benchmark yet.

Ollama restored as the permanent serving backend, vLLM removed

This reverses an earlier decision to drop Ollama, after the same day's vLLM re test sweep disqualified or discarded 9 candidates in a row, several with genuinely serving layer shaped symptoms, a silent zombie crash, intermittent empty JSON tool calls, a garbled tool call marker. Research, primary sourced against vLLM's own PR history and blog posts, not just inferred from symptom shape, found that vLLM plus bitsandbytes on ROCm support is real but young specifically on consumer RDNA GPUs, this hardware ran bitsandbytes exactly at its minimum required ROCm version floor, while llama.cpp's HIP backend, what Ollama uses, turned up zero documented quantization correctness issues across the same research, with years more real world maturity on consumer AMD hardware.

Worth being clear about, deliberately not oversimplified: most of that day's actual disqualified verdicts weren't ROCm caused, the same non convergence and narrate instead of call patterns reproduced identically on Ollama hosted candidates too, including qwen3:8b on both backends. This reverted the serving backend to fix the flaky, crashy low level tail, not a claim that it changes any model capability verdict already reached. The two bugs that originally motivated dropping Ollama, the Qwen3 think mode passthrough and a nested parameters bug, are still real and unfixed, this was an explicit, informed tradeoff, accepting those two known, narrowly scoped bugs in exchange for llama.cpp/HIP's overall serving layer maturity edge on this specific hardware, not a claim that Ollama is now bug free. Cleanup deleted the vLLM venv and its specific HF cache checkpoints, while DeepDelve's own actual dependencies sharing the same cache location got deliberately preserved.

mistral-nemo:12b re tested with the Mistral fix, disqualified, same non convergence pattern

This closed out an original blocked verdict, which had already noted this exact fix as the remaining open re test opportunity. The real weight size got checked first, per a lesson just learned from another candidate that day: the HF repo lists two packagings of the same roughly 24.5GB bf16 weights, and at the observed 4 bit ratio this comfortably fits. With bitsandbytes 4 bit, the Mistral tool parser, and that day's chat template fix applied, the request reached the model cleanly this time, no 400 error.

The isolated tool call smoke test came back 6 of 8 clean across two batches, real structured arrays, no known bug class, with two distinct intermittent failure shapes, a garbled tool call marker prefix once and one full narration with no real call once, better isolated reliability than most other candidates that day, not perfect.

The full DeepDelve benchmark run showed real, genuine engagement with the task, 13 sources fetched, 0 of 7 web search failures, 8 findings recorded, clearly not a narrate only or zero engagement failure. But it ended with the retry budget exhausted on thin coverage, final_report.md never written, all 4 of 4 completion check attempts hitting the identical problem, the same non convergence signature already documented 3 times this project across different models and backends: once thin coverage fires, the Planner doesn't act on the corrective nudge and keeps repeating similar coverage instead of genuinely redelegating deeper.

The verdict: disqualified. The original blocked status is now resolved, the infrastructure issue is confirmed fixed, real research happened this time, but the underlying capability question comes back negative, joining the thin coverage non convergence family rather than escaping it. This isn't the same failure class as mistral:7b-instruct's own same family verdict, narrate instead of call, this candidate genuinely engages with the task, it just can't escape thin coverage retries once triggered.

devstral:24b discarded on hardware grounds without a pull attempt

The real weight size got checked first this time, per the lesson just learned from qwen2.5-coder:14b-instruct, don't spend time before confirming it fits. The HF repo lists 94.3GB of weight files, but that's misleading, it contains both a single file packaging and a separate sharded set totaling the same roughly 47.1GB, the same weights packaged twice, not two different models. Applying the empirically observed 4 bit compression ratio from qwen2.5-coder-14b earlier that same day, this model's bf16 weights would land around 17GB quantized, exceeding the entire 17.1GB card before any KV cache or activation overhead is even counted. Discarded on hardware grounds, no pull attempted, no GPU time spent.

qwen2.5-coder:14b-instruct, left inconclusive after a wrong parser and about 50 percent unreliable extraction

This candidate went through three distinct rounds before landing on an honest verdict, each correction caught by the user pushing back on an over confident claim, not found independently, worth recording as is.

Round one: the model launched with the hermes tool parser and crashed, initially misdiagnosed. The first launch failed cleanly needing more KV cache than was available. Bumped up, weight loading succeeded, then the process crashed silently with zero further log output, found in a zombie state with the parent server hung forever, no traceback anywhere. This was initially attributed to a system RAM issue, a warning in the log looked suggestive, but the user asked "are you sure that's the reason?", which prompted actually reading the loader's source, and that warning turned out to be about an optional prefetch for network filesystems, correctly skipped on this local mount, its absence being the safe path, not a risky one. That theory didn't hold, so the candidate got discarded on hardware margin grounds instead.

Round two: the user asked for a retry, and the crash didn't reproduce. A clean second launch with the same flags got past the exact point that crashed before. The crash had been transient or intermittent, not deterministic, so the round one verdict was wrong on its own terms, it does start and run, just tightly. The isolated smoke test against the now live server found 3 of 3 consistent failures, but a new, different shape than any other candidate that day: the model produced a genuinely well formed, correctly structured tool call, just wrapped in the wrong XML tags for the parser in use.

Round three: the wrong parser got confirmed via primary sources and the correct one found and installed. Web research plus direct inspection of the model's own chat template confirmed that this model, unlike the vanilla instruct variant, was never trained on the Hermes tool call convention, it uses different tags, and the serving engine had no native parser for this variant. A real community project existed specifically for this gap, actively maintained, reviewed directly before installing, no unsafe code found. Relaunched with the community parser. The smoke test with the correct parser came back 2 of 4 consistent successes, real structured array, and 2 of 4 returned empty arguments despite a normal completion token count, and the plugin's own documentation independently notes that longer system prompts degrade this model's JSON generation quality even under ideal conditions, at least directionally consistent with an unreliable under load pattern, though the empty extraction itself could be either the model's own fault or the third party parser's, not disambiguated and not chased further given the time already spent on this one candidate.

The verdict: inconclusive, not disqualified and not a clean pass. Real capability under the correct format was never cleanly established either way, roughly 50 percent extraction reliability in isolated testing is too unreliable to call a pass, but it's also not the kind of confident, repeatable failure other candidates that day showed. A full DeepDelve benchmark run was never attempted. Left as a genuine open item, not closed.

hermes3:8b, disqualified, narrates fake system messages instead of calling tools

Continuing the same day's vLLM re test track. This model passed its isolated tool call smoke test with the real delegate tasks shaped nested array schema cleanly, 3 of 3, real structured array, no known bug class. There's no thinking mode in this model's chat template at all, checked directly, so nothink is simply not applicable here.

The real DeepDelve benchmark run revealed a genuinely bizarre, distinctive failure: on its very first delegation attempt, the model narrated a fabricated system error as plain response text, "It looks like the prompt exceeded the maximum context length allowed by the model... I will try again with a shorter prompt," then repeated this fake retry narrative twice more, never once calling a real tool. This error got confirmed entirely invented, not a real server rejection, by grepping the serving engine's own log for the same time window, no context length error of any kind existed there, and the actual GPU KV cache usage was only 1 to 13 percent at that point, nowhere near the context ceiling. The model fabricated a plausible sounding excuse out of nothing.

A second, independent run with a deliberately simple, different query confirmed the same not_delegated root cause with a different specific narration, this time literal pseudo code as prose, and after the completion check explicitly told it no delegation call was ever made, the model apologized in text and then immediately narrated the exact same fake call as text again instead of actually invoking the tool. This satisfies the Model Evaluation Standard's point about needing more than one run: two independent runs, different queries, same root cause each time, with two different specific hallucination shapes.

The verdict: disqualified. This is the same broad failure class as mistral:7b-instruct's own same day verdict, narrate instead of call, only surfacing under DeepDelve's real, much longer system prompt, but a more severe, specific manifestation, fabricating entirely fictional system or error text rather than just paraphrasing the intended call. Not a serving layer issue, the isolated smoke test against the same endpoint and parser was clean.

A config fix unblocks Mistral family vLLM candidates, and mistral:7b-instruct gets disqualified

Continuing the same day's vLLM re test track. mistral:7b-instruct passed its isolated tool call smoke test cleanly, a genuine improvement over the original Ollama era "rejected at schema stage" verdict. But the real DeepDelve benchmark run failed immediately with the same chat template error that had already marked mistral-nemo:12b blocked, and that earlier entry had already predicted this would recur for this model and for devstral:24b, now confirmed live for the first.

This got root caused directly at the vLLM source, not just re confirmed from the earlier blocked entry: the tokenizer explicitly raises an error whenever a chat template or its keyword arguments are passed at all. Confirmed via vLLM's own PR history this is intentional and permanent, not a version fixable bug, since Mistral's own tokenizer library doesn't support overriding its built in chat template in native Mistral tokenizer mode at all, checked both the model's own HF card and vLLM's own docs and issues before concluding this, no server side flag or tokenizer mode variant avoids it.

This got fixed on DeepDelve's side with a new, explicit, user set config option that skips building the chat template keyword arguments entirely when enabled, rather than any model family auto detection, since this project has repeatedly found string matching heuristics unreliable. This unblocks all three Mistral family vLLM candidates at once, not just this one.

With the fix applied, mistral:7b-instruct's real benchmark run got past the 400 wall and hit a different, genuine capability failure: not_delegated, on two independent runs, different queries, identical result. Zero findings, zero sources fetched, both runs. The root cause, confirmed from the real session log content: the model consistently narrates its planned delegation call as a literal markdown fenced code block in its response text instead of actually emitting a structured tool call, even under an explicit tool choice setting. This reproduces the same narrate instead of call or write failure class already documented for other candidates at the writer role stage, but this one hits at the Planner's very first delegation call, immediately, since DeepDelve's real, much longer system prompt evidently pushes this model toward narration in a way the short isolated smoke test prompt didn't trigger.

The verdict: disqualified, a genuine capability gap, confirmed not the chat template infrastructure block, that's fixed and confirmed working, the 400 is verifiably gone and the request reaches the model successfully both times. Distinct from mistral-nemo:12b's own verdict, which remained blocked not yet re tested at the time, the fix removes its blocker too but it hadn't actually been re run with the fix applied yet, a separate, still open opportunity.

The vLLM bake off re test resumed: qwen2.5:3b-instruct and llama3.2:3b both disqualified

This continued the same day's session, resuming the vLLM re test plan after mistral-nemo:12b (blocked), llama3-groq-tool-use:8b (disqualified), and qwen3:8b (disqualified) had already been closed in earlier sessions. qwen3.6 (35b a3b) was checked and discarded outright on hardware grounds before spending any GPU time, confirmed via a direct API blob size query that its FP8 checkpoint alone is 37.5GB, more than double this hardware's entire VRAM budget, and no 4 bit pre quant exists.

Qwen/Qwen2.5-3B-Instruct got disqualified with a new failure mode found on vLLM, not the original Ollama era verdict's own reason. No quantization was even needed here, easy fit at bf16. The isolated single arg tool call parsed cleanly, but a nested array schema matching the real delegation shape consistently failed, 5 or more independent reproductions: the model reliably emits genuinely malformed JSON for its own tool call, closing the inner arguments object but omitting the final closing brace for the outer wrapper, and this wasn't a token budget cutoff, the model just stopped one brace short. This got verified as a genuine model generation defect, not a parser bug, by testing the parser's own extraction directly in isolation against the real captured response text, the parser's own regex extraction was correct, it's the extracted JSON substring itself that's malformed at the source. This is a different failure mode than the original Ollama based verdict, same overall disqualified verdict, new independent evidence via a completely different backend.

meta-llama/Llama-3.2-3B-Instruct also got disqualified, and this result is significant beyond just this one candidate, it directly challenges the vLLM re test track's own founding premise. This is a gated repo, requiring a real access request and approval cycle mid session. The isolated tool call smoke test with the exact same nested array schema used for the Qwen2.5 test above came back with the array argument encoded as a JSON string instead of a real array, the textbook shape of a known Ollama bug (ollama/ollama#6155, "Support Nested Parameters for Tools"), reproduced 4 of 4 times, fully consistent. But this ran on vLLM, not Ollama. The entire premise motivating this multi session vLLM re test effort was that this bug is an Ollama serving layer artifact that may have wrongly disqualified several candidates on a backend artifact rather than genuine model incapability, and this result directly disproves that hypothesis for this candidate specifically, the identical stringified array shape reproduces independently on a completely different serving stack. This got confirmed via vLLM's own official documentation's "Known issues" section for Llama models, which states plainly that the model can generate parameters in an incorrect format, such as an array serialized as a string. vLLM's own maintainers attribute this to the model's own generation behavior, not something their parser introduces or could fix, this is Llama 3.2's own limitation, independent of serving backend. The other two candidates already have their own independent, non related disqualification reasons, so this doesn't overturn either of those verdicts, but it does mean the framing that motivated re testing all three should be treated as disproven for the one candidate where a clean head to head comparison was actually possible.

After this session, the vLLM re test track status stood at: mistral-nemo:12b blocked, llama3-groq-tool-use:8b disqualified, qwen3:8b disqualified, qwen3.6 discarded on hardware grounds without testing, qwen2.5:3b-instruct disqualified, llama3.2:3b disqualified. Remaining untested at that point: gpt-oss:20b itself, deliberately deprioritized to last since it's the already trusted default with the lowest information value to re confirm, the schema stage rejects, the writer role failures, and the lowest priority remaining candidates.

Three more production bugs, plus a Gemma 4 fine tune disqualified

Continuation of the same very long day's session. Three more real bugs got found via live testing, each caught by actually reading final_report.md after a smoke test, not just confirming a mechanism fired, hardening this session's own standing rule that a completion check firing correctly is not the same as the report being correct.

First, several grounding checks compared cited URLs against fetched URLs with plain string equality. A citation the model percent encoded didn't match a fetched URL stored with the raw Unicode character, false flagging a genuinely correct citation as hallucinated, and this was root caused as the cause of a real run coming back with an empty report despite 27 real sources fetched. A new URL normalizer, percent decoding and trimming, got applied at every URL comparison site in the grounding module.

Second, delegate_tasks is one shared closure and quota across every tier, the Planner's top level dispatch and every specialist's own dispatch to its Analyzer children draw from the same global counter, with no per task sub budget anywhere. A live smoke test on a trivial single fact query saw one WebSearcher task call delegate_tasks 6 or more times in a row, despite its own prompt saying one authoritative source is sufficient and to stop early, consuming most of the run's entire global budget on one facet. A new specialist delegation cap got added, default 3, enforced by checking whether the caller is a specialist tier, never the Planner. A real bug got caught in the fix itself before committing, via the same live testing discipline: the first version used a plain contextvar set to update the counter across separate calls, which silently never accumulated, since the SDK dispatches each tool call as a separate async task, so a set inside one call only mutates that task's own context copy. Fixed by mirroring an existing mutable list pattern instead. A second live run confirmed the cap actually holds.

Third, the task verification check had no notion that a flagged task could be superseded when the Planner renames it on retry instead of reusing the task name. A live run got a flagged task renamed 3 times, and two of the renamed variants actually succeeded, verified, real Wikipedia backed findings, but the check kept re nudging the original stale names specifically, burning the entire completion check retry budget with zero report ever written, worse than the bug this session started from. Fixed with a second pass that downgrades a flagged entry to a new "superseded" status when its dispatched instructions closely match an already verified task's instructions, reusing an existing rename detection helper. Live verification status was partial: a third live run showed no repeat of the stale nudge loop, real improvement, but the run was still going after over 35 minutes on a separate, unrelated, still open problem, and got killed before reaching a final report.

Fourth, found while doing a Model Evaluation Standard raw API test on the Gemma candidate below but it turned into a general engine fix: the default options builder now also sends an explicit reasoning effort of "none" alongside the existing thinking disable flag whenever thinking is disabled, since some models ignore the flag entirely over Ollama's OpenAI compat endpoint but respect the explicit effort setting. Confirmed harmless against the gpt-oss baseline.

yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF got disqualified, run twice, identical failure signature both times. This is not the same candidate as another Gemma 4 12B row tracked separately, a different publisher and quant of the same base architecture, evaluated separately. Model Evaluation Standard point 1 got confirmed via direct curl against the real endpoint before any benchmark: plain completion clean, tool calling clean, but the standard thinking disable flag did not suppress thinking, the reasoning field stayed populated regardless, the same passthrough bug class already documented for Qwen3. Root caused to a working alternative, the explicit reasoning effort field on the same endpoint does cleanly suppress it, this is what motivated the general engine fix above.

Two live runs against DeepDelve's actual engine, different queries, showed an identical failure: run one, the standard stress test query, and run two, a deliberately narrow single fact query chosen specifically to rule out query complexity as a confound. Both runs ended identically with a forced abort, "Agent trapped in loop. Quota exceeded multiple times for write_todos." The root cause, confirmed from the real session log's actual tool call arguments, not guessed: the model calls write_todos with byte for byte identical arguments repeatedly, the same fully checked todo list, verbatim, call after call, not incremental revision, a genuine repetition loop. It continues even after the tool starts returning an explicit error telling it to summarize and stop, the model ignores this explicit corrective instruction and calls write_todos again with the identical content. All research work was actually done, real sources fetched both runs, zero search failures, but the model never transitions to writing the final report, DeepDelve's own anti loop safety net is what stopped the run, not the model recognizing completion.

This satisfies the Model Evaluation Standard's point about needing more than one run, and it's a distinct failure mode from every other disqualified candidate documented so far, not the thin coverage non convergence pattern, not citation fabrication, not a writer role tool call failure. This is a genuine inability to break out of a literal identical tool call repetition loop even when directly instructed via a tool error message to stop, the most severe agentic reliability failure documented in this project's bake off up to that point. It's not a serving layer bug either, since nothink and tool calling were both independently confirmed clean at the raw API level beforehand, and the failure only appears once the model is actually driving the full multi turn agentic loop.

Planner redelegation loop fix, plus two new production bugs found by live testing

This addressed several still open items from a prior session. Root caused why the Planner renames the same research angle across retries instead of redispatching under the same task name, which had recurred as 43 calls for one angle under a resumed run. The cause: two checks' own directives said "redelegate... phrased differently," which a model conflates with inventing a new task name, nothing anywhere said otherwise. Fixed the directive wording, and added a non blocking text similarity nudge in delegate_tasks that flags a likely rename against the run's own prior dispatches. Live confirmed firing across two separate live runs, reduces but doesn't eliminate renaming, since a prompt nudge alone can't guarantee compliance.

A live smoke test hit a more severe case of an empty response problem than an earlier fix had assumed: FindingsWriter produced nothing usable on both the original dispatch and its immediate retry, six consecutive completion check attempts in one run, exhausting the budget with findings.md never written despite 61 real findings existing the whole time. A deterministic, non LLM salvage path got added: the writer dispatch loop now accepts a deterministic fallback, wired only from the FindingsWriter call site as the real evidence text already assembled elsewhere, written directly as findings.md when both attempts fail instead of losing the cycle. A second, unrelated bug got found and fixed while verifying this against the real failing run's actual data: the URL extractor didn't stop at a trailing backtick, so the common inline code citation style broke every fetched URL match, which would have defeated the new fallback on the exact content it exists to rescue.

Caught by actually reading a live run's final_report.md content, not just confirming completion checks fired mechanically: a balanced two facet query, both genuinely "covered" per the coverage calculation, produced a final report that was 100 percent about one facet, the other vanished entirely at the FindingsWriter consolidation stage, and nothing caught it, since the thin coverage check saw balanced research volume and the findings underuse check compares the report against findings.md, which had already lost the data. A new check comparing findings.md's cited URLs against the real per task research record now fires when a covered task has zero of its real URLs represented. Replayed directly against the real failing run's data to confirm it would have caught the omission. A sixth GRPO reward function dimension also got added mirroring this check, for if and when fine tuning resumed, inert prep only at the time, not yet wired into training.

Later the same day, a fresh live smoke test confirmed the deterministic FindingsWriter salvage firing organically for the first time, live, outside any replay or unit test. But the run still failed overall for a separate, real reason: FindingsWriter repeatedly cited a fabricated near duplicate URL across 7 independent dispatches, despite the grounding pipeline already embedding a verification warning directly into that finding's own summary text at generation time, the exact same shape already logged two days earlier with a different domain, now confirmed to recur. Researched properly before fixing: embedded negative instruction warnings are documented as fragile by mechanism, naming forbidden content can prime its reproduction, and two primary papers on retrieval augmented generation both structurally filter flagged evidence out before the generator sees it rather than annotate and hope. The shared citability predicate now excludes findings carrying a verification warning marker, mirroring an existing relevance warning exclusion. This deliberately reversed an earlier design decision that had explicitly chosen not to exclude verification flagged findings, reasoning they "may still coexist with other real, usable content," a bet the live recurrence showed doesn't hold. Since the same evidence string feeds both the FindingsWriter dispatch instructions and the deterministic salvage fallback, this one change fixed both the recurring fabrication problem and the "salvage inherits poisoned evidence" problem simultaneously. Replayed directly against the real failing run's actual data: confirmed the fabricated citations no longer appear in the reassembled evidence post fix.

Live confirmed the same day on a fresh run: an early attempt flagged a fabricated citation via the findings ungrounded check, and unlike the earlier run's 7 attempt loop on the exact same bug, it never recurred in either findings.md or final_report.md. FindingsWriter converged normally on the next attempt instead of looping. The run's eventual outcome was a separate, pre existing, correctly handled failure mode, running out of wall clock budget while incorporating more sources, not a regression or a new bug.

Later still the same day, two research flagged pending ideas got revisited. One stayed deferred as a real future architectural direction. The other turned out to be based on an incorrect assumption, caught by re reading the actual dispatch code instead of trusting an earlier note: the full grounding pipeline, including NLI contradiction detection, already ran for Searcher tier dispatches. The real, narrower gap was that Analyzer leaf dispatches never got this at all, only a narrow reconstructed URL check with no stub detection, no quote fidelity check, no content level check, no NLI. Analyzer findings, reading and summarizing a single fetched document, exactly the shape most likely to misreport a figure, had materially weaker grounding coverage than Searcher findings for no principled reason. Fixed by adding the same full grounding call already proven at the Searcher tier to the Analyzer leaf branch too, alongside, not replacing, the existing reconstructed URL check, since the two catch genuinely different things. Zero new machinery was needed, everything already existed and was already proven in production via the Searcher tier check.

Live confirmed the same day on a fresh run: two Analyzer tier findings got flagged with problem types that were structurally impossible before this change, confirming the new grounding call was genuinely executing for Analyzer dispatches now, not just present in the diff.

VERIMAP Phase 1: a structural per task verification ledger

Also shipped the same very long day, after scoping in a dedicated plan. The literature review had flagged a paper called VERIMAP, where a planner encodes an explicit verification function per subtask, executed by a separate verifier before a coordinator proceeds. Exploring the actual code confirmed DeepDelve's whole check pipeline produces exactly one verdict per attempt for the whole run, no per task pass or fail record existed anywhere, and several bugs fixed this session, the starvation guard, the uneven investment check, the findings underuse check, are all individually patched symptoms of that same missing dimension.

A design tension got identified and resolved before building anything: the paper's own mechanism has the Planner author each verification function, which directly conflicts with this project's own documented principle of avoiding handing local models new structured conventions to follow, precisely because they've repeatedly proven unreliable at it. This got reconciled by having the engine compute the ledger structurally from already existing ground truth instead, keeping the paper's real contribution, that a task has its own checkable verification state, while dropping the part of its mechanism that doesn't fit DeepDelve's own hard won lesson.

A new per task verification ledger got added, keyed by task name, recomputed fresh every completion check attempt from findings and the citability predicate, purely additive, no new Planner facing field, no new prompt convention. A task with no findings yet is left out of the ledger entirely, still pending, not a problem. A new completion check, the first genuinely task scoped check in the pipeline, fires when a task's every finding was excluded by the citability predicate, naming that specific task in its directive rather than nudging the whole run generically, distinct from the thin coverage check (zero findings at all) and the uneven investment check (uneven counts across covered tasks), this one catches "produced findings, but every one turned out fabricated, off topic, or contradicted."

Explicitly deferred for a future phase: actually independent per task redispatch that bypasses the Planner's own turn, since this session's ledger still only produces one whole run verdict per attempt, same shape as every other check, and the deeper dispatch loop rework is real, separate scoping work, worth doing only after seeing how often "flagged" recurs on real runs.