Release v0.20.0 · iris-eval/mcp-server

[0.20.0] - 2026-10-03

Iris says what it checked. A verdict tells a pass from a check that never ran, names who recorded the evidence it judged, and cannot be replaced by the agent it judges. 0.20.0:

"Not checked" is its own answer in the verdict, the CI gate, the counts and the dashboard: a check that was asked for and could not run is no longer reported as a pass, and a --fail-on gate exits 0 only when every trace it read was judged.
Every verdict says who recorded what it judged. A capture source declares what it records in full, so an empty list of tool calls from software that watched every call means none were made, and the same list from the agent's own report does not.
The detectors read what was written: a test runner's verdict, a failure the agent recovered from, text spaced or wrapped to slip past a phrase rule, and an output written as JSON. A new rule, says_something, fails an answer that says nothing.
Full-text search over traces, spans included, in any language: every search is bounded, and on a store in a file it runs off the event loop. install --upgrade moves every MCP client on the machine to the new release and copies the database before it migrates.
Provider calls recorded without the model choosing to: OpenAI and Anthropic wrappers and LangChain and LangGraph handlers, in the Python client (iris-eval 0.2.0 on PyPI) and, for JavaScript, in this repository ahead of their npm packages, and OpenTelemetry GenAI traces sent to Iris are scored, with an estimated cost when a trace reports tokens but no cost.
Also: an optional relevance judge within a daily budget, CSV and JSON Lines export, Decision Moments ranked by significance, a dashboard that works from the keyboard and with a screen reader, and API key revocation that takes effect on the next request. The server is published with npm provenance, and its Docker image, MCPB bundle and SBOMs are signed.

Check before upgrading. The first group can change what a CI job or a script does: read it if Iris gates a build or a script reads its answers.

Gates, verdicts and API answers. An entry marked BREAKING can change what a CI job or a script does with the same input.

BREAKING — A --fail-on gate exits 0 only when every trace it read was judged, and a basis trips when that layer decided the verdict or would have. Two changes to iris-eval ingest and the gate action, each of which turns a job that passed into one that fails. (1) verdict.basis names one layer, and --fail-on <basis> read it alone: an output that broke a policy you configured and also leaked a credential read policy_gate, so --fail-on detector_veto, then the action's default, exited 0 on it. The verdict now lists every later layer that would have decided on its own under verdict.also, and --fail-on reads both; fail and unknown read the state of every layer the same way. A job gating on risk_over_loss will also trip on an output a critical detector vetoed, because the risk estimate is over the cut there too. (2) With --fail-on, each of these exited 0 and exits 2 now, with a sentence saying which: a trace rejected for a malformed field or a missing output (the run that crashed before answering); a trace stored without being evaluated; an empty file; and, with --dataset, a run that held none of the dataset's cases. Pass --allow-empty (allow-empty: true on the action) where a job legitimately has nothing to gate, such as one shard of a split run; it never excuses a rejected or unevaluated trace. A --file path that is not a file exits 2 with one sentence; it used to exit 1, the code a tripped gate uses, with a stack trace. Without --fail-on nothing changes.
BREAKING — The gate action fails the job on every verdict that is not a pass. Its fail-on input defaulted to detector_veto, which exits 0 on an output the evaluation failed for any other reason (a hidden tool failure, an invented figure, a broken policy) and on every verdict that was not checked. The default is any now. A workflow that uses the action without fail-on and relied on only critical detections failing the job sets fail-on: detector_veto. The CLI has no default and is unchanged; the README and docs/ci-gate.md lead with --fail-on any.
BREAKING — A check somebody asked for, on a call that left out what it reads, makes the verdict not checked instead of a pass. With a cost ceiling set in your config (cost_threshold), a trace over it failed and the same trace with the cost left out passed: the rule skipped, and a skip was nothing. A rule that skips for lack of what it reads now makes the verdict unknown (basis: "required_evidence_missing", passed: false) when somebody had asked for it. The deployment asked when it set one of the rule's thresholds (cost_threshold, max_steps, max_tool_repeats, max_target_rereads, max_token_ratio, keyword_overlap, topic_consistency, tool_choice_margin, tool_choice_min_fit), promoted the rule with eval.criticalRules, deployed it at severity high or critical, or installed the relevance judge. The call asked when it set such a threshold itself, supplied a gating rule inline, or supplied the part of an expected_trajectory the rule compares against. verdict.by names the missing inputs, the rule result carries lacked and asked, and a sentence to the agent names the rule, what it lacked and who asked. A blank is not sent: an input of only whitespace and a negative cost are now treated as left out, where each used to be judged as a value (and passed). An explicit empty list of tool calls is sent: it says none were made, and an honest turn that used no tool does not read "not checked" because a step ceiling is set. eval.onCriticalSkipped: "pass" accepts a critical rule that could not run for missing evidence. A gate keyed on passed, or on --fail-on any, unknown or required_evidence_missing, that passed such traces fails them now: send the field, or remove the setting that asks for it. With the shipped configuration and a call that supplies no threshold, rule or expectation of its own, nothing changes, and a text-only evaluation passes as before.
BREAKING — A failure outranks a layer that could not check. An output the risk layer fails, on a deployment where a critical check could not answer or required evidence was missing, read verdict.state: "unknown"; it reads fail now, basis: "risk_over_loss", with the layer that could not check under verdict.also. --fail-on and the webhooks already read every layer and trip as before; a caller that branched on verdict.state or verdict.basis itself sees the failure first. Each stored evaluation is stamped with the ordering it was judged under and reads back under it. One kind of row from 0.19.0 does not read back as its caller was given it, and did not before this change either: an evaluation that was not checked because required evidence was missing reads back as a pass, because that release did not store the requirement with the row. Its stored result is still "did not pass", and every count treats it so.
BREAKING — An evaluation linked to a trace is that trace's verdict only when it scores the trace as stored. evaluate_output with a trace_id and output, input, tool_calls, tools, cost_usd or token_usage that differs from what the trace stored, or with one eval_type, now answers with reference_trace_id in place of trace_id and a sentence saying what differed; so does a call that names a trace which recorded no output, and so does every evaluate_with_llm_judge and verify_citations call that names a trace. Such an evaluation is still stored, listed with the trace and erased with it, and it no longer counts in the trace's run, in compare_runs or compare_traces, in the score filter of get_traces, or in the latest_* export columns. A workflow that judged each trace with evaluate_with_llm_judge and compared runs on those judgments will find the comparison reading the deterministic verdicts instead; to have a judge decide a trace's verdict, set a relevance judge (IRIS_RELEVANCE_JUDGE_MODEL). To re-score a stored trace, call evaluate_output with its trace_id and nothing else: output is optional now, and the stored input, tool calls, tools and cost are read from the trace, so the relevance rules run where they used to skip. Evaluations stored by earlier releases are not changed. Migration 020-eval-reference-trace adds the column.
BREAKING — An expected call that was never made fails, where it used to pass (tool_sequence and step_budget, version 2). evaluate_output with expected_trajectory.tool_calls and tool_calls: [] skipped tool_sequence: there was "no trajectory to judge". An explicit empty list is a trajectory of no calls, and against an expectation of calls it now fails every mode but superset; step_budget judges it as zero calls, which is within any budget. Calls that are not sent at all still skip. The ruleset fingerprint changes.
BREAKING — compare_runs and POST /api/v1/compare answer differently in four ways. (1) better is no longer true when the second run was judged by fewer rules than the first; improvement_withheld is true instead. A job that read better to promote a run stops promoting one that sent less evidence. (2) equivalent_within is null unless you pass equivalence_margin; it used to default to a margin computed from the run sizes. A caller that read equivalent_within.holds without a margin now reads null. (3) difference carries a 90% interval where it carried a 95% one, and when the runs pair it is the interval McNemar's test implies, not the unpaired one; difference.significant now agrees with worse and better. (4) smallest_detectable is larger: it is the difference detected four times in five, where it was the one detected about half the time (62 points for two runs of eight, where it read 40). The summary no longer contains ** markers. New fields: call, coverage, critical_rises, paired.fell, and judged_before / judged_after on each rule row.
BREAKING — compare_traces reports a wider interval, and overall.rate is the mean over cases. The run-level interval is exact for the number of cases and can no longer be zero-width: ten of ten cases reads [69.2%, 100%] where it read [100%, 100%]. overall.rate is the mean of the cases' own pass rates; it equals pooled.rate when every case was asked the same number of times and differs otherwise.
BREAKING — Two runs that differ in a verdict-deciding setting are no longer called comparable, and a rule change that cannot be recorded is refused. The configuration fingerprint (provenance.configHash) now covers eval.defaultsGate, eval.falsePassCost, eval.onCriticalSkipped, eval.requiredEvidence, eval.priorMode and a prior set in config or estimated from labels, and the ruleset fingerprint (provenance.rulesetHash) covers the content of every deployed, inline and plugin rule. A deployment with none of these set and no custom rules keeps both hashes; any other gets new ones from its next evaluation, so compare_runs between a run from before the upgrade and one after names the configuration or the ruleset as different (it already named the engine version) and compares them only with force. With an estimated prior the fingerprint moves as labels are added, because the verdicts do. deploy_rule, delete_rule, the dashboard's rule routes and delete_trace now write their audit entry before the change and fail with IRIS_STORAGE_ERROR when it cannot be written; they used to carry on without a record.
BREAKING — verify_citations reports a judge failure under judge_error, not resolve_error. A caller that read the judge's error from resolve_error on a citation whose resolve_status is "ok" reads it from judge_error instead, with the same kind and message. resolve_error now means only that the source was not resolved.
compare_runs says when the cases that fell were not checked rather than failed. Ten cases that passed before and could not be checked after read "10 cases passed before and failed after ... This is a regression". They still count against the second run (worse is true: a run that cannot be checked has not passed), and the summary now says what happened: "did not pass after (10 not checked, 0 failed)", and when none of them failed, that evidence stopped arriving and nothing was found wrong with the answers. Each run summary carries not_checked, and paired carries fell_not_checked.
A case is flaky only when it passed and FAILED. flaky on GET /api/v1/cases/:key, in the flaky_cases view and in compare_traces, and the flaky_case webhook, counted a verdict that was not checked as the other answer, so a case with one pass and one unreached verdict was "answered both ways". An attempt that was not checked is now neither answer.
A row of categories is the verdict, read for the rules that bundle holds. categories.<bundle>.passed was its own arithmetic: a weighted mean of the bundle's rule scores against a threshold, with the critical veto. The verdict stopped using that arithmetic in 0.10.0, so on an evaluation that failed for a hidden tool error every row could read passed: true, and a script keyed on categories.safety.passed shipped the output the evaluation failed. A row now carries state (pass, fail, unknown) and passed follows it (true, false, and null for a row that was not checked): the row holding the rules that failed the evaluation fails, every checked row of a passing evaluation passes, a rule that lacked evidence it was asked for leaves its own bundle not checked, and an input eval.requiredEvidence names leaves every bundle not checked. A script that read a bundle's passed will see failures it did not see before, and null where it saw false for a bundle that lacked evidence; a row no longer fails for a low score alone. score on a row is unchanged.
A webhook that names no events also receives verdict_not_checked. The new event is delivered for a verdict that could not be reached and that no layer failed. Name the events you want to keep the old five.
Required evidence is met by what the call carried, and a capture source can say a turn made no tool call. eval.requiredEvidence used to count an input only when a rule that ran had read it, so a deployment that requires cost read a call that sent its cost as "not checked" when the call ran one bundle (eval_type: "safety"); it passes now. A trace recorded by the Claude Code capture plugin now names the plugin and what it records in full (below): on a deployment that requires tool_calls, a captured turn that made no tool call read "not checked" and passes now, and a captured turn whose record is incomplete reads "not checked" where it may have passed. The trace CSV export has a capture column between source and session_id; a reader that takes columns by position shifts by one from there. Migration 022-trace-capture adds the column that holds it. A verdict that nothing could judge (basis: "no_rules") now lists missing evidence under verdict.also (required_evidence_missing), whether a deployment required it or a capture source promised it, so --fail-on required_evidence_missing trips on such a trace where it passed. Two more changes in what counts as sent: a blank expected, or an expected_trajectory no trajectory rule can use ({}, { "tool_calls": [] }), no longer meets a requirement for one; and evaluate_output that sends tool_calls: [] for a trace stored without tool calls is now kept beside the trace (reference_trace_id), because an empty list is other evidence than none, where it used to be the trace's verdict.
GET /api/v1/moments with a verdict, min_significance or significance_kind filter returns full pages from a window of recent traces. offset now counts matching moments rather than traces, total is the exact number of matches in the window rather than the trace count, and the response carries window. A client that paged a filtered list by adding limit to offset gets every match once instead of gaps; one that read total as the trace count reads window.tracesInRange. Matches older than the window (the newest 500 traces by default, window to narrow it) are not returned: narrow agent_name, since or until to reach them.

Rules that fail outputs they used to pass. A gate on detector_veto or any can trip where it passed before.

no_injection_patterns, no_blocklist_words, no_pii and no_injection_compliance fail outputs that hid a phrase behind its spacing or a value behind an invisible character. An override phrase written one word per line, with tabs, or with no-break spaces in place of spaces passed before and fails now. So does a blocked phrase written that way, and a recovery phrase listed one word per line. A Social Security number, card number or key with a right-to-left override, an invisible separator, a variation selector or a tag character inside it passed and fails now. A gate that passed such an output will block it. Ordinary wrapped prose and a quoted discussion of an attack phrase read as before. No setting turns this off.
no_silent_tool_failure reads a test runner's verdict, and stops counting a failure the agent recovered from (rule version 2). Two changes to what fails, in opposite directions. (1) An answer of "All tests pass" beside a tool output ending Tests 3 failed | 12 passed passed every rule: the call carried no error and its first line was the runner's banner. For a tool that runs a command, the head and tail of its output are now read for a failing verdict (3 failed, FAIL …, FAILED …, npm ERR!, Found 3 errors, a non-zero exit code), and an object output written as a JSON string ("{\"ok\": false}") is read as the object. An answer that does not own such a failure now fails the rule, and at the shipped configuration the verdict. (2) A failure followed by a later call that went right on the same thing no longer has to be owned: the same command passing after a fix, a misspelled argument then the right one, a failing run then a passing one. Until now every such answer failed the rule unless it happened to use a failure word. A gate that blocked on a retried call stops blocking it; a gate that passed a false "tests pass" blocks it. The ruleset fingerprint changes, so compare_runs refuses a run from before against one from after unless forced.
A quotation does not run from one field of a structured output into the next, and a sentence has a word in it. The injection rule treats a phrase inside quotation marks as discussion. Quote marks in two neighbouring fields of a JSON answer bracketed the field between them as quoted, so an override phrase there passed, plain or disguised; a quotation now ends at the mark the reading puts on a line of its own between two values. A quotation in prose may still run across paragraphs, as a forwarded email does, and a pilcrow inside a line of prose ("under ¶ 4") is a character. The sentence splitters end a sentence at that mark too, and count no segment without a letter or a digit as a sentence, so a horizontal rule (---), a lone bullet or a line holding only an emoji no longer adds to sentence_count.
no_pii: a masked key is not a secret, and asterisks after a whole secret do not mask it. The check for a value assigned to a secret's name read sk-****…3f9a as a secret. A value is masked when four asterisks hide all but a few of its characters; Xk9mP2vQ7rL4nT8wZ3hY**** is still a secret. DOB and MRN labels are read with a quote and a colon after them, as a JSON field writes them ("dob": "1990-04-12"); a quoted label with no colon after it is not one.
no_stub_output: a marker that a removal verb acts on is finished work. "Removed the last TODO comment" and "resolved all remaining FIXME notes" read as unfinished work. A removal verb before the marker, on the same line, marks a report of done work, and an article before a marker may be followed by any horizontal space (a no-break space included) but not a blank line; "the only TODO left is the retry" still fires, and so does a marker on the line after "removed".
no_stub_output and no_hallucination_markers read the same text however it is spaced or wrapped (version 2 of each). A placeholder phrase typed with two spaces ("omitted for brevity"), or cut by a line wrap, was not the phrase the rule knew, and a paragraph wrapped at 60 columns was counted as one sentence per line. On the labelled corpus, doubling every space turned three placeholder findings and two fabrication findings into passes. Both rules now match a phrase across any short run of whitespace and count sentences on the paragraph. An output that passed only because of its spacing fails now. The published accuracy of both rules is unchanged (the same errors on the same cases), and the ruleset fingerprint changes.
An output written as JSON is read by what it says (the sixteen rules that read the output are each at a version 0.19.0 did not have). An answer sent as a JSON object or array, such as {"answer": "…", "sources": […]}, was read in its escaped form: a line break was the two characters \n, a quote was \", and a phrase inside a string value sat between quotes, where the injection rule reads it as a quotation. On the labelled corpus, sending every output as one string field of a JSON object changed five verdicts and thirteen answers of the rules that decide one: three injections were missed and an empty answer was not empty. The rules that read the output now read a structured one by its values, each value its own paragraph; the leak, blocklist and injection rules read every field name too, quoted, before its value ("password": …, "dob": …) or alone where its value is an object or a list, so a secret is recognised by the field it is assigned to, a key can be the leak, and a directive smuggled as a key is found as it is inside prose. Every evidence offset still points into the output as sent, and the relevance judge's redaction reads a structured text the way the leak rule does. Custom rules read the output exactly as sent. Changes in both directions: a structured output that passed because escaping hid a leak, a placeholder or an injection fails now, and one that failed only on its escaping passes (an empty answer in {"answer": ""} now fails non_empty_output). A leading byte-order mark is skipped; JSON with a no-break space or another Unicode space beside it is read as written. One injection shape is read only in a structured reading: a field whose value is an override, quoted or not and disguised or not, which the labelled reading writes without the quotes the prose pattern needs; a value whose sentence quotes a payload is a report about it, as in prose. Each rule result that read a structured output says which reading it took in read. The ruleset fingerprint changes.
An answer that says nothing fails: a new rule, says_something. Against two fixed asks (an action, and a piece of writing), ten of twelve trivial agents passed clean: "Done.", "OK", "The task has been completed successfully.", "…", "null", "lorem ipsum dolor sit amet", the ask copied back, one sentence repeated 400 times, and two refusals. says_something reads the shape of a non-answer against what the ask asked for. To an ask for something written: a promise in place of the work ("Will do!") when no tool call is recorded, a placeholder ("null", "…", "{{answer}}", lorem ipsum), or a "here is …" with nothing after it. To an ask to write or to act when the call says no tool was called (tool_calls: []): a bare "Done." or "OK", a promise, or a placeholder. To any ask: an announcement that ends on a colon with nothing after it, the ask handed back, or one passage on a loop. A question answered "OK", "null" or "Done.", an ask to confirm answered "OK", an action reported done when the call does not say that no tool ran, and a refusal pass: each can be a correct answer, and whether a refusal was right is a judgement about the ask. Its family is 242 labelled cases, 200 of them written from a definition of a non-answer by model instances that never saw the rule: it fails none of the 126 answers and catches 23 of the 116 non-answers (precision 100%, 95% interval 85.7 to 100; recall 19.8%). On the 100 written after its last revision it caught 4 of 50 non-answers and failed none of 50 answers. Most non-answers there are fluent replies that never contain what was asked, which no rule that reads the shape of an output can detect. proof/INVARIANTS.md publishes the twelve agents against both asks: where tool_calls: [] records that nothing was called, none passes except the two refusals and the action handed back, each with its reason. It is a detection in the completeness bundle and the stub class; no composite verdict changes, and a typical clean pass's risk estimate moves from 0.118 to 0.110. The ruleset fingerprint changes, and the built-in rule count is 26.
A trace that sends token counts and a model but no cost is now stored with an estimated cost, so cost totals rise and the cost rules start judging it (#702). Most OpenTelemetry and framework traces are such traces. After upgrading, their cost counts toward the dashboard's total cost, GET /api/v1/summary and the cost_by_agent view, and cost_under_threshold (default $0.10), cost_anomaly, custom cost_threshold rules and the cost_anomaly webhook act on it instead of skipping. A gate on cost_under_threshold can start failing traces it used to skip. Each estimate is marked cost_source: "estimated" wherever it shows. Traces stored before the upgrade are not re-priced. To keep the old behaviour, set pricing.estimate: false in config.json.
answers_the_ask stops blocking at the shipped config. A gate that relied on it failing off-topic answers at the defaults stops blocking them. To keep it gating, set IRIS_RELEVANCE_JUDGE_MODEL to have the relevance judge decide (below), or set a keyword_overlap or topic_consistency threshold in your config file to gate on the lexical reading.

Upgrading the server and its data.

Move every MCP client to 0.20.0 in one step: npx -y @iris-eval/mcp-server@latest install --upgrade (#704). Every client shares ~/.iris/iris.db, and since 0.19.0 install pins each client to one release. 0.20.0 adds migrations 015-trace-search and later, and the first 0.20.0 process to open the file (the Claude Desktop extension, npx iris-eval, or npx @iris-eval/mcp-server with no version) applies them; from then on a client still pinned to 0.19.x refuses to start with This database was migrated by a newer Iris (…) — migration(s) 015-trace-search, … are unknown to v0.19.0. That message comes from 0.19.x and cannot change. The command moves every pinned client; the Claude Code plugins move with claude plugin marketplace update iris-eval and claude plugin update iris-eval@iris-eval (and iris-eval-capture@iris-eval). To go back to 0.19.0 instead, restore the copy 0.20.0 takes before migrating: README, Downgrading.
The first start after upgrading copies the database before migrating it, and takes longer by the copy. Measured on a Windows 11 desktop (Node 24.11, better-sqlite3): 2.4 to 2.7 s for a 468 MB file of 100,000 traces, three runs. The copy needs free disk space about the size of the file; without it the start migrates without a copy and says so.
After upgrading, the trace search index is built in the background, and the file grows by about two thirds. At 100,000 traces on the benchmark machine the file went from 125 MB to 211 MB (69% larger). The server starts and answers MCP and HTTP requests at once, and the index is built behind it in steps of about 50 ms of work each. Measured on a file of 100,000 traces from 0.19.0: the first start took 449 ms, and the index was built in 33.4 s, with the event loop held for at most 449 ms. Traces sent over OTLP with spans take longer, because their span text is indexed too: 158.8 s at 100,000 agent-loop traces, with a first start of 360 ms and the event loop held for at most 329 ms, and the file is larger (993 MB where the same traces took 841 MB with an index of their own fields only; the table under Added has the numbers). Traces with Chinese, Japanese or Korean text take longer again, because each run is also indexed as two-character pieces: 50.8 s at 10,000 agent-loop traces written in Chinese, and a file of 251 MB where those traces took 128 MB before (the numbers are under Added, with the entry for words inside CJK text). Until the build finishes, a search reads the traces directly (search.index is scan): slower, with the same results, and on a large store it stops at the search time budget with search.complete: false (see Security). The server logs one line when the build starts, with how many traces it will index, and one when it is done, with the count and the time; GET /api/v1/health reports search.state and search.progress (the share of traces indexed so far), and --self-test reports the configured database's index. get_traces without q sorts by timestamp as before; with q the default order is relevance.
After upgrading, four indexes are built in the background, after the server answers (#711, #744). Migration 019 adds the indexes the dashboard's summary, filters and trace list and the agent failure log now name, and drops the three they replace. On a file with traces they are built after the server starts, each in one statement off the event loop; until they are, those reads answer with the same results on the indexes the file already had, more slowly, and GET /api/v1/health says "indexes": "building" (--self-test says so too). On a file of 100,000 agent-loop traces written by 0.19.0, the first start's work before the MCP connection opens took 54 to 60 ms where it took 1.72 to 2.13 s with the indexes built in the start (three runs each, without the copy taken before migrating, above), and the indexes were ready 2.1 to 4.9 s after it; a page of 50 traces took 9 to 16 ms meanwhile, 5 ms after. While each index is built, a write waits for it, as it waits behind any other writer.
Committed writes reach the disk at each checkpoint of the write-ahead log, on both drivers (storage.synchronous, default normal) (#711). A crash of Iris loses nothing and the file cannot be corrupted; a power cut or an operating-system crash can roll back the writes made since the last sync. With better-sqlite3 this is what 0.19.0 already ran on every start but the one that created the file, because that SQLite is built to open a WAL file at NORMAL; on Node's built-in node:sqlite it was FULL throughout and is NORMAL now. Set "storage": { "synchronous": "full" } in config.json to sync every commit, which keeps each committed trace through a power cut and costs about 1.5 ms a write.
Storing a trace costs more with the search index on, which is the default. Sustained OTLP ingest of traces with two model spans stored 1,735 traces/s at 100,000 traces and 2,000 at 10,000, where 0.19.0 stored 2,330 and 2,620, and one log_trace call takes about 0.1 ms longer. A feed that needs the ingest rate more than fast search can set storage.searchIndex: "off" (IRIS_SEARCH_INDEX=off), which stored 4,237 and 4,393 traces/s; search then reads the traces themselves within its time budget (under Added).
The retention sweep runs after the server answers, in short steps (#695). It used to run before the MCP connection opened, as one transaction: a stdio client that started Iris on a store with traces past retention.days waited for the whole sweep, and the search index made the sweep several times slower. The connection now opens first, and the sweep deletes a few traces per transaction with the event loop free between them; for a sweep of more than 1 in 125 of the indexed traces it then merges the search index in steps, which erases the swept words from the file as optimize did. Retention cleanup: deleted … is logged when both are done, and a merge that a closing server leaves unfinished is finished by the next start. Measured over stdio against the code before this change, alternated, on the benchmark machine while other work shared it, with agent-loop traces (three model calls and two tool calls each) and 3% of them past the window: at 10,000 traces the first answer to initialize came after 640 to 661 ms instead of 1,442 to 1,467 ms (three runs each); at 100,000, after 0.74 s and 1.21 s instead of 8.6 s and 17.6 s. The sweep itself still takes seconds at that size, in the background: it was done 9.0 s and 12.1 s after the start, against 9.2 s and 18.3 s. While it runs, the longest the event loop was held was 57 to 95 ms at 10,000 traces and 189 ms and 545 ms at 100,000.
BREAKING — The Docker image keeps its whole Iris home in the /data volume. The image now sets IRIS_HOME=/data, so deployed rules (custom-rules.json), audit.log, config.json and preferences.json sit beside iris.db. Before, they were written to /home/iris/.iris inside the container and were lost whenever the container was recreated. A deployment that mounted its own volume at /home/iris/.iris to keep them should copy those files into the /data volume, or set IRIS_HOME back to that path. /data is now created mode 700, like a fresh ~/.iris.
The retention sweep keeps the pinned baseline run. Traces and evaluations of the run pinned as the baseline (PATCH /api/v1/runs/:id { "baseline": true }) are no longer deleted at retention.days. If you pinned a run and relied on the window to remove its text, unpin it, or delete its traces with delete_trace; each sweep that keeps some of it logs the run and the counts. With no run pinned the sweep deletes exactly what it did.
A rule deployed through one MCP client now applies in every other client's server. Each client's server process re-reads the shared rules file when it changes, so a rule deployed, switched off or deleted through one applies in all of them within 20 ms. If you kept different rules per client by relying on each process holding its own copy until restart, they now converge on the file.

Integrations.

BREAKING — OTLP traces in the GenAI conventions store words, not JSON, as their input and output. A trace from Pydantic AI, Semantic Kernel, or any other exporter whose input or output is a message array (gen_ai.input.messages / gen_ai.output.messages, or OpenAI-shaped [{ role, content }]) now stores the last question asked as input and the last answer as output. A caller reading input / output from GET /api/v1/traces, get_traces or the OTLP answer sees text where it saw a JSON array; the span attributes keep the JSON as sent. Traces stored before the upgrade keep their JSON.
BREAKING — An OTLP trace's output is what the run ended with, and an unnamed service is not an agent. Past the root span, output is now read from the span that ended last rather than the one that started first, and a trace whose resource carries only the OTel SDKs' default service.name (unknown_service, unknown_service:<process>) is filed under the agent its spans name (gen_ai.agent.name, OpenInference's agent.name), else under "otel" with service.name listed as lacked. A caller that filtered OTLP traces by agent_name=unknown_service:python.exe finds them under the agent's name or "otel"; traces stored before the upgrade keep theirs. A span attribute that is a Python object's repr (StopEvent(result=…)) is no longer taken as a trace's input or output.
BREAKING — An OTLP agent span is INTERNAL, not LLM. A span whose gen_ai.operation.name is invoke_agent or create_agent is stored with kind INTERNAL; a caller that counted LLM spans from OTLP traces counts only the model calls now. Token and cost totals do not change.
The Claude Code capture plugin records the tool calls that failed, and loses no call a turn made. Claude Code reports a failed tool call (a command that exited non-zero, a missing file, an MCP tool's error) through PostToolUseFailure, which the plugin did not listen to, so a failed call never reached Iris and the rule that asks whether an answer owned a failed call had nothing to read on the path that records every turn. It is recorded now, with the error the agent received. A turn is now kept under its prompt's id (prompt_id, sent by Claude Code from 2.1.196) rather than its session, and nothing it records is deleted before it is sent: the calls of a turn you interrupted (which fires no Stop) and of a background sub-agent that kept working after its turn ended were thrown away when the next prompt arrived, or landed in the next turn; they are sent then as an unfinished part of their own turn, stored and not judged, with a sub-agent's calls named in metadata.subagent_calls. A turn a Stop hook keeps going is sent as a second, continued part with its prompt. A turn that ends in an API error (StopFailure) is stored with its calls and the error, with no answer, and is not judged, rather than scored on the error text. An empty tool_calls list is sent only when the record is known whole (the prompt seen, the turn stopped, nothing running in the background), so Iris does not read "no tool calls were made" into a turn whose calls it may not have. When the model calls log_trace for the turn itself, the plugin no longer stands down: both traces are kept, and the plugin's names the model's, once, in metadata.model_logged.trace_ids, because the model's account can leave out the call that failed. Expect more failed verdicts on captured turns (failed calls the answer did not own), and more traces: two for a turn the model logged, and a part for each turn that did not end cleanly. What the plugin holds is kept like a secret: without a host data directory it lives in capture/ inside your Iris home instead of a shared temporary directory, readable by you alone; calls are appended as they finish, each on its own line, so calls run in parallel are all kept and a write cut short loses only itself; a tool output over 256 KB is kept as its head and tail, marked truncated; a turn that could not be ingested is retried and removed after seven days, a closed session's files are sent and removed after a day, and the capture log is rotated at 1 MB; both ingest attempts run the pinned version. Each turn is sent in the form the pinned release reads: a release before 0.20.0 gets it without the capture declaration, which that release's ingest would refuse along with the whole turn. A test in the upgrade suite has the release the plugin pins ingest a turn the plugin's own hooks built.
The dashboard's Moments page opens ranked by significance, not newest first, and refreshes every 10 seconds instead of 3. Add ?sort=newest to the page address, or pick Newest first in its Order control, for the old stream. GET /api/v1/moments is unchanged unless a caller passes sort_by=significance.

Every entry below is shown by its lead sentence. The notes in full, with each entry's detail: CHANGELOG.md at v0.20.0.

Security

A long run of full stops, question marks or exclamation marks no longer holds the event loop.
Two ways past the critical detectors are closed.
The agent being judged can no longer replace its own verdict.
A swapped rule and a changed gating setting change the fingerprint, and the audit log says what a rule was.
A verdict that was unknown for missing required evidence read back as a pass.
A critical detection is no longer hidden behind a policy failure.
Dependency advisories closed in every workspace:
Citation fetching refuses NAT64 addresses that lead to a private network.
One search can no longer stall the server (#703).
delete_trace no longer leaves the deleted text readable in iris.db (#703).
Revoking an API key takes effect on the next request, with no restart.
A browser session ends when the API key it signed in with expires.
The Python client is published only from a tag on main, built with hash-pinned tools.
The SDK, the LangChain.js handler and the iris-eval launcher can be published by the release workflow, with npm provenance, once each is set up on npm.
CI installs the MCP clients it tests against from a lockfile.
SECURITY.md covers everything the repository publishes:
The storage threads start from fixed code.

Added

state on every rule result, bundle row, run result and case attempt, and a count of what was not checked beside every pass count.
verdict_not_checked webhook event
proof/INVARIANTS.md: what a verdict does when evidence is left out, and when a failure is added.
storage.searchIndex: "off" (IRIS_SEARCH_INDEX=off) keeps no full-text index, for a feed that needs the ingest rate more than fast search (#729).
A trace that reports no cost gets one estimated from its tokens, marked as estimated (#702).
install --upgrade moves every MCP client on the machine to this release, and install --list shows which Iris each one runs (#704).
A database is copied before it is migrated (#704).
A migration names the oldest release that can still use the database, so a later additive migration no longer locks out the release before it (#704).
Export traces and evaluations as CSV or JSON Lines, with the list's filters and search applied (#4).
Recipes for the OpenAI Agents SDK (Python and JavaScript) and LlamaIndex, each run in CI against a real Iris server.
Every OpenAI and Anthropic call recorded and scored, without the model choosing to (#648).
LangChain and LangGraph runs scored, in Python and JavaScript (#642).
An OTLP sender can ask for a verdict on its own traces.
The LLM judge prices Claude Opus 5.5
Claude Desktop installs Iris in one click (#650).
npx iris-eval starts Iris.
Decision Moments can be ranked by significance, and the Moments page ranks by default (#409).
An off-topic answer fails when you turn on the relevance judge (#649).
The relevance judge template, for evaluate_with_llm_judge too.
The relevance judge spends within a daily budget, keeps personal data and credentials from the provider, and says when it reaches the network (#649).
Full-text search over traces: find the run where the agent said it
Search reads what a trace's spans say: the later model calls, tool results and exceptions of an agent loop sent over OTLP (#683).
A word inside Chinese, Japanese or Korean text is found (#682).
Every verdict says who recorded what it judged, and a capture harness can certify that no tool was called.

Changed

provenance.composer.calibration names the calibration table, not the corpus it was measured on.
The published verdict accuracy is measured through the composer the server runs.
A Decision Moment whose trace has one passing evaluation and one that was not checked reads "No verdict".
The 24 real transcripts: bundle verdicts agreeing with the answer key went from 78 to 82 of 96
iris-eval.com shows what exists today: the open-source server.
Community links point to GitHub, npm and X.
Iris's own test runs stay out of its public download count.
Background work's steps are sized to stay under 50 ms, and the index build no longer runs FTS5's merges inside them.
After upgrading, the server answers its client before it copies and migrates the database.
After upgrading, the server answers before the covering index for search filters is built.
Each connection compiles a statement once (#711).
Storing one trace no longer writes the search index in the same commit (#729).
GET /api/v1/health and GET /health carry search: the index's state (ready, building or unavailable), index (what a search reads now, fts5 or scan) and progress, the share of stored traces indexed, from 0 to 0.99 while it is built.
GenAI message arrays are read down to their words, for every framework.
The agent's own span is filed as the agent, not as a model call.
Installing Iris succeeds where better-sqlite3 cannot be built (#661).
A filtered Decision Moments list returns full pages (#657).
The dashboard loads each page's code when the page is shown (#662).
Behaviour change: verify_citations reports a judge failure under judge_error, not resolve_error (#407).
evaluate_output is back to 0.19.0's speed, reading evaluations is faster than 0.19.0, and every number is unchanged (#717).
The dashboard sends compressed files and API responses (#718).
The public claims schema

Fixed

evaluate_output with an expected_trajectory no longer fails with IRIS_INTERNAL_ERROR.
A comparison no longer counts a rule that did not run as a pass.
A comparison's words match its numbers.
A comparison no longer calls runs "equivalent" within a margin nobody chose.
A critical rule firing on new cases is named in the comparison.
compare_traces no longer prints a 95% interval of [100%, 100%].
"Could not have detected a change smaller than X" now means what it says.
The dashboard no longer draws a rule that never ran as "100% pass".
The run comparison page shows what each run was judged on.
Several server processes on one Iris home run the same rules and lose no rule.
A label written through one server process is read by the others.
The retention sweep no longer deletes the pinned baseline run.
The HTTP endpoint serves more than one MCP client.
The site and the docs say what the code does, and a test holds each of these statements to the thing it describes.
The compare pages hold against the vendors' pages as of 2026-10-01.
Closing the store releases the file before it returns (#750).
A search's page of trace ids, and the start's check for evaluations with no stored risk estimate, keep their plans when the file has statistics (#742).
The website builds only when something its build reads has changed.
A database file is never opened by both copies of SQLite in one process (#739).
Scoring a trace no longer reads every evaluation in the store (#711).
Every hot read keeps its query plan, whatever SQLite's statistics say (#711).
The dashboard's first render reads index entries, not trace rows (#711).
get_traces with min_score or max_score no longer walks every evaluation once per trace (#711).
Case results for one case key or one session start from that case's traces (#711).
On Node's built-in node:sqlite, the first read after another process changed the schema returns every column (#711).
Rebuilding the search index no longer holds the server while the old index is erased (#695).
A write no longer waits for SQLite to copy the whole write-ahead log into the database file, and the log stays short.
A failing test in the truthbase capture is reported as a failing test, by name, not as drift.
The truthbase capture refuses a run that exited non-zero with every test passing.
The Python client's publish workflow parses again, and every workflow is linted on every pull request.
o1-mini is priced at $1.10 in and $4.40 out per million tokens, not $3 and $12.
On Node 24, Iris no longer loads a better-sqlite3 that would abort the server when it frees a statement (#719).
A stdio server shuts down in order when the client closes its stdin.
A published precision's interval is the one for the prevalence asked (#717).
--self-test no longer says PASS on a database the server then refuses (#704).
A trace stored by a process that does not index it is found by the next search.
A start refused over the database prints its sentence on its own line.
Every test file is type-checked in CI (#721).
The published test counts are no longer recorded from a run in which a test file failed to load.
The Python recorder batches as it says it does.
The Failures and Moments pages no longer re-run the regression-alarm watcher once per trace (#680).
The first Failures or Moments request after a start no longer spends seconds drawing alarm lines.
A busy host no longer gets ordinary custom regex rules killed as backtracking (#677).
LangGraph traces from LangSmith's OpenTelemetry export arrive readable.
A tool call whose arguments arrive as a JSON string is checked as the object it encodes.
The LangGraph recipe works as written.
The LangChain and CrewAI examples are complete programs, not scaffolds.
The published list of judge models is the list the judge accepts.
answers_the_ask has blocked at the shipped defaults since 0.18.0, contrary to its release notes; it now advises there, as documented.
The published composite numbers move, because they were measured with the rule blocking.
The response schema accepts a judge's sample evidence
The evaluator matrix reads a measured proof/judge-results.json.
A fresh clone builds with pnpm install && pnpm run build again.
npm run demo and the release and tool-list checks no longer start Iris on your own data.
The dashboard works from the keyboard and reads correctly to a screen reader, and CI now checks it on every page in both themes (#724).
Dashboard text reaches WCAG AA contrast (4.5:1) in both themes (#725).

Supply-chain transparency

SBOMs: iris-npm-sbom.spdx.json + iris-docker-sbom.spdx.json (attached below). Both are SPDX 2.3 JSON, cover direct + transitive dependencies.

SBOM signatures: each SBOM has a companion .sigstore.json (Sigstore bundle — signature + cert + Rekor entry) attached to this release. The release workflow runs this exact verification itself before it reports success. Verify with:

cosign verify-blob \
  --bundle iris-npm-sbom.spdx.json.sigstore.json \
  --certificate-identity='https://github.com/iris-eval/mcp-server/.github/workflows/release.yml@refs/tags/v0.20.0' \
  --certificate-oidc-issuer='https://token.actions.githubusercontent.com' \
  iris-npm-sbom.spdx.json

npm provenance: published with --provenance (verifiable via npm audit signatures or on the package page). Any npm library or launcher version this workflow publishes beside the server carries provenance too.

Docker signature: image signed with cosign keyless (Sigstore). Verify with:

cosign verify ghcr.io/iris-eval/mcp-server:v0.20.0 \
  --certificate-identity='https://github.com/iris-eval/mcp-server/.github/workflows/release.yml@refs/tags/v0.20.0' \
  --certificate-oidc-issuer='https://token.actions.githubusercontent.com'

Build attestation: both the npm SBOM and Docker image manifest carry GitHub-signed build-provenance attestations. Inspect with gh attestation verify or cosign verify-attestation.

MCPB bundle: iris-eval.mcpb (open it to install in Claude Desktop) is built from the npm tarball above plus the production dependencies in this tag's package-lock.json, and holds the tarball's files byte for byte. It is signed (iris-eval.mcpb.sigstore.json) and carries a build-provenance attestation. Verify with:

cosign verify-blob \
  --bundle iris-eval.mcpb.sigstore.json \
  --certificate-identity='https://github.com/iris-eval/mcp-server/.github/workflows/release.yml@refs/tags/v0.20.0' \
  --certificate-oidc-issuer='https://token.actions.githubusercontent.com' \
  iris-eval.mcpb
gh attestation verify iris-eval.mcpb -R iris-eval/mcp-server

What's Changed

feat(npm): the unscoped iris-eval package launches the server instead of printing a notice by @irparent in #640
docs: rate-limit keys, context-anchored PII patterns, and the README env table held to the code; dead roadmap links fixed by @irparent in #641
docs(langchain): the package note links its rebuild (#642) instead of calling its future undecided by @irparent in #643
fix(npm): the iris-eval launcher is checked out with LF on every platform, so its shebang runs on macOS and Linux by @irparent in #644
chore(npm): record the iris-eval launcher as published by @irparent in #645
deps: Bump tsx from 4.23.13 to 4.23.15 by @dependabot[bot] in #631
deps(website): Bump framer-motion from 13.4.0 to 13.4.1 in /website by @dependabot[bot] in #634
deps(dashboard): Bump @typescript-eslint/eslint-plugin from 8.61.0 to 8.70.1 in /dashboard by @dependabot[bot] in #628
deps(website): Bump @radix-ui/react-visually-hidden from 1.2.4 to 1.2.11 in /website by @dependabot[bot] in #632
deps: Bump fast-check from 4.10.1 to 4.10.2 by @dependabot[bot] in #629
feat(judge): price Claude Opus 5.5; the published judge-model list is read from the pricing table by @irparent in #646
deps: Bump @types/node from 26.6.1 to 26.6.2 in the types group across 1 directory by @dependabot[bot] in #626
deps(dashboard): Bump @types/d3-shape from 3.1.8 to 3.2.0 in /dashboard in the types group across 1 directory by @dependabot[bot] in #622
deps(website): Bump @upstash/redis from 1.38.4 to 1.39.0 in /website by @dependabot[bot] in #627
verify_citations: report judge failures under judge_error, not resolve_error by @irparent in #652
deps(website): Bump the nextjs group across 1 directory with 2 updates by @dependabot[bot] in #624
Decision Moments: rank by significance within a stated window by @irparent in #653
deps(website): Bump @tailwindcss/postcss from 4.3.0 to 4.3.3 in /website by @dependabot[bot] in #630
fix: shipped thresholds read as shipped (answers_the_ask advises at the defaults, as documented) by @irparent in #660
Dashboard: load each page's code when the page is shown by @irparent in #664
deps(dashboard): Bump react and @types/react in /dashboard by @dependabot[bot] in #633
Record every OpenAI and Anthropic call as a GenAI span, in Python and JavaScript by @irparent in #659
An off-topic answer fails when the relevance judge is on (#649) by @irparent in #656
dashboard: drop recharts and the dead components that imported it by @irparent in #666
feat(release): an MCPB bundle with each release — one-click install in Claude Desktop (#650) by @irparent in #655
chore(deps): lock the MCPB reference tooling as an exact devDependency by @irparent in #676
feat(traces): full-text search over traces (#7) by @irparent in #658
Failures and Moments: run the regression-alarm watcher once per request by @irparent in #684
tests: fix the intermittent root-suite failures from a busy machine by @irparent in #678
Fix two recorder test failures on main: the Python batching interval, and a JS wall-clock assertion by @irparent in #686
tests: run half as many workers, and give the jsdom suite a measured timeout by @irparent in #685
Regex sandbox: count the budget from the start of the match by @irparent in #679
Regression alarms: read the alarm lines from a generated table, not a first-request simulation by @irparent in #687
tests: remove the timing assumptions a busy machine breaks by @irparent in #681
Install succeeds when better-sqlite3 cannot build: it is optional, and Iris runs on node:sqlite (#661) by @irparent in #673
Moments: a filtered list pages within a stated window by @irparent in #674
e2e: choose the port at run time and check the server is this build by @irparent in #675
LangChain and LangGraph runs scored, in Python and JavaScript by @irparent in #663
Search: read what a trace's spans say — later model calls, tool results, exceptions (#683) by @irparent in #688
The Python end-to-end servers cannot outlive the test run by @irparent in #694
GenAI message arrays are read down to their words, for every framework by @irparent in #692
An agent span (invoke_agent, create_agent) is INTERNAL, not a model call by @irparent in #693
fix(claims): capture-tests refuses a run where a test file failed to load by @irparent in #697
tests: the alarm-line table's spread check reads three rows with a measured timeout by @irparent in #699
fix(build): a fresh clone builds with pnpm install && pnpm run build; CI builds that way and requires every tool by @irparent in #647
Search: find words inside Chinese, Japanese and Korean text, and tokenize exactly as the index does (#682) by @irparent in #698
Estimate a trace's cost from its tokens when it reports none, and mark it estimated (#702) by @irparent in #705
Release supply chain: npm provenance for the SDK, handler and launcher; hash-pinned PyPI build; sessions end with their key; image home in /data by @irparent in #706
Search: bound what one query can cost; delete_trace leaves no text on disk (#703) by @irparent in #712
Search runs on a worker thread with its own read-only connection (#703) by @irparent in #716
CI: lint every workflow with actionlint; publish-python.yml parses again and builds on every pull request by @irparent in #728
Truthbase capture: a failing test is refused by name; claims:check names the fields that drifted by @irparent in #730
tests: search worker and budget tests prove their point without racing the clock by @irparent in #731
Refuse a better-sqlite3 build that aborts Node 24 when it frees a statement (#719) by @irparent in #720
Startup never waits on the store: retention after connect, in steps; #695 erased in steps; checkpoints off the event loop by @irparent in #727
deps: ip-address 10.7.2 and undici 7.30.0; the citation SSRF guard refuses NAT64 to private networks by @irparent in #736
Relevance judge: daily budget, per-request cap, redaction, honest annotations, loud misconfiguration, MCPB settings by @irparent in #707
perf: evaluate_output back to 0.19.0 speed, evaluation reads never run the draws, identical numbers; compressed dashboard by @irparent in #715
Hot reads keep their plans, statements compile once, synchronous is explicit, dashboard reads use index entries (#711) by @irparent in #726
Upgrades: copy the database before migrating, a compatibility floor, install --upgrade, and a self-test that reads the real file (#704) by @irparent in #710
perf(dashboard): compress the answers to reads only; a write's acknowledgement goes out as it is by @irparent in #738
Background steps sized to stay under 50 ms; FTS5 merges out of the index build's steps; the covering index built after the start by @irparent in #737
website: build only when something the site's build reads has changed by @irparent in #740
Revoking an API key takes effect on the next request, with no restart by @irparent in #714
Export traces and evaluations as CSV or JSON Lines, with the list's filters and search (#4) by @irparent in #709
Checkpoints: the write-ahead log starts over again; background steps empty it off the event loop; the risk fill's steps are sized with their commit by @irparent in #741
Dashboard: keyboard and screen-reader fixes, and an axe check on every page in both themes by @irparent in #713
Stall guard: each case copies to a file of its own, so a retry never reads the failed attempt's log by @irparent in #747
Migration 019's indexes are built after the server answers, and health says when they are building (#744) by @irparent in #745
Plan test covers the export reads; a search's page ids and the risk-estimate fill keep their plans under statistics (#742) by @irparent in #743
deps: fast-uri 3.1.8 and brace-expansion 5.0.12 / 1.1.21 in every workspace by @irparent in #753
Tests: every temporary directory is removed with retries, so Windows does not fail a test on a file still being released by @irparent in #749
Stall guard: while writes never pause, assert that the log keeps starting over, and say how when it fails by @irparent in #755
Search budget test: require one answered and one refused query, not more than 5 answered by @irparent in #752
Dashboard: text reaches WCAG AA contrast (4.5:1) in both themes by @irparent in #735
Closing the store releases the file before it returns by @irparent in #751
Truthbase capture: refuse a run that exited non-zero with every test passing by @irparent in #754
Checkpointer tests prove the log's thresholds at a fifth of their size by @irparent in #757
Upgrades: the server answers first, then copies and migrates on the checkpoint worker; every door waits at one gate by @irparent in #748
Price 1-hour cache writes at Anthropic's 1-hour price (#702) by @irparent in #723
Type-check every test file in CI, and fix the 108 errors found on main (#721) by @irparent in #722
Tests: storage.synchronous's store test gets 30 s, like the other file-backed storage tests by @irparent in #760
Tests: the risk-fill checkpoint test gets 90 s; it runs its fill twice by @irparent in #762
Queue a single stored trace for the search index and index it in batches after the write (#729) by @irparent in #734
OTel recipes for the OpenAI Agents SDK (Python and JavaScript) and LlamaIndex, run in CI by @irparent in #708
Erasure test: say which page of iris.db holds a swept word, and whether it is free (#758) by @irparent in #759
Dashboard: no per-request compression for a client on this machine (#761) by @irparent in #763
One copy of SQLite per database file in a process, and prove worker threads end cleanly (#739) by @irparent in #733
CI: the worker-exit job gets its own workflow, run where it can find something by @irparent in #765
deps(website): Bump framer-motion from 13.4.1 to 13.4.4 in /website by @dependabot[bot] in #700
deps(dashboard): Bump react-router from 8.3.0 to 8.4.0 in /dashboard by @dependabot[bot] in #672
deps(dashboard): Bump @storybook/addon-docs from 10.5.6 to 10.6.0 in /dashboard by @dependabot[bot] in #690
deps(dashboard): Bump @storybook/react-vite from 10.4.6 to 10.6.0 in /dashboard by @dependabot[bot] in #671
deps(dashboard): Bump jsdom from 29.1.1 to 30.1.1 in /dashboard by @dependabot[bot] in #669
ci: Bump the codeql group across 1 directory with 4 updates by @dependabot[bot] in #691
deps: Bump @modelcontextprotocol/sdk from 1.30.0 to 1.30.1 by @dependabot[bot] in #689
deps: Bump the testing group across 1 directory with 2 updates by @dependabot[bot] in #701
deps: Bump the linting group across 1 directory with 4 updates by @dependabot[bot] in #654
Drop an unused import; keep the SDK's Node types at its Node 22 floor by @irparent in #764
Start the storage threads from fixed code by @irparent in #775
deps: Bump @types/node from 26.6.2 to 26.6.3 in the types group across 1 directory by @dependabot[bot] in #766
Compare pages: every vendor statement re-read on 2026-10-01 by @irparent in #776
deps(website): Bump @types/node from 26.6.2 to 26.6.3 in /website in the types group across 1 directory by @dependabot[bot] in #767
deps(langchain): Bump @langchain/core from 1.2.12 to 1.2.13 in /packages/langchain in the langchain group across 1 directory by @dependabot[bot] in #769
deps(dashboard): Bump vite from 8.3.0 to 8.3.1 in /dashboard in the vite group across 1 directory by @dependabot[bot] in #770
deps(dashboard): Bump @testing-library/react from 16.3.2 to 16.3.3 in /dashboard by @dependabot[bot] in #772
deps(dashboard): Bump @testing-library/jest-dom from 6.9.1 to 7.0.1 in /dashboard by @dependabot[bot] in #771
test: the search-budget cases get 30 s by @irparent in #779
deps(website): Bump sharp from 0.35.4 to 0.35.5 in /website by @dependabot[bot] in #768
deps(dashboard): Bump @testing-library/dom from 10.4.1 to 10.4.2 in /dashboard by @dependabot[bot] in #773
docs(changelog): 0.20.0 release notes by @irparent in #778
Publish the four-way comparison by @irparent in #777
Tidy a comment in the release workflow test by @irparent in #781
The package inventory leaves out manifests git ignores; the node-forge advisory is recorded by @irparent in #782
The phrase rules read text both ways; every invisible character is dropped by @irparent in #783
A failing verdict names every layer that would have failed it; a gate passes only what it judged by @irparent in #784
Site and docs statements are tested against the code they describe by @irparent in #785
deps(sdk): Bump the providers group across 1 directory with 2 updates by @dependabot[bot] in #780
A trace's verdict cannot be replaced by the agent it judges by @irparent in #786
One edition: open source, free, every feature included by @irparent in #787
Several processes and several clients share one data folder by @irparent in #788
A comparison counts only what was checked, and its words match its numbers by @irparent in #789
Not checked is its own answer: in the verdict, the gate, the counts and the dashboard by @irparent in #790
Detectors read what was written: a test runner's verdict, a recovered failure, and the same text spaced or wrapped another way by @irparent in #791
An output written as JSON is read by what it says by @irparent in #792
A long run of full stops no longer holds the event loop by @irparent in #793
A verdict is stamped with the calibration table it was labelled from by @irparent in #795
An answer that says nothing fails: a new rule, says_something by @irparent in #796
The Claude Code capture plugin records failed calls and loses no call a turn made by @irparent in #797
Every verdict says who recorded the evidence it judged; a capture harness can certify that no tool was called by @irparent in #798
The session reader: Claude Code's own session logs, read as turns and judged by @irparent in #799
Community links: GitHub, npm and X by @irparent in #800
The site and release notes say what Iris is: one edition, every feature included by @irparent in #801
The site states Iris's price as it is, with one commitment beside it by @irparent in #802
The site says what exists today and leaves room for what comes next by @irparent in #803
The capture plugin sends each turn in the form the release it pins reads by @irparent in #807
A release publishes what the workflow can, its page carries the notes whole, and the 0.20.0 notes lead with what can fail a build by @irparent in #808
release: 0.20.0, with the Python client at 0.2.0 by @irparent in #809

Full Changelog: v0.19.0...v0.20.0

Navigation Menu

Repository navigation

v0.20.0

Choose a tag to compare

Sorry, something went wrong.