v3.9.0

Answer checks now catch an answer that drops its source's condition, and are much faster on long answers.

npm install @devilsdev/rag-pipeline-utils@^3.9.0
# and, if you use the MCP server:
npx @devilsdev/rag-mcp-server@3.9.0 ./docs

If you install @devilsdev/rag-core, rag-guardrails, rag-connectors or rag-eval directly, upgrade them to 3.9.0 as well.

Highlights

• A condition left out is caught. "Remote work is allowed two days per week" is no longer grounded by "Remote work is allowed two days per week with manager approval", and "Access is allowed" no longer by "Access is allowed, but not for guests". The check reports condition_mismatch; turn it off with conditionCheck: false or --no-condition-check.
• Text is read as typed. Text from PDFs and web pages (zero-width characters, soft hyphens, full-width digits, no-break spaces) is normalised before it is compared, sentences end where a reader would end them ("Dr. Lee" and "10 a.m." don't end one; a list item does), and citation markers ([1], ¹) no longer fail a correct sentence.
• Long answers are checked in linear time. A 1,000-sentence answer that took about 9 s is checked in under half a second, and the MCP server counts its input limits the way the check does.
• Stable scores across source sets. A duplicated or unrelated source no longer changes a verdict, and computeFaithfulness, groundedness and PipelineEvaluator use the same word weights as checkAnswer.
• Fixes across the library: the chunkers keep all their text, FileVectorStore scores are true cosine similarities and merge on save, streams end with one complete done chunk and honour timeout, CostTracker checks the budget before a run, DAG node timeouts and concurrency take effect, and AuditLogger gains close() and a working webhook.

After upgrading

Some changes need action:

• Re-tune minRelevanceScore for FileVectorStore. Its scores are now cosine similarity (1 for an identical vector); they used to grow with the query vector's length.
• Re-ingest to pick up the chunker fixes: the recursive (default), fixed-size and sentence chunkers no longer drop or split text, so chunks can change.
• Raise timeout if you stream long answers. A run's timeout now covers a returned stream until it ends.
• Regenerate saved baselines that compare multi-source scores or hallucination classes. Scores move for answers checked against several sources, and hallucination classes change in many cases while rates and verdicts don't.

Benchmarks

On the condition-dropped benchmark, F1 on the test split is 85.9% lexical and 90.1% with a Claude judge on the released code. On the public RAGTruth and HaluEval sets, F1 moved by at most 0.01: the stricter condition check adds a few false alarms on RAGTruth, and HaluEval has three more misses. All numbers, including those that got worse, are on the Benchmarks page.

Known gaps

The condition check is deliberately narrow. A negated contrast with no scope word ("but not guests", "but guests are not admitted") is not yet read as a restriction, and neither is a restriction attached to a different subject ("Staff may enter" against "Guests may enter, but staff need approval"). 3.10 plans restrictions that follow their subject, which covers these cases.

Thanks to the readers whose reports shaped this release.

The scoped packages, @devilsdev/rag-mcp-server and create-rag-pipeline are released at 3.9.0 too. Full details: CHANGELOG · Blog post