You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
OpenAIConnectorbaseURL for OpenAI-compatible APIs (Groq,
Together, the Hugging Face router, vLLM, LM Studio, Ollama's /v1
endpoint). With a custom baseURL the key is never read from OPENAI_API_KEY, so an OpenAI key is not sent to another provider; pass apiKey, or omit it for local servers that need none.
validatePlugin(stage, plugin, { registry }) returns { valid, errors, warnings } for a plugin without throwing, for use in
plugin tests. registry: true also applies the rules pluginRegistry.register() enforces.
ParallelEmbeddermode ('auto', 'batch', 'single').
checkAnswer({ question, answer, sources }) checks an answer from any
framework (LangChain, LlamaIndex, Mastra, the AI SDK, your own code)
against its sources: grounded, groundedness, the unsupported
sentences, per-sentence citations and evaluation scores. Sources may be
strings, { id, content }, LangChain Documents or LlamaIndex nodes; normalizeSources is exported for the conversion alone.
Caching: createRagPipeline({ cache }) caches query embeddings,
retrieval results and answers (true, a store, or per layer with time to
live). MemoryCache (LRU and TTL) and RedisCache (node-redis or
ioredis) are included; any get/set store works. A layer caches only
when its component identifies itself (getCacheKey(role) or cacheKey;
all built-ins do), retrieval is keyed by the store's state and the query
vector, answers by the exact documents and generation options, and
answers are not cached when the temperature is above 0. Retrieval is
cached before filterResults and the reranker, so guardrail ACLs run on
every request. Nothing from a failed, aborted or timed-out run, or from an
unfinished stream, is stored. Results carry cache per layer and the
pipeline has cacheStats().
@devilsdev/rag-pipeline-utils/testing: deterministic, offline test
helpers. HashEmbedder, EchoLLM (answers with the top document or a set
answer, records every call, streams), fixtures.documents and createTestPipeline(). Shares classes with the main entry and is not
loaded by it.
computeSemanticSimilarity(answer, reference, { embedder }): cosine
similarity of the two embeddings, 0 to 1. checkAnswer adds it as evaluation.scores.semanticSimilarity when given a referenceAnswer and
an embedder.
rag-pipeline query options: --trace (time per stage), --explain
(passage scores, and the passages supporting each answer sentence, with
unsupported sentences flagged), --stats (latency, estimated tokens and
cost), --streaming and --timeout <ms>; global --quiet. --json
includes trace, stats and explanation when asked.
createAiSdkAdapter({ ai, embeddingModel, languageModel }) wraps
Vercel AI SDK models as an embedder and an llm for createRagPipeline(), with streaming, embedMany batching and the run's AbortSignal. You pass the AI SDK functions, so it is not a dependency and
works with AI SDK 5, 6 and 7 from ESM or CommonJS. The RAG system prompt
is sent as both instructions (AI SDK 7) and system (5 and 6). Tested
against AI SDK 6 and 7 with their mock models.
Changed
Faster CLI. Commands load only their own dependencies: --help
takes about 0.5 s and query about 0.5 s instead of about 3.6 s. Loading
the CLI drops from 1281 modules and 93 packages to 55 and 13. The HTML,
Markdown and CSV loaders load jsdom, marked and csv-parse when they
read a file.
Faster import.require('@devilsdev/rag-pipeline-utils') takes about
125 ms instead of about 515 ms and loads 31 modules from 9 packages
instead of 245 from 54 (published bundle, Node 22). axios, jsonwebtoken, ajv and chokidar now load when the feature that needs
them is first used (the Anthropic, Cohere and Ollama connectors, JWTValidator, plugin registration and config validation, hot reload).
The shared pluginRegistry loads its contracts on first use; registries
created with new PluginRegistry() still load them in the constructor.
Fixed
ExecutionTracer.exportTraces('json') threw "circular structure"
with the default captureOutputs: true: traces stored the live result,
which carries the trace. Traces now store JSON snapshots, which also keeps
later changes to a result out of its recorded trace.
ParallelEmbedder failed with the built-in connectors, which embed
one string at a time, because it always called embed(batch) with an
array. In the default 'auto' mode it now embeds texts one by one for
those connectors and uses embedBatch(texts) when an embedder has it;
other embedders are still called with the array.