Skip to content

v2.4.4 — Trust-signal release: benchmarks, stability policy, comparison, demo

Choose a tag to compare

View all tags
@alikahwaji alikahwaji released this 19 Apr 02:08
· 179 commits to main since this release
v2.4.4
73cd5f5

v2.4.4 — Trust-signal release: benchmarks, stability policy, comparison, demo

Docs and examples only. Zero changes to src/. Closes the entire "this-month" repositioning review with five additive deliverables.

📊 BENCHMARKS.md (root, ships in tarball)

Methodology-first benchmarks doc. Deliberately does not publish a headline "N ops/sec" number. RAG performance depends on provider, hardware, corpus, and queries; one number hides all of that and ages badly.

What you get:

  • Philosophy + what we measure (framework overhead) vs. don't (E2E provider-bound)
  • Three run presets: npm run benchmark:quick / :full / default (already in package.json)
  • JSON report schema + environment disclosure template
  • End-to-end harness template for honestly benchmarking your own stack
  • Interpretation guide (percentiles, throughput, memory, common pitfalls)
  • CI regression pattern + community contribution flow

📐 SEMVER.md (root, ships in tarball)

Public API stability contract. Defines:

  • Public surface = entry-point exports + CLI + plugin contracts in contracts/
  • What counts as breaking (and what doesn't)
  • Deprecation process: minimum one-minor warning window before removal
  • Supported runtime: Node.js ≥ 18 today, ≥ 20 planned for next major
  • TypeScript and ESM/CJS commitments
  • Pairs with the Supported Versions matrix in SECURITY.md

You can now auto-merge patch updates with confidence and plan major upgrades from a published contract.

🔐 Security Capabilities matrix

New docs page: Security Capabilities

Honest scorecard distinguishing:

  • 🟢 Implemented — battle-tested, has tests, in production
  • 🟡 Recommended — primitives + docs ship, you wire it
  • 🟠 Example — reference code, audit before production
  • 🔴 Not in scope — explicitly out of scope

Covers pre-retrieval / retrieval-time / post-generation guardrails plus supply-chain, auth, logging, input validation, and data governance. Use it for vendor-evaluation checklists and threat-model reviews.

🧩 Concepts: rag-core / rag-connectors / rag-eval / rag-guardrails

Four new docs pages map the framework to four conceptual modules without splitting the codebase:

Reduces perceived blast radius for security review without code splits. A real monorepo split is on the v3.0 roadmap.

⚖️ Comparison page

vs LangChain.js / LlamaIndex.TS — honest decision framework. Includes explicit "where each one wins" sections, a migration friction table, and explicit non-goals. Currency dated to April 2026 so future readers can recalibrate.

🚀 Runnable demo: examples/fastify-rag-demo

Minimal end-to-end RAG service in ~150 lines of server.js:

  • Fastify 5 + OpenAI + in-memory retriever + 3-layer guardrails
  • POST /query returns { answer, citations, groundednessScore, evaluation, retrievedCount }
  • POST /ingest, GET /health
  • CORS + per-IP rate limiting
  • Dockerfile + Fly.io fly.toml + Railway railway.json — deploy in one command
  • scripts/smoke.js for a 30-second end-to-end test
cd examples/fastify-rag-demo
cp .env.example .env  # set OPENAI_API_KEY
npm install && npm start
node scripts/smoke.js  # ingest + query + assert citations

Deploy:

fly launch && fly secrets set OPENAI_API_KEY=sk-... && fly deploy
# or
railway up

📦 Tarball

  • 542 KB · 225 files
  • Six root docs ship: README, LICENSE, CHANGELOG, SECURITY, BENCHMARKS, SEMVER

Migration

No API changes. Patch-safe upgrade per SEMVER policy.

Full changelog

CHANGELOG.md → 2.4.4

Install

npm install @devilsdev/rag-pipeline-utils@2.4.4

npm: https://www.npmjs.com/package/@devilsdev/rag-pipeline-utils/v/2.4.4

This release closes the entire "this-month" review (items #9, #11, #12, #13, #14). Items #15 (monorepo split) and #16 (rename decision) remain v3.0 candidates.

Assets 2
Loading