Skip to content

Releases

v1.0.2

Choose a tag to compare

@apundhir apundhir released this 23 Sep 15:52
9c3f5bf

What's Changed

Enterprise hardening, namespace isolation, unmasked CI quality gates, and community governance.

🚀 Highlights

  • Namespace Isolation (PRD-173): Relocated src/app and src/harness under dedicated src/rag_benchmarking/ namespace to eliminate global site-packages pollution. Provided canonical public entrypoints at rag_benchmarking and rag_benchmarking.sdk (#24).
  • Unmasked CI Quality & Security Gates (PRD-174):
    • Removed suppression || true from python -m mypy src in CI, resolved all typing errors across src/, and enabled dynamic multi-version Python matrix testing (3.11 and 3.12).
    • Enforced blocking pip-audit security audits without continue-on-error: true, establishing audited waivers for transitive, unfixed advisories (#25).
    • Enforced strict license gating (sys.exit(1)) under the permissive policy in LCC CI workflow.
  • Community Health & Governance (PRD-175):
    • Upgraded CODE_OF_CONDUCT.md to full Contributor Covenant 2.1 with explicit 48-hour response SLA and designated channels (conduct@aiexponent.com, security@aiexponent.com).
    • Overhauled CONTRIBUTING.md with a 10-minute low-code evaluation dataset contribution track (data/golden/qa.jsonl), modern verification commands, and conventional commits.
    • Added make lint multi-gate verification target (#26).
  • Dynamic Version Resolution (#23): Derived __version__ from package distribution metadata, asserted version equality in health tests.
  • Ecosystem URL Migration (#22): Updated shipped package metadata, README, and repository links to direct aiexponent organization URLs.
  • Brand Refresh (#19, #20, #21): Migrated to AiExponent Caret identity, refreshed OpenGraph assets with EU AI Act status token, and purged retired split-A assets.

Full Changelog: v1.0.1...v1.0.2

What's Changed

  • feat(brand): refresh to caret identity (2026-06-11 rebrand) by @apundhir in #19
  • chore(brand): OG card v2 — install line + truthful status token by @apundhir in #20
  • chore(brand): remove retired split-A assets by @apundhir in #21
  • chore(urls): point shipped URLs at the aiexponent org by @apundhir in #22
  • fix(app): derive version from installed package metadata by @apundhir in #23
  • refactor(pkg): isolate namespace under rag_benchmarking to eliminate site-packages pollution (PRD-173) by @apundhir in #24
  • fix(ci): unmask mypy type checking, enforce pip-audit and dogfood LCC (PRD-174) by @apundhir in #25
  • docs(community): replace stub Code of Conduct and modernize Contributing guide (PRD-175) by @apundhir in #26
  • docs(changelog): document 1.0.2 release and reconcile version drift (PRD-176) by @apundhir in #27

Full Changelog: v1.0.1...v1.0.2

v1.0.1

Choose a tag to compare

@apundhir apundhir released this 23 Sep 05:30
8cfcb93

What's Changed

Same-day cleanup release closing the three non-blocking items from the v1.0.0 sign-off round (#18).

⚠️ Breaking Change

  • /v1/evaluate/agent response payload key renamed scores → metrics: Brings the agent endpoint into shape-parity with /v1/evaluate (which has always emitted metrics). If your code reads r["scores"], update it to r["metrics"].

Changed

  • SDK docstring (G6-A): RagEval.evaluate() documents canonical ground_truths: list[str] plural-list shape, preserving backward compatibility for ground_truth=.
  • RAG Pipeline Judge Default (G6-B): Flipped fallback LLMClient.gemini_model from gemini-1.5-flash to gemini-2.5-flash.

Full Changelog: v1.0.0...v1.0.1

What's Changed

  • release: v1.0.1 — close the three non-blocking items from the v1.0.0 sign-off by @apundhir in #18

Full Changelog: v1.0.0...v1.0.1

v1.0.0

Choose a tag to compare

@github-actions github-actions released this 10 May 18:13
1d1a8d9

What's Changed

• release: v1.0.0 — Path A reframing · schema reconciliation · authenticity pass by @apundhir in #17

Full Changelog: https://github.com/aiexponent/rag-benchmarking/commits/v1.0.0

rag-benchmarking:v0.2.0-rc1

Choose a tag to compare

@apundhir apundhir released this 07 Feb 13:28

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.

[0.2.0-rc1] - 2026-02-07

Added

  • Authentication: Implemented API Key middleware. Clients must now provide X-API-Key header.
  • Observability: Added correlation IDs (X-Trace-Id) to requests and logs.
  • Structured Logging: Enhanced RAGEngine logs with event-specific metadata (retrieval counts, loop metrics).
  • Deployment Guide: Comprehensive deployment documentation.

Changed

  • Refactor: Extracted RAG logic from api/query.py into src/app/engine/rag_engine.py.
  • Error Handling: Replaced generic 500 errors with specific status codes (503 for Service Unavailable).
  • Settings: Externalized prompt templates to environment variables/settings.

Security

  • Added API_KEY validation for /v1/query and /v1/evaluate endpoints.

[0.1.0] - 2026-01-30

Added

  • Initial POC release.
  • Retrieval-Augmented Generation pipeline using Qdrant and Gemini/OpenAI.
  • RAGAS evaluation suite.
  • Docker Compose setup.