Repository navigation
Releases
Releases · aiexponent/rag-benchmarking
v1.0.2
What's Changed
Enterprise hardening, namespace isolation, unmasked CI quality gates, and community governance.
🚀 Highlights
- Namespace Isolation (PRD-173): Relocated
src/appandsrc/harnessunder dedicatedsrc/rag_benchmarking/namespace to eliminate globalsite-packagespollution. Provided canonical public entrypoints atrag_benchmarkingandrag_benchmarking.sdk(#24). - Unmasked CI Quality & Security Gates (PRD-174):
- Removed suppression
|| truefrompython -m mypy srcin CI, resolved all typing errors acrosssrc/, and enabled dynamic multi-version Python matrix testing (3.11 and 3.12). - Enforced blocking
pip-auditsecurity audits withoutcontinue-on-error: true, establishing audited waivers for transitive, unfixed advisories (#25). - Enforced strict license gating (
sys.exit(1)) under the permissive policy in LCC CI workflow.
- Removed suppression
- Community Health & Governance (PRD-175):
- Upgraded
CODE_OF_CONDUCT.mdto full Contributor Covenant 2.1 with explicit 48-hour response SLA and designated channels (conduct@aiexponent.com,security@aiexponent.com). - Overhauled
CONTRIBUTING.mdwith a 10-minute low-code evaluation dataset contribution track (data/golden/qa.jsonl), modern verification commands, and conventional commits. - Added
make lintmulti-gate verification target (#26).
- Upgraded
- Dynamic Version Resolution (#23): Derived
__version__from package distribution metadata, asserted version equality in health tests. - Ecosystem URL Migration (#22): Updated shipped package metadata, README, and repository links to direct
aiexponentorganization URLs. - Brand Refresh (#19, #20, #21): Migrated to AiExponent Caret identity, refreshed OpenGraph assets with EU AI Act status token, and purged retired split-A assets.
Full Changelog: v1.0.1...v1.0.2
What's Changed
- feat(brand): refresh to caret identity (2026-06-11 rebrand) by @apundhir in #19
- chore(brand): OG card v2 — install line + truthful status token by @apundhir in #20
- chore(brand): remove retired split-A assets by @apundhir in #21
- chore(urls): point shipped URLs at the aiexponent org by @apundhir in #22
- fix(app): derive version from installed package metadata by @apundhir in #23
- refactor(pkg): isolate namespace under rag_benchmarking to eliminate site-packages pollution (PRD-173) by @apundhir in #24
- fix(ci): unmask mypy type checking, enforce pip-audit and dogfood LCC (PRD-174) by @apundhir in #25
- docs(community): replace stub Code of Conduct and modernize Contributing guide (PRD-175) by @apundhir in #26
- docs(changelog): document 1.0.2 release and reconcile version drift (PRD-176) by @apundhir in #27
Full Changelog: v1.0.1...v1.0.2
v1.0.1
What's Changed
Same-day cleanup release closing the three non-blocking items from the v1.0.0 sign-off round (#18).
⚠️ Breaking Change
- /v1/evaluate/agent response payload key renamed
scores→metrics: Brings the agent endpoint into shape-parity with/v1/evaluate(which has always emittedmetrics). If your code readsr["scores"], update it tor["metrics"].
Changed
- SDK docstring (G6-A):
RagEval.evaluate()documents canonicalground_truths: list[str]plural-list shape, preserving backward compatibility forground_truth=. - RAG Pipeline Judge Default (G6-B): Flipped fallback
LLMClient.gemini_modelfromgemini-1.5-flashtogemini-2.5-flash.
Full Changelog: v1.0.0...v1.0.1
What's Changed
Full Changelog: v1.0.0...v1.0.1
v1.0.0
What's Changed
• release: v1.0.0 — Path A reframing · schema reconciliation · authenticity pass by @apundhir in #17
Full Changelog: https://github.com/aiexponent/rag-benchmarking/commits/v1.0.0
rag-benchmarking:v0.2.0-rc1
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog,
and this project adheres to Semantic Versioning.
[0.2.0-rc1] - 2026-02-07
Added
- Authentication: Implemented API Key middleware. Clients must now provide
X-API-Keyheader. - Observability: Added correlation IDs (
X-Trace-Id) to requests and logs. - Structured Logging: Enhanced
RAGEnginelogs with event-specific metadata (retrieval counts, loop metrics). - Deployment Guide: Comprehensive deployment documentation.
Changed
- Refactor: Extracted RAG logic from
api/query.pyintosrc/app/engine/rag_engine.py. - Error Handling: Replaced generic 500 errors with specific status codes (503 for Service Unavailable).
- Settings: Externalized prompt templates to environment variables/settings.
Security
- Added
API_KEYvalidation for/v1/queryand/v1/evaluateendpoints.
[0.1.0] - 2026-01-30
Added
- Initial POC release.
- Retrieval-Augmented Generation pipeline using Qdrant and Gemini/OpenAI.
- RAGAS evaluation suite.
- Docker Compose setup.