v0.4.4

Bug Fix\n\n- Added individual JSON object parsing for malformed responses\n- 4-strategy fallback for JSON parsing\n- Handles escaped characters and unescaped quotes in LLM responses\n\nFull Changelog: v0.4.3...v0.4.4

v0.4.3

Bug Fix\n\n- Simplified JSON parsing with 3-strategy fallback\n- Strategy 1: Clean and parse JSON\n- Strategy 2: Regex extraction of Q&A pairs\n- Strategy 3: Individual field extraction\n- Handles malformed LLM responses gracefully\n\nFull Changelog: v0.4.2...v0.4.3

v0.4.2

Bug Fix\n\n- Add regex fallback for JSON parsing when LLM returns malformed JSON\n- Improved control character handling in JSON responses\n- Better error handling with graceful degradation\n\nFull Changelog: v0.4.1...v0.4.2

v0.4.1

Bug Fix\n\n- Improve JSON parsing resilience in synthesis module\n- Handle control characters and unescaped newlines in LLM responses\n\nFull Changelog: v0.4.0...v0.4.1

v0.4.0

What's Changed

Bug Fixes

• Improve JSON parsing resilience in synthesis module (_parse_response)
• Handle malformed JSON from LLM (trailing commas, extra text, single quotes)

New Features

• Added CI/CD integration module for automated evaluation in pipelines
• Added CLI test command for running evaluations
• Added corpus and related modules example notebook

Tests

• Added 10 new tests for JSON parsing edge cases
• All 31 synthesis tests pass

Full Changelog: v0.3.0...v0.4.0

v0.3.0 - Production-Grade Features

What's Changed

• feat: Phase 9 - Corpus Health Auditor (THE DIFFERENTIATOR) by @himanshu231204 in #26
• feat: Phase 10 - LLM-as-Judge Metrics (NLI-based scoring) by @himanshu231204 in #27
• feat: Phase 11 — Component Diagnosis by @himanshu231204 in #28
• feat: Phase 12 — Synthetic Test Data Generator by @himanshu231204 in #29

Full Changelog: v0.2.0...v0.3.0

v0.2.0

Release v0.2.0

Features:
- Comprehensive CLI UX improvements (global flags, dry-run, shell completion, JSON output)
- Enhanced validate, delete, list, init, and doctor commands

Bug Fixes:
- Comprehensive bug hunt (Critical through Low severity)
- Dataset path config and undeclared provider dependencies
- Hallucination test fixture deepeval module replacement
- CLI ANSI code stripping in tests and --no-interactive flag

Documentation:
- Provider guides for all LLM, retriever, and embedder providers
- Examples section with RAG tutorial guide
- CLI improvements documentation

v0.1.1

Full Changelog: v0.1.0...v0.1.1

v0.1.0

What's Changed

• docs: Add mandatory Git workflow rules by @himanshu231204 in #1
• fix: Correct all dates from 2025 to 2026 by @himanshu231204 in #2
• docs: add open source community files by @himanshu231204 in #3
• feat: Phase 2 - Data Layer (Dataset Loaders & Validation) by @himanshu231204 in #4
• feat: Phase 3 - Metrics System by @himanshu231204 in #5
• Feature/phase 4 reports by @himanshu231204 in #6
• Feature: Phase 5 - Provider Layer with LLM and Retriever Adapters by @himanshu231204 in #7
• Feature: Phase 6 - Plugin System by @himanshu231204 in #9
• Refactor/cli modular structure by @himanshu231204 in #11
• docs: Phase 8 - Documentation by @himanshu231204 in #12
• feat: Add production-grade GitHub Actions CI/CD pipeline by @himanshu231204 in #13
• feat: add PDF dataset loader (PDFDatasetLoader) by @himanshu231204 in #14
• feat: wire full evaluation pipeline (retriever → LLM → metrics) by @himanshu231204 in #15
• Potential fix for code scanning alert no. 1: Jinja2 templating with autoescape=False by @himanshu231204 in #17
• feat: add retriever providers + embedder abstraction by @himanshu231204 in #16
• Fix HTML report template guidance, numeric metric aggregation, and output path handling by @himanshu231204 with @Copilot in #18
• Fix for Statement has no effect by @himanshu231204 in #19

New Contributors

• @himanshu231204 made their first contribution in #1
• @himanshu231204 with @Copilot made their first contribution in #18

Full Changelog: https://github.com/OpenAgentHQ/openagent-eval/commits/v0.1.0