feat: Phase 2 - Data Layer (Dataset Loaders & Validation)

[himanshu231204]

Summary

Implement the complete Data Layer for OpenAgent Eval (Phase 2), providing dataset loading and validation capabilities for JSON, JSONL, CSV, and HuggingFace formats.

Motivation

Phase 2 is the next step after the Phase 1 Foundation. The data layer is essential for the evaluation pipeline to load and validate datasets before evaluation.

Files Modified

New Files

• openagent_eval/datasets/base.py - BaseDatasetLoader abstract interface, Dataset and DatasetItem dataclasses
• openagent_eval/datasets/models.py - Pydantic validation models (DatasetItemModel, DatasetModel)
• openagent_eval/datasets/json_loader.py - JSON format dataset loader
• openagent_eval/datasets/jsonl_loader.py - JSONL (JSON Lines) format dataset loader
• openagent_eval/datasets/csv_loader.py - CSV format dataset loader
• openagent_eval/datasets/hf_loader.py - HuggingFace dataset loader (optional dependency)
• ests/unit/test_datasets/test_base.py - Tests for base models (16 tests)
• ests/unit/test_datasets/test_models.py - Tests for Pydantic models (10 tests)
• ests/unit/test_datasets/test_json_loader.py - Tests for JSON loader (11 tests)
• ests/unit/test_datasets/test_jsonl_loader.py - Tests for JSONL loader (10 tests)
• ests/unit/test_datasets/test_csv_loader.py - Tests for CSV loader (11 tests)
• ests/unit/test_datasets/test_hf_loader.py - Tests for HuggingFace loader (13 tests)

Modified Files

• openagent_eval/datasets/init.py - Updated with public API exports

Architectural Decisions

• BaseDatasetLoader is an abstract base class that all loaders implement
• Dataset and DatasetItem are dataclasses (not Pydantic) for lightweight usage
• DatasetItemModel and DatasetModel are Pydantic models for validation
• HuggingFace loader is optional (requires datasets package)
• All loaders use the existing exception hierarchy (DatasetNotFoundError, InvalidDatasetError, DatasetValidationError)
• Field mapping in HF loader handles common HuggingFace dataset field name variations

Breaking Changes

None - this is purely additive.

Testing

• 71 new unit tests added (all passing)
• 121 total tests passing (50 from Phase 1 + 71 new)
• All external dependencies mocked (no real API calls)
• Tests cover success paths, error paths, and edge cases

Remaining TODOs

• Phase 3: Metrics System
• Phase 4: Reports
• Phase 5: Providers
• Phase 6: Plugin System

Assumptions & Limitations

• HuggingFace loader requires optional datasets package (pip install openagent-eval[datasets])
• CSV loader expects UTF-8 encoding
• JSONL loader skips blank lines gracefully

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.