Skip to content

Brand Alignment: Evaluation-First Naming

Pre-release
Pre-release

Choose a tag to compare

View all tags
@joslat joslat released this 25 Jan 21:59
· 321 commits to main since this release
v0.3.0-beta
1c62f0e

[0.3.0-beta] - 2026-01-25

Brand Alignment: Evaluation-First Naming 🎯

This release implements comprehensive renamed APIs to better reflect AgentEval's primary identity as an AI Agent Evaluation Toolkit. All "Test" terminology in public APIs has been renamed to "Evaluation" to align with the framework's positioning.

⚠️ BREAKING CHANGES

Interface Renames

Old Name New Name
ITestHarness IEvaluationHarness
IStreamingTestHarness IStreamingEvaluationHarness
ITestableAgent IEvaluableAgent
IWorkflowTestableAgent IWorkflowEvaluableAgent

Class Renames

Old Name New Name
MAFTestHarness MAFEvaluationHarness
WorkflowTestHarness WorkflowEvaluationHarness
TestOptions EvaluationOptions
TestOutputWriter EvaluationOutputWriter
TestMetadata EvaluationMetadata

Method Renames

Old Name New Name
RunTestAsync() RunEvaluationAsync()
RunTestStreamingAsync() RunEvaluationStreamingAsync()
RunTestSuiteAsync() RunEvaluationSuiteAsync()
TestHarnessFactory property EvaluationHarnessFactory property

File Renames

Old Name New Name
ITestHarness.cs IEvaluationHarness.cs
ITestableAgent.cs IEvaluableAgent.cs
MAFTestHarness.cs MAFEvaluationHarness.cs
WorkflowTestHarness.cs WorkflowEvaluationHarness.cs
TestModels.cs EvaluationModels.cs
TestOutputWriter.cs EvaluationOutputWriter.cs
stochastic-testing.md stochastic-evaluation.md
Sample14_StochasticTesting.cs Sample14_StochasticEvaluation.cs

Unchanged (Universal Terminology)

The following names are intentionally kept as they represent universal industry terminology:

  • TestCase - Standard testing terminology used across all frameworks
  • TestResult - Conflict resolution with existing Core.EvaluationResult type
  • TestSummary - Consistent with TestResult
  • AgentEvalTestBase - xUnit integration base class
  • StochasticRunner - Neutral name, not test-specific
  • *Tests.cs files - xUnit naming convention

Changed

  • Terminology: "stochastic testing" → "stochastic evaluation" throughout codebase and documentation
  • Terminology: "test harness" → "evaluation harness" throughout codebase and documentation
  • XML Documentation: Updated all public API comments with evaluation-first language
  • C# Naming Conventions: Fixed parameter names to use camelCase (evaluationOptions instead of EvaluationOptions)
  • Documentation: Title case capitalization fixes in markdown headers
  • Documentation: Fixed all broken links to stochastic-testing.md (now stochastic-evaluation.md)
  • TOC: API Reference section now renders consistently with other menu items
Assets 2
Loading