Skip to content

0.15.0-beta — Azure AI Foundry integration

Pre-release
Pre-release

Choose a tag to compare

@joslat joslat released this 08 Jul 20:58
· 167 commits to main since this release

Azure AI Foundry integration — live hybrid evals with source chips and graceful bypass

Samples 12 and 13 now connect to a real Azure AI Foundry project (\AZURE_FOUNDRY_ENDPOINT) and run Foundry's built-in evaluators (task adherence, relevance) side-by-side with AgentEval-local metrics — one agent run, one source-tagged HTML report, one per-source console summary.

Highlights

  • *\AIConfig.FoundryEndpoint* — reads \AZURE_FOUNDRY_ENDPOINT; \Uri.TryCreate\ so a malformed value returns
    ull\ rather than throwing
  • *\TracingAgentEvaluator* — Foundry-oriented diagnostic wrapper printing [Foundry ▶]\ / [Foundry ✔]\ / [Foundry ⚠]\ / \ERROR\ / \SKIPPED\ with per-item detail
  • Graceful bypass for microsoft/agent-framework#6991 — \FoundryEvals\ sends the now-rejected \�zure_ai_evaluator\ criteria type; the bypass detects HTTP 400 and returns a structured \skipped\ result so local AgentEval scores survive unaffected
  • Sample 12 & 13 — per-source/per-component console breakdown (☁/⚙ icons, scores, Foundry portal URLs)
  • Source chips in HTML reports — ☁ Foundry (blue) / ⚙ AgentEval (green) on \hybrid.\ and \ oundry.\ keys

Bug fixes

  • \MeaiToEvalResultBridge: fix mixed-scale normalisation (0–1 vs 1–5 for Foundry metrics) + epsilon comparison for floating-point safety near 1.0
  • \UnifiedEvalReport: empty-string \Error\ no longer treated as a failure; single-query hierarchy flattened
  • \AgentEvaluatorEvalLeaf: \SkippedLeaf\ for graceful bypass; \ReportUrl\ in evidence; \IsNullOrEmpty\ fix
  • \WeightedSumAggregation: both \skipped\ and \�rror\ labels excluded from weighted scoring
  • \HybridEvalInterop.SkippedResults: sets \Status\ for reliable detection
  • \HtmlEvalResultRenderer: restore !isSkipped\ guard on score-inline span; \OrdinalIgnoreCase\ for prefix checks

Tests

21 new unit tests covering scale normalisation, epsilon boundary, skipped-leaf behaviour, report URL evidence, and error-neutral aggregation.

See CHANGELOG.md for the full entry.