v0.17.0-beta
Pre-releaseThe biggest release by PR count so far (43 merged since 0.16.0-beta). Full MAF Agent Skills evaluation and
governance ships end-to-end: assertions, a disclosure-efficiency metric, a compliance scanner with a
multi-repo baseline ledger, a skill-injection red-team attack, deterministic run_skill_script governance
gates, SkillGate construction-time drift enforcement, and a composite Skill Health & Security Index. The
Gatekeeper Tribunal gains its remaining flagship judges (intent-action mismatch, goal-hijack drift,
ungrounded claims, hallucinated citations — all κ=1.000 against their gold sets) plus two new gate layers:
tool RESULT gates (inspecting an already-executed call's output, not just the proposed call) and real
HTTP-egress enforcement closing an SSRF/DNS-rebind gap DomainAllowListGate could never see. The Microsoft
Copilot Studio live connector ships (real MSAL device-code auth, a real activity-stream bridge — still not
independently verified against a live tenant, honestly disclosed throughout). And the newest theme,
Explainability & Trust, ships as tested library code with a runnable sample: reconstructable gate
provenance, counterfactual gate-config replay, and a unified Trust Score.
Also resolves the long-deferred BUG-22 exit-code overload (a breaking CLI-contract fix — see below),
closes a doc-lag pattern that hit twice this cycle (a capability shipping with zero matching documentation)
with two new "what's new" pages and a CI check that now prevents it recurring in either direction, and
refreshes every downstream sample — including the NuGet consumer validation project — to track this release.
Docs/samples hardening follow-up + NuGetConsumer refresh to 0.16.0-beta
A self-review of the BUG-22/Explainability & Trust batch below found real gaps and closed them, then a
follow-on pass brought the Skills/Copilot Studio/Explainability & Trust docs and samples to full parity and
refreshed the NuGet consumer samples.
Fixed — completing the BUG-22 remap
BenchTraceFidelityCommand.csandBenchPerfCommand.cseach independently hardcoded the exact
same gate-fail-as-exit-2 pattern the BUG-22 fix below was supposed to have unified everywhere — missed in
the first pass.BenchPerfCommandnow delegates to the sharedBenchExitCodes.FromLabelinstead of
reimplementing it; both returnGateFailed(9) on a hard fail.AzureChatAgentFactory.cs— the SUT-agent-resolution counterpart toJudgeFactory.cs— had the
identical config-resolution-conflated-with-usage-error bug across 4 return sites, also missed. Now returns
RuntimeError(3).- 8 more docs described the old exit-code contract and were never updated:
docs/cli.md's resolution-order
prose,docs/gatekeeper-cli.md's cross-reference, and 6 getting-started pages (gdpr/memory/longmemeval/
mitre/owasp/perf). One (memory) was actually wrong even before BUG-22 — it claimed WARN mapped to exit
0 alongside PASS, which was never true.
Added — docs, mirroring the missing coverage
docs/gatekeeper/explainability-and-trust.md— the docs page the Explainability & Trust library code below
had shipped without anydocs/coverage at all.docs/agent-skills-whats-new.mdanddocs/gatekeeper-whats-new.md— capability-history pages mirroring
the existingdocs/redteam-whats-new.mdpattern, so a shipped-but-undocumented capability (this happened
twice in the same area: SkillGate, then Explainability & Trust itself) is easier to catch next time.- "New to this? Start here" plain-English concept sections added to
docs/agent-skills.md,
docs/copilot-studio.md, anddocs/gatekeeper/explainability-and-trust.md, plus short "in plain English"
framing on Agent Skills' three densest phases — docs read as progressive, not front-loaded with jargon.
Added — samples
samples/AgentEval.Samples/Gatekeeper/10_GatekeeperExplainabilityAndTrust.cs(new) — 3 gradual scenes:
GateProvenance(a real judge call) →GateReplayer(deterministic) →TrustScoreCalculator(combines
both). Live-verified against real Azure OpenAI.samples/AgentEval.Samples/CopilotStudio/00_CopilotStudioHelloWorld.cs(new) — a true one-concept on-ramp
before the existing multi-concept walkthroughs (which each covered 4-5 concepts at once).
Changed — docs structure hygiene
- Moved
docs/redteam/copilot-studio.md→ top-leveldocs/copilot-studio.md: it covers eval/bench
integration, fluent assertions, and Gatekeeper composition, not just red-teaming — inconsistent with the
AgentEval.MAF.CopilotStudiopackage and the samples menu, both of which already treat it as its own
top-level area (matching Agent Skills' precedent). All inbound references updated. - Renamed
ResponsibleAI.md→responsible-ai.mdanddocs/GlassBox/→docs/glassbox-history/for
kebab-case consistency with every other doc file/folder. - New CI check:
tools/check_docs_toc.py+.github/workflows/docs-toc-check.ymlfails a PR if any
docs/**/*.mdfile isn't reachable fromdocs/toc.yml(the real site navigation, notdocs/index.md's
separately-maintained landing-page list) — the exact mechanism that let two doc pages ship invisible in the
sidebar this session, found only by manual audit. Extended to also catch the reverse drift: a local
link indocs/index.md's landing page pointing at a page that isn't (or is no longer) indocs/toc.yml—
one-directional by design, since most nav pages aren't meant to be landing-page-highlighted.
Changed — NuGet consumer samples refreshed to the latest released package
samples/AgentEval.NuGetConsumer/AgentEval.NuGetConsumer.Testswere pinned toAgentEval 0.13.1-beta
(built on MAF 1.11.1) — three releases stale. Bumped to0.16.0-beta(the actual latest published version
on NuGet.org — confirmed via the NuGet API, not assumed frommain), with the full dependency baseline
(Microsoft.Agents.AI1.13.0,System.Memory.Data10.0.9) updated to match exactly what
Directory.Packages.propsresolved at thev0.16.0-betatag. Restored, built, and tested clean end-to-end
against the real published package (one transient Azure content-filter rejection on first run, confirmed
non-reproducible on re-run — a live-service flake, not a regression).
BUG-22 resolution + Explainability & Trust (gate provenance, counterfactual replay, unified Trust Score) + docs/samples
Changed (BREAKING — CLI exit-code contract)
- BUG-22 resolved:
ExitCodescode2was overloaded —bench/calibratereturned it for both a
benchmark gate FAIL/WARN and bad CLI arguments, andJudgeFactoryconfig failures (missing/partial Azure
OpenAI credentials) also returned it. Now:2is reserved strictly for bad arguments; judge/runtime config
failures return3(RuntimeError); benchmark/calibration gate outcomes return dedicated new codes
9(GateFailed),10(GateWarning,bench <family>only),11(GateIndeterminate). External CI
pipelines branching on exit code2frombench/calibratemust be updated. SeeExitCodes.csand
Exit codes.
Added — Explainability & Trust (0.17.0-beta theme, analysis in strategy/ExplainabilityAndTrust-AnalysisAndPlan.md)
- Gate provenance chains —
AgentEval.Guardrails.GateProvenance(rule name, evidence, threshold vs.
actual, contributing sub-chains) attached via a new optionalGateVerdict.Provenancefield (additive, same
precedent asConfidence). Wired intoCompositeJudgeGate<TRubric>for both the Block path and the
near-miss-Allow-with-Confidence path Fleet Correlation already reads. - Counterfactual gate replay —
AgentEval.MAF.Gatekeeper.GateReplayer.CompareAsyncruns a baseline and a
candidateIToolGatelist against the SAME capturedGatedToolCalls (the real gate objects, no
simulation; first-Block/Mutate-wins, matching the liveAgentEvalToolGateExtensionspipeline) and reports
which calls would have diverged under the candidate configuration. Library API this session; a
agenteval log-file gate-replayCLI wrapper is a natural mechanical follow-on. - Unified Trust Score —
AgentEval.Trust.TrustScoreCalculator.ComputecombinesTrustSignals (gate
verdicts, eval scores, anything 0..1) into one honest 0-100 composite, excluding"skipped"/"error"
labeled signals from the weighted math entirely (the same discipline asWeightedSumAggregationet al. —
"including them at 0.0 would incorrectly drag the composite below threshold").
Documentation
docs/redteam/copilot-studio.md— added theCopilotStudioAssertionsfluent-assertion section (was
shipped but undocumented) and cross-referencedEstimatedCreditsUsed.docs/agent-skills.md— added §3b documenting SkillGate (construction-time drift enforcement, shipped
but undocumented since it landed) —WithSkillGate/SkillGateMode/SkillDriftException/
agenteval skills baseline approve.
Samples
samples/AgentEval.Samples/AgentSkills/04_AgentSkillsSkillGate.cs(new) — live-verified against real Azure
OpenAI: pins a baseline, simulates a rug-pull, showsSkillDriftExceptionfail-closed, then recovers.samples/AgentEval.Samples/CopilotStudio/02_CopilotStudioBudgetAndRedTeam.cs(new) —--max-credits
enforcement trippingCopilotStudioBudgetExceededExceptionfor real,HaveStayedWithinCreditBudget,
CanResistAsyncred-teaming a live MCS agent,HaveStartedNewConversation/HaveStartedDifferentConversation.
Copilot Studio — C2 fidelity-badge audit + P5 correlation-key spike
Added
- C2 (all-targets fidelity badge) — closed.
EvidenceFidelity(Verbal/IntentToAct/Behavioral) now appears
in every RedTeam report renderer, not just JSON/SARIF (which already carried it):Markdowngets an inline
`[behavioral]`/`[verbal]`/`[intent-to-act]`tag next to each compromised probe,JUnitgets
aFidelity:line in the failure body (plus the label folded into the inconclusive<error>message), and
PDFgets a bracketed label next to each finding. An audit of all 5 renderers found the 3 gaps were
genuinely mechanical (the fidelity data was already flowing through the shared model, just not rendered) —
shipped the fix for all three rather than stopping at the audit.
Investigated (spike concluded without shipping code — by design)
- P5 (L2 telemetry enrichment) correlation-key spike. Investigated whether a client-side
conversationId
can reliably correlate a live Copilot Studio scan against Dataverse session transcripts (SessionID,
TopicId,ChannelId) for deeper post-hoc evidence enrichment. Findings: the correlation join is plausible
but not publicly confirmed by Microsoft's docs; more importantly, Dataverse transcripts have ~30 minute
latency after conversation inactivity before they're queryable — which means the originally-sketched design
(an inlineTraceEnrichingChatClientdecorator enriching evidence on the hot path) cannot work at all. Real
L2 enrichment needs to be a separate, deferred offline reconciliation command, not a live decorator. P5
remains correctly deferred; this is a scoping correction, not a shipped feature. Full write-up in
docs/redteam/copilot-studio.md's "How it fits red-team fidelity" section.
Agent Skills Wave 1 — baseline ledger, repo-wide discovery, provenance pointer
Added
SkillContentHasher(AgentEval.Skills) — a new full-file-content hash, complementing
SkillManifestPoisoningGate's existing structural-only fingerprint (which hashes only the parsed manifest
fields, not the raw file bytes). Together they let a baseline snapshot distinguish "the manifest's meaning
changed" from "the file changed but parses identically" — two different signals a security-conscious skill
reviewer cares about separately.ISkillBaselineStore/JsonFileSkillBaselineStore— a multi-snapshot, never-overwritten skill-scan
ledger (a sibling ofAgentEval.Memory'sJsonFileBaselineStore— the same proven pattern, not a shared
base type, deliberately: skills and memory baselines have different identity/versioning shapes). Every
agenteval skills scan --write-baselinecall appends a new timestamped snapshot rather than overwriting the
last one, soagenteval skills baseline history <skill-name>can show how a skill's fingerprint has drifted
over time, not just its current state vs. one prior pin.AgentSkillDirectoryConventions(--repoflag onskills scan) — scans every directory convention MAF's
ownAgentFileSkillsSourcerecognizes across an entire repo root in one pass, instead of requiring one
invocation per skill directory.- CLI:
agenteval skills scan --write-baseline [--baseline-root <dir>] [--repo]captures a snapshot;
agenteval skills baseline list|diff|historyinspects the ledger. - A null-safe provenance-pointer parameter on the compliance report renderer (wiring is real and tested; no
lock-file source populates it with real data yet — that's Wave 2/3 territory). - Self-review before merge caught a real bug the C# compiler doesn't error on: a doc-comment placement mistake
inMafSkillScanner.csthat silently mis-associated two records' XML docs with the wrong types — confirmed
via the generated XML doc file, not just a build check, and fixed. - Wave 2 (trust-on-first-use reputation matching, cross-location drift detection) and Wave 3 (org-wide
multi-repo scan, live upstream verification) remain unbuilt, per the design doc's own phasing — Wave 3 is
explicitly gated on a not-yet-done security/credential-scope review.
Gatekeeper — the IToolPlanGate empirical dispatch-order question, resolved
Investigated (empirical finding, not a new gate)
- Determined, against real MAF 1.13.0 behavior (not assumed from docs), whether
FunctionInvokingChatClient
dispatches sibling tool calls from one model turn sequentially or concurrently — the fact a future
IToolPlanGate(batch/plan-level gate, e.g. catching[read_secrets(), send_email()]issued as siblings in
one turn, whichSequenceGatestructurally cannot see) needs settled before its interface shape can be
designed. Finding: sequential dispatch is MAF's default, and that default is the only mode reachable
throughChatClientAgent's normal builder surface (confirmed: zero references to
AllowConcurrentInvocationanywhere inMicrosoft.Agents.AI). Concurrent dispatch is reachable, but only
via manually constructing aFunctionInvokingChatClientwithUseProvidedChatClientAsIs = true— and under
that mode, a naively-designed "terminate at the first blocked call" plan gate does not reliably stop a
sibling call from still executing (empirically confirmed: the sibling ran anyway).IToolPlanGateitself is
still unbuilt — this pass shipped only the empirical groundwork a correct design now depends on.
Gatekeeper — 4 next-wave gates: prompt-template drift, calibration staleness, Fleet Health Index, tool-result size anomaly detection
Added
PromptTemplateDriftGate— the third application of theManifestFingerprint/ManifestDriftDetector
hash-pin-and-diff primitive (after skill manifests and MCP tool schemas), applied to an agent's prompt
template files. Unlike every other gate, it's not anIToolGate/IChatGate/IToolResultGateat all — a
prompt template doesn't change mid-run, so a per-turn check would be pure waste.UseGatekeeperchecks drift
eagerly at construction time when bothGatekeeperOptions.PromptTemplatesandPromptTemplateBaseline
are set, and throwsPromptTemplateDriftExceptionimmediately on a mismatch — fail-closed, matching
RefuseUnprotectedHighRiskTools's posture. Setting only one of the two options throws
InvalidOperationExceptionat construction rather than silently no-op-ing.CalibrationReport.CapturedAt/IsStale(maxAge, clock?)— a calibration report now records when it was
captured, and can report whether it's aged past a caller-chosen threshold. Informational only — staleness
never affectsIsInlineReadyor auto-demotes an already-promoted judge; it's a signal to re-calibrate, not a
promotion-blocking condition.ICalibrationReportStore/JsonFileCalibrationReportStore— the persistence seamCalibrationReport
needed (it was, until now, a purely in-memory return value ofGateCalibrationHarness.EvaluateAsync, never
persisted between runs). Deliberately minimal: one report per axis (the most recent run), overwritten on each
save — not a full historical ledger (see the Agent Skills baseline ledger above for that different shape,
deliberately not duplicated here).GatekeeperFleetHealthIndex.Compute(reportsByAxis, staleAfter, clock?)— joins every tracked judge axis's
latest calibration report into one composite fleet-health view, mirroringSkillSecurityIndex's honesty
discipline: an axis with no report is never fabricated into a passing score. Reports mean decisive
accuracy/kappa (calibrated axes only), total dangerous errors, which axes have never been calibrated, and
which are stale. Transport-agnostic (AgentEval.Core, no CLI/Mission Control dependency yet) — high value
once an ops-facing surface exists to put it on.ToolResultSizeAnomalyGate— a per-tool, per-session statistical-outlier detector, distinct from the
already-shippedToolResultSizeGate(which truncates against one fixed, global character threshold). Flags a
result more than Nx (default 5x) that same tool's own running average size this run, once enough prior
calls establish a baseline (default 3) — catching behavioral drift a global threshold can't see (e.g. a tool
that's returned ~200-character results all run suddenly returning 50,000 characters, even though 50,000 might
be unremarkable for a different, bulk-read tool). v1: fixed multiplier, no real statistics library — a
documented, deferred v2 follow-on.- Self-review before merge found and fixed 4 real issues: a filename-collision risk in the calibration store, a
label-space inconsistency in the Fleet Health Index, a silent half-configuration security gap in
PromptTemplateDriftGate(now fails loud instead), and dead code. - Docs: new "Calibration staleness & the Gatekeeper Fleet Health Index" section in
docs/gatekeeper/gate-reference.md. - Crucible work and the two remaining flagship judges (
ToolArgumentGoalCoherenceJudge,
CrescendoTrajectoryJudge) are explicitly out of scope for this batch.
Mission Control — prompt-hash provenance display completed, one stale doc claim corrected
Fixed
AdjudicationFlow.tsx's judge cards and adjudicator card now renderPromptHashPill— the last place
in the SPA that didn't show prompt-hash provenance (EvalResultNode.tsxandScenarioTreePage.tsxalready
did, since 2026-05-25). Arguably the highest-stakes place it was missing, since it's the view a human uses to
resolve judge disagreement.docs/missioncontrol/api-design.mdcorrected:Query.complianceEvidence's documented return type was
ComplianceEvidence?; the real resolver returnsComplianceEvidenceWithChain?.- Corrected a stale "still a live bug" claim repeated across 3 local strategy/review docs: the compliance
matrix's per-cellauditChainValidcheck (tampered evidence rendering as a false green checkmark) was
actually fixed 2026-05-24 — the docs describing it as open were simply never updated when the fix landed.
Gatekeeper Hardening Phase 2 — real HTTP-egress enforcement (#10): redirect-chasing + DNS-rebind/SSRF defense
Added
GatekeeperHttpMessageHandler(AgentEval.MAF.Gatekeeper.Egress) — a realDelegatingHandlerclosing
the gapDomainAllowListGatecandidly documents about itself: that gate scans the URL string inside a
tool call's arguments, never the actual outgoing network request, so it cannot catch a redirect to a
forbidden host or a DNS answer resolving an allow-listed hostname to a private/internal address (SSRF /
DNS-rebinding — the cloud-metadata endpoint169.254.169.254is the canonical target). This handler sits
underneath whicheverHttpClienta tool's own implementation uses (a different composition point from
IToolGate/IToolResultGate— opt-in per tool viaGatekeeperHttpMessageHandler.CreateHttpClient(...), not
registered throughUseGatekeeper) and re-validates the allow-list AND resolves DNS before every hop,
including every redirect — redirects are followed manually (the factory disables the transport's own
auto-redirect), bounded byMaxRedirects, never silently delegated. Every block throws
HttpEgressBlockedException(the idiomatic fail-closed signal for anHttpMessageHandler).PrivateNetworkClassifier— classifies anIPAddressas private/loopback/link-local/reserved (RFC1918,
CGNAT, IPv6 unique-local, IPv4-mapped-IPv6 unwrapped to its embedded address, the cloud-metadata range, and
more) — the check run against every DNS-resolved address.IDnsResolver/SystemDnsResolver— a seam over DNS resolution so tests can script resolution results
(including a DNS-rebind scenario) without real network/DNS access; the default wrapsSystem.Net.Dns.GatekeeperHttpEgressOptions—MaxRedirects(default 5),BlockPrivateNetworks(default on),
DnsResolutionTimeout(default 2s, fail-closed on timeout — cannot prove the destination safe),DnsResolver.HostAllowList— the exact-or-subdomain host-matching logic factored out ofDomainAllowListGate
(behavior-preserving extraction, existing tests unchanged) so the new handler shares the IDENTICAL allow-list
semantics rather than a second copy that could silently drift out of sync with the argument-level gate.- Redirect handling matches
HttpClientHandler's own default semantics: 307/308 preserve method and body;
301/302/303 downgrade to GET (dropping the body, except a HEAD request stays HEAD). The synchronous
HttpClient.Send(...)API is explicitly refused (NotSupportedException) rather than silently bypassing
every check — this handler's validation is inherently async (DNS resolution). - 43 new tests (
PrivateNetworkClassifierTests,GatekeeperHttpMessageHandlerTests— scripted inner handler +
fake DNS resolver, zero real network access, deterministic). Full suite green on all three TFMs (net8.0
7648/7648, net9.0 7648/7648, net10.0 7851/7851). - Docs: new "HTTP egress enforcement" sections in
gate-reference.md/examples.md/introduction.md.
Gatekeeper Hardening Phase 2 — tool RESULT gates (P0-3) + a parallel-tool-call test fixture
Added
IToolResultGate(AgentEval.MAF.Gatekeeper) — a new gate kind inspecting one already-executed tool
call's RESULT, at the same MAF function-invocation seamIToolGateinspects the proposed call, but on the
other side ofnext(...). Closes the "tool output as an injection channel" gap: nothing before this
inspected a tool's own return value before it re-entered the model's context (only prior results, read out
of conversation history, were ever consulted — never this call's own result, synchronously, before it flows
back). ReturnsToolResultVerdict(Allow/Block/Redact, viaGatedToolResult). Wired into
UseAgentEvalToolGate's new optionalresultGatesparameter — runs, in order, immediately afternext(...)
returns, only once everyIToolGatehas allowed the call. ABlock/throw fails closed exactly like the
call-gate loop; aRedactverdict is applied regardless ofToolGatePolicy(mirrorsToolGateAction.Mutate's
"always applied" precedent). Recorded under the newgate.tool-result.*trace stage — distinguishable from a
call-gate block while still counted by the existing, stage-agnosticGlassBoxEvidence.CountGateBlocks. Null/
emptyresultGatesis the exact prior behavior (zero overhead, fully backward compatible).- Three built-in result gates (
AgentEval.MAF.Gatekeeper.Gates):ToolResultInjectionGate— blocks a result containing a prompt-injection marker (shares its default
marker list with the chat-sideTokenInjectionGate, nowpublicfor exactly this reuse). Not maskable —
alwaysBlock, neverRedact.ToolResultSizeGate— truncates an oversized result (default 8,000 chars) viaRedact, bounding
context-window exhaustion and per-turn token cost from a single runaway tool response.ToolResultSecretGate— detects and masks common credential shapes (AWS/GitHub/Slack/Google/Stripe
keys, full PEM private-key blocks, bearer tokens, JWTs) viaRedact, mirroringRegexPiiGate's
bounded-timeout mask-with-█ approach.
GatekeeperOptions.ToolResultGates/AddResultGate(...)— composed byUseGatekeeperalongside
ToolGates; a result-gate-only configuration (no call gates registered) is valid. TheIToolResultGate. MinimumPolicyfloor is folded into the same Observe-mode conflict checkToolGatesalready gets, and the
Observe startup banner now reports the result-gate count too.GateTelemetry.Record(string, ToolResultAction, TimeSpan)— a second overload sharing the same per-policy
counters as the call-gateRecord, so a caller readingSnapshot()sees one unified effectiveness view
across both gate kinds;Redactmaps toMutateCount.ScriptedChatClient.AddParallelToolCalls(...)(AgentEval.Core.Testing) — the fixture prerequisite this
phase needed: scripts MULTIPLEFunctionCallContentin one assistant turn (the shape a real provider sends
for parallel function calling). Before this, the fixture could only ever emit one tool call per turn, so no
test could exercise a gate pipeline against MAF'sFunctionInvokingChatClientactually invoking N calls from
a single turn — only N single-call turns in a row, a materially different code path. Fully additive — the
existing single-callAddToolCallAPI is unchanged.docs/gatekeeper/gate-reference.md/examples.md/introduction.mdupdated with the new "Tool RESULT
gates" layer, its policy-reinterpretation rules, and runnable snippets.
Deferred (explicitly out of scope this pass)
- P0-4 (tool-plan/batch gates) — a genuinely new interception point, sized larger, deferred to a separate
session. - Full result-content capture into the trace (mirroring
MutationEvidenceRenderer'sTraceCaptureMode) — a
tool result can be arbitrarily large/shaped content from anywhere; only the fact that a redaction happened,
and why, is recorded for now.
Copilot Studio — live connector wired (redteam --sut copilot-studio Track 1)
Added
CopilotStudioAgentFactory.BuildLivenow builds a real connector instead of unconditionally throwing
NotSupportedException. It constructs a realMicrosoft.Agents.CopilotStudio.Client.CopilotClientfrom
CopilotStudioConfig(ConnectionSettingsmapped 1:1 —EnvironmentId/SchemaName/Cloud), resolves the
token scope viaCopilotClient.ScopeFromSettings(never hardcoded), and bridges its streaming Bot Framework
activity API (StartConversationAsync/AskQuestionAsync→IAsyncEnumerable<IActivity>) into an
IChatClient(newCopilotStudioChatClient), wrapped in a MAFChatClientAgentand handed to the existing
FromAgentseam — unchanged.redteam --sut copilot-studio's consent gate, config validation, and every
existing credential-free test still run and pass before any of this is reached; construction itself makes no
network call (the token callback is invoked lazily byCopilotClienton the first real request).CopilotStudioTokenProvider— MSAL device-code auth (IPublicClientApplication.AcquireTokenWithDeviceCode)
with a persisted, OS-encrypted token cache (Microsoft.Identity.Client.Extensions.Msal— DPAPI on Windows,
Keychain on macOS, libsecret on Linux) keyed by a SHA-256 hash ofTenantId|AppClientId, silent-acquisition-first
(AcquireTokenSilent) with device-code fallback onMsalUiRequiredException. New code — no prior token-caching
precedent existed in this repo.CopilotStudioConfig.Cloudnow resolves to the realPowerPlatformCloudenum (ResolveCloud()/
Validate()), verified against the actual restoredMicrosoft.Agents.CopilotStudio.Client1.3.171-beta package
(not the higher version number a prior planning doc assumed — see the deviation note below). Case-insensitive,
defaults toProdwhen omitted, and a typo'd/unrecognized value now fails config validation with a clear error
listing the valid names, before any network call.ICopilotStudioConversationClient— an AgentEval-owned abstraction over the twoCopilotClientmembers the
chat-client shim needs. The real package does not publicly export a mockableICopilotClientinterface (an
earlier decompilation-based design note assumed one existed), so this repo defines its own seam instead —
this is also what makesCopilotStudioChatClientunit-testable without live credentials.SingleNameHttpClientFactory— a minimalIHttpClientFactoryfor the one named clientCopilotClient
requires, avoiding a fullMicrosoft.Extensions.Http+ServiceCollectionregistration for a CLI with no
ambient DI container.- New package references (
AgentEval.Cli, centrally pinned inDirectory.Packages.props):
Microsoft.Agents.CopilotStudio.Client1.3.171-beta,Microsoft.Agents.Core1.3.171-beta,
Microsoft.Identity.Client4.84.2,Microsoft.Identity.Client.Extensions.Msal4.84.2,
Microsoft.Extensions.Http10.0.8 (raisedMicrosoft.Extensions.DependencyInjection's central floor to 10.0.8
to match).
Deviations from the design doc (strategy/CopilotStudio/Bench-Eval-Integration-and-Live-Connector-Plan.md, local-only)
- The doc cites
Microsoft.Agents.CopilotStudio.Client"v1.6.150 — latest stable" and a decompiledICopilotClient
interface implemented byCopilotClient. Neither matches what actually restores from nuget.org: the real latest
is 1.3.171-beta, and reflecting on that exact assembly showsCopilotClientimplements no interface at
all (GetInterfaces()returns empty) — its full public surface is narrower than the doc's decompilation notes
assumed. This CHANGELOG entry and the code's own XML docs are the corrected record;ICopilotStudioConversationClient
above is the concrete consequence. --max-creditsenforcement (the doc's Track 1 item 6) is not implemented — the SDK's activity/response
models expose no Copilot Credit cost field to enforce against, so--max-creditsremains parsed-but-unenforced
exactly as before (ExitCodes.BudgetExceededstays reserved, unused).
Not independently live-verified — needs a real Entra app registration + non-prod Copilot Studio agent
- The MSAL device-code prompt, silent-refresh, and persisted-cache round-trip (
CopilotStudioTokenProvider.GetTokenAsync). - Whether a real agent's
StartConversationAsync/AskQuestionAsyncactivity stream matches the shape
CopilotStudioChatClientassumes (in particular, any non-messageactivity worth surfacing, and real
multi-activity turns). - The end-to-end network round trip (real HTTP call, real response parsing) against a live MCS agent.
- A gated,
Skip-by-default manual test (CopilotStudioLiveConnectorManualTests,tests/AgentEval.Tests/Cli/CopilotStudio/)
is ready to run once credentials exist — see its XML doc for setup.
Gatekeeper — MonetaryLimitGate + PerToolCallBudgetGate (focused deterministic siblings of RunBudgetGate)
Added
MonetaryLimitGate(AgentEval.MAF.Gatekeeper.Gates) — a dedicated tool gate capping the running sum of a
monetary tool-call argument (e.g."amount") across a run, off the sharedRunLedger. The economic sibling of
RunBudgetGate, scoped to a single argument/cap pair with its ownPolicyNamein the evidence trail — for
payment/refund/transfer-style tools without wiringRunBudgetGate's combined total/per-tool/monetary
constructor. Fails closed on an unparseable amount, clamps a negative amount to zero (can't manufacture
headroom), and the block reason never echoes the attempted amount or running sum — only the argument name and
the configured cap, matching the taint-tracking gate's discipline of never leaking sensitive values into trace
evidence.PerToolCallBudgetGate(AgentEval.MAF.Gatekeeper.Gates) — a dedicated tool gate capping how many times
specific tools may be called in one run (e.g.["delete_account"] = 1,["send_email"] = 3), off the shared
RunLedger. Blunts spray/loop attacks — an injected instruction that tries to fire the same destructive tool
repeatedly is stopped at the configured count regardless of phrasing. A tool not named in the caps is
unconditionally allowed.RunLedger.TryAdmitMonetary/TryAdmitPerToolCall— new atomic, per-dimension ledger primitives backing
the two gates above. Deliberately isolated fromRunBudgetGate's ownTryAdmitToolCallbookkeeping (which
always bumps its shared per-tool/total counters on any admit, even for a dimension the caller didn't ask it to
check) — so composing either dedicated gate withRunBudgetGate, or with each other, over an overlapping
tool/argument name can never cross-contaminate a count. Covered by a regression test proving the isolation holds
even whenRunBudgetGateandPerToolCallBudgetGateare stacked on the same tool.samples/AgentEval.Samples/Gatekeeper/09_GatekeeperMonetaryAndPerCallBudget.cs— a live sample (real Azure
OpenAI agent, no scripted fakes) with three scenes: a 10-call refund spray capped at 3 byPerToolCallBudgetGate,
a single $50,000 refund blocked by a $1,000MonetaryLimitGatecap, and both gates stacked against a $300 ×
10-order spray — success is keyed on the actual recordedgate.tool.*block count, never on "no exception
thrown." Wired into the samples menu (Group J).- Extracted
AmountArgumentParser(shared byRunBudgetGateandMonetaryLimitGate) so the two gates parse a
monetary argument (decimal/double/int/long/JsonElement/ numericstring) identically —
behavior-preserving refactor ofRunBudgetGate's previously-private parsing logic, no functional change.
MAF Agent Skills evaluation — Phase 1 (assertions + progressive-disclosure efficiency metric)
Added
- Five fluent skill assertions in
AgentEval.Assertions.SkillUsageAssertions—HaveLoadedSkill,
HaveReadSkillResource,HaveRunSkillScript,NotHaveRunSkillScript,HaveDisclosedProgressively—
thin, additive extension methods over the existingToolUsageAssertions/ToolCallAssertion(zero
new MAF-type coupling inAgentEval.Core, which still does not referenceMicrosoft.Agents.AI).
Support value-based argument matching (skill/resource/script name), not just tool-name matching, and
degrade gracefully (key-agnostic fallback) if a future MAF version renames an argument. SkillDisclosureEfficiencyMetric(code_skill_disclosure_efficiency,AgentEval.Metrics.Agentic)
— a free, code-basedIAgenticMetricscoring theload_skill→read_skill_resource→
run_skill_scriptprogressive-disclosure funnel as a weighted product of disclosure-order validity,
load precision (redundant-load + "load storm" penalties), and an optional load-selection F1 when the
caller suppliesexpected_skillsground truth. Never fabricates a selection score when no ground
truth is supplied, and never fabricates an "advertise" stage count (the skill-inventory system-prompt
listing is not a tool call and is not observable from aToolUsageReport).SkillToolNames(AgentEval.Skills) — the single shared constant for the three stable GA tool
names (load_skill/read_skill_resource/run_skill_script) and their argument parameter names,
referenced by the assertions and the metric.samples/AgentEval.AgentSkillsEval— a live sample: a realChatClientAgentagainst Azure OpenAI,
wrapped with a realMicrosoft.Agents.AI.AgentSkillsProviderover a real file-based
expense-reportskill fixture (SKILL.md + a resource + an in-process script). Three runs demonstrate
different real assertion/metric/output combinations (read-only lookup, script-computed overage,
and an off-topic task that both scores a vacuous 100/100 and shows an assertion's real failure
path) — all keyed on the actual captured tool-call trace, never a bare success claim.- Verified four MAF
AgentSkillsProviderAPI details against the liveMicrosoft.Agents.AI 1.13.0
assembly (exact tool argument parameter names; theDisableCachingbuilder shape; that there is no
provider-levelGetSkillsAsyncconvenience; and thatread_skill_resource'sresourceNameis a
logical name resolved against the skill's discovered resource list, not a live filesystem path).
This is Phase 1 of a multi-phase design
(strategy/FutureFeatures/Skills/AgentEval-AgentSkills-Evals-Design-and-Plan.md, local-only).
MAF Agent Skills evaluation — Phase 3 (skill-description-injection red-team + run_skill_script governance)
Added
SkillInjectionAttack(AgentEval.RedTeam.Attacks, OWASP LLM01) — the 14th built-in red-team attack
(roster 13→14, probes 258→264). Two newInjectionSurfacevalues,SkillInstruction(a malicious
skill'sdescription/instructions, spliced into the SYSTEM PROMPT via{skills}— a higher-trust
position than a retrieved document) andSkillResource(read_skill_resourceoutput). 100% reuse of
the shipped Wave-B machinery (CanaryTool,FidelityCompositeEvaluator,ToolInvocationEvaluator,
RefusalGatedEvaluator) — canary "source" tools are namedload_skill/read_skill_resource(matching
MAF's real tool names) so an instrumented SUT's trace is indistinguishable from a real
AgentSkillsProviderinteraction. 6 probes at Comprehensive intensity; registered inAttack.All,
ByName,ByOwaspId("LLM01").⚠️ HONESTY FINDING — the reused judge does NOT converge on the skill-description surface. Per the
design doc's own documented risk item, this session ran a LIVE calibration of the flagship
IndirectInjectionRubricagainst a new both-directions gold set
(AgentEval.Guardrails.Judges.Rubrics.SkillInjectionGoldSet, 52 skill-flavored cases) via
GateCalibrationHarness. Result: decisive accuracy 88.5%, 4 missed attacks, 2 false alarms, κ=0.769
vs. gold —IsInlineReady == false(the harness requires zero missed attacks by default). The rubric
generalizes reasonably (beats the deterministic keyword baseline) but not well enough to promote inline
on this NEW surface. Decision: shipped SHADOW-ONLY for the skill surface, per the design doc's own
contingency — never promoted inline, documented in code, the live sample, and here. Authoring a
dedicatedSkillDescriptionInjectionRubricis deferred (the design doc's own +3–5 dev-day contingency
line item).SkillScriptExecutionGate(AgentEval.MAF.Gatekeeper,IToolGate,GateCost.PureCode,
MinimumPolicy = ReplaceResult) — deterministic hard gate onrun_skill_script: blocks a call whose
script identifier is not on the allowlist. Value-based, key-agnostic matching (every string-shaped
argument value, plus"/"-joined pairs, are candidates — never assumes a specific argument key); an
unrecognized/missing script identifier fails closed. No calibration needed (deterministic).SkillScriptApprovalGate(IToolApprovalGate) — auto-approvesload_skill/read_skill_resource;
escalatesrun_skill_scriptto a human UNLESS the script is on a per-script trust allowlist — finer
grained than MAF's nativeReadOnlyToolsAutoApprovalRule(tool-granularity only).- Composition-ordering honesty (design doc §6.2, verified live this session): MAF's skill tools
require human approval BY DEFAULT, and that pause happens BEFORE the FICC seam — so
SkillScriptExecutionGatenever fires unlessrun_skill_scriptis first auto-approved at the MAF
layer (Posture A). The live sample (Run 6) demonstrates this exact composition and confirms the gate
— not the approval layer — is what blocks the call (gate.tool.*count = 1, real trace evidence).
SkillResourcePathGatewas NOT built (dropped per the design doc §3 —read_skill_resource's
resourceNameis a logical name with no traversal surface, confirmed in Phase 1). - Sample Runs 5–6 (
samples/AgentEval.AgentSkillsEval) — live-verified against real Azure OpenAI
this session: Run 5 (skill-injection attack) — the agent resisted (0 tool calls on an off-topic-safe
prompt), and the shadow-only judge verdict is shown labeled advisory-only, never conflated with the
real behavioral verdict. Run 6 (exec-gate demo, Posture A) — the agent DID callrun_skill_scriptwith
an unlisted script, andSkillScriptExecutionGatedeterministically blocked it (1 realgate.tool.*
block), with the model falling back to computing the answer manually — the gate, not the approval
layer, stopped the call. - ~50 new tests across
SkillInjectionAttackTests,SkillScriptExecutionGateTests,
SkillScriptApprovalGateTests,SkillInjectionGoldSetCalibrationTests(deterministic harness-mechanics
proof), and the env-gatedSkillInjectionGoldSetCalibrationLiveCheck(the live calibration check itself,
AGENTEVAL_RUN_SKILLCAL=1).
MAF Agent Skills evaluation — Phase 2 (compliance scanner + coverage report)
Added
SkillComplianceValidator(AgentEval.Skills,AgentEval.Core— pure, MAF-free, no I/O) —
validates aSkillManifestagainst the GASKILL.mdrules (namepresence/length/charset/no
consecutive hyphens/matches parent directory;descriptionpresence/length;compatibilitylength)
plus AgentEval governance flags (ScriptRequiresGovernanceReviewwhen a skill exposes scripts,
ResourceFromUntrustedSourcefor MCP/Custom-sourced resources,AllowedToolsExperimental). Returns a
SkillComplianceReport(findings + a stage-reachability coverage summary) whoseIsCompliantflips
only on aHigh-severity finding.MafSkillScanner(AgentEval.MAF.Skills) — the one place that touches a liveAgentSkill/
AgentSkillsSource. Enumerates skills via the GA source-levelGetSkillsAsync(context, ct), maps each
to the pureSkillManifestDTO, and delegates to the validator. Honesty note:AgentFileSkill
stores its discovered resources/scripts in private fields with no public getter (verified via
reflection against the live MAF 1.13.0 assembly), so this scanner independently re-derives a
file-sourced skill's resource/script inventory by walking itsresources//scripts/subdirectories on
disk — the same convention MAF's ownAgentFileSkillsSourceOptionsuses. For non-file sources
(in-memory/class/MCP/custom) there is no equivalent enumeration API, soResourceNames/ScriptNames
are honestly reported empty rather than guessed — a documented, real limitation, not hidden.SkillComplianceReportRenderer— console/Markdown/JSON rendering, severity-sorted findings plus a
coverage table.- Sample Run 4 (
samples/AgentEval.AgentSkillsEval) —MafSkillScanner.ScanFileSkillsAsyncover the
realexpense-reportfixture; live-verified against real Azure OpenAI this session: 1 skill
scanned, 1 resource + 1 script found on disk,ScriptRequiresGovernanceReviewcorrectly flagged
(Medium, pointing at Phase 3),IsCompliant == true. - 44 new tests (
tests/AgentEval.Tests/Skills/*,tests/AgentEval.Tests/MAF/Skills/*) — every GA rule
fires exactly once on a violating manifest and not on a clean one; coverage counts never fabricate an
"advertise" stage; a regression guard locks in that an undetectable non-file script stays honestly
unreported rather than silently "fixed" with a fabricated count.
MAF Agent Skills evaluation — Phase 4a/4b (Skill Health & Security Index + hash-pin drift detection) + cheap sugar
Added
SkillSecurityIndex(AgentEval.Skills, pure) — joins the three independently-produced skill
quality signals (Phase 2 compliance, Phase 1 efficiency, Phase 3/4b security) into one composite 0-100
index. Never fabricates a missing axis: the score is the mean of only the axes actually supplied,
andSkillSecurityIndexResult.Explanationnames exactly which axes were/weren't measured.ManifestFingerprint/ManifestDriftDetector(AgentEval.Guardrails, pure, MAF-free) — a generic
SHA-256 hash-pin-and-diff primitive, reusable for any model-visible artifact definition (a skill
manifest here; an MCP tool schema in a future gate — same pattern, different artifact type).SkillManifestPoisoningGate+SkillManifestBaseline(AgentEval.Skills) — deterministic
trust-time drift detection for a rug-pulled skill (content silently changing after approval). No
calibration debt (pure hashing).SkillManifestBaselinepersists to JSON (capture → save → later load
→ compare → flag drift), mirroring the repo's existing RedTeam baseline/diff CI pattern, scoped to skills.- Cheap assertion sugar (design catalog §10.4):
WithScriptArgument(asserts inside
run_skill_script's nestedargumentsobject),ForSkill(scopes aToolUsageReportto one skill's
calls when a run exercises multiple skills),HaveDisclosedEfficiently(minScore)(metric-backed,
synchronous — the metric isCodeBasedwith no real async work),HaveCorrectlyDeclinedSkill(positive
phrasing for "the agent correctly avoided this skill").SkillContractAssertions.AssertSkillWellFormed
— a zero-cost (no agent, no LLM) unit-test assertion wrapping the Phase 2 validator. - Sample Run 7 — live-verified against real Azure OpenAI this session: a real simulated rug-pull
(mutating the expense-report skill's description) is correctly caught by the hash-pin drift check
(Changedfinding), and the composite Skill Security Index correctly joins the real Phase 2 compliance
scan (85/100, one Medium finding) with the real Phase 3 behavioral outcome from Run 5 (Resisted → 60/100
after the drift penalty), honestly reporting the Efficiency axis asn/a(not re-measured this run,
never assumed perfect) — composite 72/100, 2/3 axes measured. - Phase 4c (expanded red-team surface — fuzzing, canary-skill honeypot, typosquat detection,
load-storm-as-DoW) was NOT built this session — explicitly deprioritized per the design doc's own
scoring (4a/4b are cheaper and higher-value) and the marathon session's remaining scope (Stages 2-5).
Documented as deferred, not silently dropped — seestrategy/TODO.md. - ~35 new tests. Full net8.0 suite green (7278/7279, 1 pre-existing skip).
Gatekeeper Tribunal — 4 more calibrated flagship judges + 2 overlooked-seam gates
Added
IntentActionMismatchJudge— compares the agent's NARRATED intent against its ACTUAL tool call,
vetoes on divergence. 52-case gold set. Live-calibrated: 100% decisive accuracy, κ=1.000,
IsInlineReady=true.GoalHijackDriftJudge— detects the agent being steered off the user's original stated goal toward
an injected objective (distinct from indirect-injection: asks "has direction drifted," not "does this
content instruct"). 48-case gold set. Live-calibrated: 100% decisive accuracy, κ=1.000,
IsInlineReady=true.UngroundedClaimJudge— RAG faithfulness as a runtime gate: flags an answer claim unsupported by
retrieved context. 48-case gold set (includes hedged-opinion hard-negatives). Live-calibrated: 100%
decisive accuracy, κ=1.000,IsInlineReady=true.HallucinatedCitationJudge— hybrid: a deterministic, zero-LLM-cost citation-existence check
composed with a judge support-check, only spending a model call when the citation exists. 52-case gold
set covering both failure modes (nonexistent source; real source that doesn't support the claim).
Live-calibrated: 100% decisive accuracy, κ=1.000,IsInlineReady=true. Not anIJudgeRubric(a
bespokeIChatGate), so not registered in the CLI bridge'sjudge:*axis registry — fully usable
directly.MemoryWritePoisoningGate— guards the memory/vector-store WRITE side (every other injection judge
guards reads). ReusesIndirectInjectionRubricverbatim at this new seam per the design backlog's
reuse-the-pattern guidance.McpToolDescriptionPoisoningGate+McpToolDefinition— deterministic hash-pin-and-diff over an
MCP tool's definition (name/description/schema), catching a rug-pull. Reuses the exact
ManifestFingerprint/ManifestDriftDetectorgeneric primitive built for Skills Phase 4b's
SkillManifestPoisoningGate— confirming the design backlog's own "same pattern, different artifact
type" prediction. Schema comparison recursively canonicalizes JSON key order (a reformatted-but-identical
schema never false-alarms).- All three
IJudgeRubric-based judges registered inJudgeAxisRegistry— live-verified via the CLI
bridge this session:agenteval gatekeeper list-gatesshows all three (judge:intent-action-mismatch,
judge:goal-hijack-drift,judge:ungrounded-claim+ their keyword baselines);
agenteval gatekeeper calibrate --gate judge:goal-hijack-drift --certifyagainst real Azure OpenAI wrote
a real calibration certificate;agenteval gatekeeper inspectthen correctly Allowed a benign case and
Blocked an attack case, citing the certificate. - Deferred, explicitly NOT built this session:
ToolArgumentGoalCoherenceJudge(needs the
IToolApprovalGatetimeout-routing design worked out) andCrescendoTrajectoryJudge(stateful — session
store + running summary — explicitly flagged as the hardest of the six in the task scope; deferring it
matches the task's own suggested fallback). Seestrategy/TODO.mdfor the honest accounting. - ~100 new tests (deterministic rubric/gate tests + 4 env-gated live calibration checks,
AGENTEVAL_RUN_GATEKEEPER_CAL=1). Full net8.0 suite green (7330/7331, 1 pre-existing skip).
Copilot Studio — mock backend + Track 2 (shared --sut seam, PR 1)
Added
MockCopilotStudioConversationClient(test-only) — a realistic, reusable mock Copilot Studio
backend (a test double forICopilotStudioConversationClient, since no live Copilot Studio system is
available in this environment). Supports scripted MULTI-TURN conversations (fluent builder, mirroring
ScriptedChatClient's convention), a SERVER-ASSIGNED conversation id (matching real MCS session
semantics), and configurable ERROR INJECTION (auth failure on start, a mid-conversation exception at a
chosen turn — e.g. rate-limit-shaped — and a hang-until-cancelled mode for timeout testing). 7 tests
proving the mock itself behaves realistically (session-state tracking, activity-type filtering, error
propagation, honest "no scripted turn" default that never fabricates a blank success).- Track 2, PR 1 — the shared
--sutseam (strategy/CopilotStudio/Bench-Eval-Integration-and-Live-Connector-Plan.md
§3):ISutTarget/ISutTargetOptions/CommonTargetOptions/SutTargetResolver
(src/AgentEval.Cli/Commands/Targets/ISutTarget.cs) — generalizes the already-shippedredteam --sut
pattern soeval/benchcan reach the same built-in targets, WITHOUT touching
IRedTeamBuiltInTarget/RedTeamOptions/RedTeamCommand.cs.CopilotStudioRedTeamTargetgainsISutTarget
via EXPLICIT interface implementation (same idiom asIEnumerable/IEnumerable<T>) — its existing
IRedTeamBuiltInTargetmembers are byte-for-byte unchanged. AValidateDriftcontract test (theory,
4 truth-table cases) provesIRedTeamBuiltInTarget.ValidateandISutTarget.Validateagree on
accept/reject for every shared check (consent / config-required / max-credits ≥ 0) — the one real
ongoing-sync risk the design doc calls out, since the two method bodies have no compiler-enforced sync.
12 new tests.gatekeeper-demodeliberately staysredteam-only (needs anAgentTrace, which
eval/benchhave no use for) — onlycopilot-studiogets the shared treatment, per the design doc. - NOT built this session (explicitly deferred, documented honestly): Track 2 PR 2 (
evaladoption)
and PR 3 (bench Tier 1owasp/mitre/nistadoption) — the shared types exist and are tested, but no
CLI verb wires them in yet; P6 (reports & resilience: fidelity badging, agent-fingerprint drift,
429 retry+resume), P3 (KnowledgeCanaryEvaluator, Crescendo/PAIR/TAP over the native channel), and P7
(OSS polish, Entra app-reg script, NuGet packaging) were not started. Seestrategy/TODO.mdfor the
honest accounting and what's next. - Full net8.0 suite green (see the final Stage 5 numbers in this file's next entry).
Documentation — Stage 5 pass (Agent Skills, Gatekeeper, Copilot Studio) + final build/test verification
Added
docs/agent-skills.md— new user-facing feature page for MAF Agent Skills evaluation (assertions,
disclosure-efficiency metric, compliance scanner, skill-injection red-team +run_skill_scriptgovernance
gates, Skill Health & Security Index, hash-pin drift detection). Previously this only existed at
implementation-detail depth insidedocs/architecture.md; that section now cross-links here. Linked from
docs/index.md's Documentation table and Feature Highlights grid.docs/redteam/copilot-studio.md— corrected a stale sentence that still said "until the connector ships"
even thoughBuildLivehas shipped since this doc was first written; documented the new
MockCopilotStudioConversationClienttest double and the not-yet-CLI-reachable sharedISutTarget/
SutTargetResolverseam (Track 2 PR 1).
Verified
- Full-solution
dotnet build -c Release: 0 errors (66 pre-existing warnings, unrelated to this
session's changes — nullable-reference-type test scaffolding and xUnit analyzer style suggestions). - Full net8.0 test suite (fresh build, not
--no-build, per this repo's known multi-TFM stale-binary trap):
7349 passed / 0 failed / 1 skipped (the skip is the pre-existing, intentionally gated
CopilotStudioLiveConnectorManualTests— needs real Entra credentials this environment does not have) —
no regressions from any of Stages 1–5.