[docs] Update glossary - weekly full scan#8951
Conversation
Add 7 new terms introduced in recent changes: - ArtifactNamingHelper: template-based artifact naming helper used by HangDump/CrashDump extensions - AzureFoundry: new MTP extension for Azure AI Foundry (Azure OpenAI) integration - HtmlReport: self-contained HTML test report extension (was missing from glossary) - IChatClientProvider: new AI provider abstraction interface in Microsoft.Testing.Platform.AI - JUnitReport: new MTP extension for JUnit XML test report generation (experimental) - Microsoft.Testing.Platform.AI: new AI extensibility abstractions package for MTP - testconfig.json: MTP per-project configuration file with environmentVariables support Based on commit 1b64756 which introduced Microsoft.Testing.Extensions.AzureFoundry, Microsoft.Testing.Extensions.JUnitReport, Microsoft.Testing.Platform.AI packages, and related docs (RFCs 001-AI-Extensibility, 002-TestConfig-EnvironmentVariables, ArtifactNamingHelper.md). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Evangelink
added
area/documentation
There was a problem hiding this comment.
Pull request overview
Updates the repository glossary (docs/glossary.md) to include newly introduced Microsoft.Testing.Platform (MTP) concepts and extensions discovered by the weekly full scan, improving discoverability of recent platform features (AI extensibility, new report extensions, and testconfig.json behavior).
Changes:
- Added glossary entries for
ArtifactNamingHelper,AzureFoundry,HtmlReport,IChatClientProvider,JUnitReport,Microsoft.Testing.Platform.AI, andtestconfig.json. - Documented key behaviors and enabling mechanisms for new extensions/features (CLI switches, placeholder tokens, controller process model).
Show a summary per file
| File | Description |
|---|---|
| docs/glossary.md | Adds 7 new glossary entries covering new MTP extensions (AI + reports) and config concepts. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 2
| ### JUnitReport | ||
|
|
||
| An MTP extension (`Microsoft.Testing.Extensions.JUnitReport`) that emits a JUnit-style XML test report at the end of a test run. The report conforms to the Jenkins/Surefire `<testsuites><testsuite><testcase>` schema and is accepted by Jenkins (`junit` step), GitLab CI (`junit:` artifact reports), Azure DevOps (`PublishTestResults@2` with `testResultsFormat: 'JUnit'`), CircleCI, GitHub Actions test reporters, and most other CI tooling. MTP's hierarchical [TestNode](#testnode) tree is preserved as a `<property name="testpath" value="…"/>` element inside each `<testcase>`, allowing tools to reconstruct hierarchy. Auto-registers via MSBuild by setting `<EnableMicrosoftTestingExtensionsJUnitReport>true</EnableMicrosoftTestingExtensionsJUnitReport>`. Currently **experimental** — the API, CLI options, and on-disk format may change without notice. Enable via `--report-junit`; override filename with `--report-junit-filename`. |
| ### ArtifactNamingHelper | ||
|
|
||
| A shared static helper compiled into MTP extensions via file linking (no NuGet service registration or InternalsVisibleTo required) that provides template-based naming for test artifact files (dump files, report files, etc.). Templates are strings containing `{placeholder}` tokens (case-sensitive, lowercase): `{pname}` (process name), `{pid}` (process ID), `{asm}` (entry-assembly name), `{tfm}` (target framework moniker, best-effort runtime detection), and `{time}` (high-precision UTC timestamp). Legacy `%p` patterns from earlier hang-dump versions continue to work. Custom per-call overrides can replace default placeholder values via a `Dictionary<string, string>`. Used by the [HangDump](#hangdump) and [CrashDump](#crashdump) extensions. |
Evangelink
left a comment
Evangelink
left a comment
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
Expert Review — PR #8951 · [docs] Update glossary - weekly full scan
Overall verdict: REQUEST_CHANGES — one broken path reference (concrete, verifiable) and one NIT on attribution.
22-Dimension Assessment
| # | Dimension | Verdict | Notes |
|---|---|---|---|
| 1 | Algorithmic Correctness | N/A | Docs only |
| 2 | Threading & Concurrency | N/A | Docs only |
| 3 | Security & IPC Contract Safety | N/A | Docs only |
| 4 | Public API & Binary Compatibility | N/A | Docs only |
| 5 | Performance & Allocations | N/A | Docs only |
| 6 | Cross-TFM Compatibility | N/A | Docs only |
| 7 | Resource & IDisposable Management | N/A | Docs only |
| 8 | Defensive Coding at Boundaries | N/A | Docs only |
| 9 | Localization & Resources | N/A | Docs only |
| 10 | Test Isolation | N/A | Docs only |
| 11 | Assertion Quality | N/A | Docs only |
| 12 | Flakiness Patterns | N/A | Docs only |
| 13 | Test Completeness & Coverage | N/A | Docs only |
| 14 | Data-Driven Test Coverage | N/A | Docs only |
| 15 | Code Structure & Simplification | N/A | Docs only |
| 16 | Naming & Conventions | ✅ LGTM | New terms are consistent with existing glossary style |
| 17 | Documentation Accuracy | ❌ ISSUES | See inline comments below |
| 18 | Analyzer & Code Fix Quality | N/A | No src/Analyzers/ changes |
| 19 | IPC Wire Compatibility | N/A | Docs only |
| 20 | Build Infrastructure & Dependencies | N/A | Docs only |
| 21 | Scope & PR Discipline | ✅ LGTM | Focused documentation update |
| 22 | PowerShell Scripting Hygiene | N/A | No .ps1 changes |
Findings Detail
🔴 NIT (blocks merge) — testconfig.json: wrong file path
Line 187 — The entry references docs/RFCs/002-TestConfig-EnvironmentVariables.md, but that file does not exist in the repository. The actual document is at:
docs/microsoft.testing.platform/002-TestConfig-EnvironmentVariables.md
docs/RFCs/002-... points to 002-Framework-Extensibility-Custom-Assertions.md — a completely unrelated RFC. When rendered on GitHub, this becomes a broken relative link that would confuse readers. Inline comment left with a suggested fix.
🟡 NIT — ArtifactNamingHelper: %p support is in HangDump, not in the helper
Line 17 — "Legacy %p patterns from earlier hang-dump versions continue to work" is accurate but is attributed to ArtifactNamingHelper, when the actual %p → processId replacement is performed by HangDumpProcessLifetimeHandler as a separate post-processing step after ArtifactNamingHelper.ResolveTemplate is called. ArtifactNamingHelper only understands {placeholder} syntax. Moving the sentence to the HangDump entry (or scoping it with "handled by the HangDump extension") would prevent confusion.
What Looks Good
- Technical accuracy of all other entries verified against source:
IChatClientProvider— four members (IsAvailable,HasToolsCapability,ModelName,CreateChatClientAsync) matchIChatClientProvider.csexactly ✓AzureFoundry— three required env vars (AZURE_OPENAI_ENDPOINT,AZURE_OPENAI_DEPLOYMENT_NAME,AZURE_OPENAI_API_KEY) matchAzureOpenAIChatClientProvider.cs✓JUnitReport— CLI option names--report-junit/--report-junit-filenamematchJUnitReportGeneratorCommandLine.csconstants ✓HtmlReport—--report-htmloption matchesHtmlReportGeneratorCommandLine.cs✓Microsoft.Testing.Platform.AI— design RFC pathdocs/microsoft.testing.platform/001-AI-Extensibility.mdconfirmed ✓
- Cross-reference integrity — all
[term](#anchor)links resolve to entries that exist (or are added by this PR):#ichatclientprovider,#microsofttestingplatformai,#azurefoundry,#hangdump,#crashdump,#testnodeall valid ✓ - Alphabetical ordering — new entries are placed correctly within their letter sections ✓
- Glossary style — entries follow the established pattern (NuGet package name in backticks, cross-links via
[Term](#anchor), experimental warnings in bold) ✓
Generated by Expert Code Review (on open) for issue #8951 · sonnet46 3.9M
|
|
||
| ### testconfig.json | ||
|
|
||
| The per-project configuration file for Microsoft.Testing.Platform, placed at the project root and read at test startup. Supports multiple top-level sections; a key one is `environmentVariables`, which declares environment variables to set on the test host process — mirroring the `<EnvironmentVariables>` element of legacy `.runsettings` and removing the need to write a custom `ITestHostEnvironmentVariableProvider` (introduced in RFC 002, `docs/RFCs/002-TestConfig-EnvironmentVariables.md`). When the `environmentVariables` section is present and non-empty, MTP activates the **controller process model**: the launching process becomes the controller, injects the declared variables into `ProcessStartInfo`, and spawns the actual test host as a child process. |
There was a problem hiding this comment.
Broken path reference — docs/RFCs/002-TestConfig-EnvironmentVariables.md does not exist
The path docs/RFCs/002-TestConfig-EnvironmentVariables.md is incorrect. The actual file lives at:
docs/microsoft.testing.platform/002-TestConfig-EnvironmentVariables.md
docs/RFCs/002-... is 002-Framework-Extensibility-Custom-Assertions.md — a completely different RFC. Calling this "RFC 002" without qualification is therefore also misleading.
Suggested fix:
... removing the need to write a custom `ITestHostEnvironmentVariableProvider` (see `docs/microsoft.testing.platform/002-TestConfig-EnvironmentVariables.md`). When the ...Or, if the "RFC 002" label is intentional (matching the numbering within docs/microsoft.testing.platform/), clarify the scope:
... (introduced in platform RFC 002, `docs/microsoft.testing.platform/002-TestConfig-EnvironmentVariables.md`). ...|
|
||
| ### ArtifactNamingHelper | ||
|
|
||
| A shared static helper compiled into MTP extensions via file linking (no NuGet service registration or InternalsVisibleTo required) that provides template-based naming for test artifact files (dump files, report files, etc.). Templates are strings containing `{placeholder}` tokens (case-sensitive, lowercase): `{pname}` (process name), `{pid}` (process ID), `{asm}` (entry-assembly name), `{tfm}` (target framework moniker, best-effort runtime detection), and `{time}` (high-precision UTC timestamp). Legacy `%p` patterns from earlier hang-dump versions continue to work. Custom per-call overrides can replace default placeholder values via a `Dictionary<string, string>`. Used by the [HangDump](#hangdump) and [CrashDump](#crashdump) extensions. |
There was a problem hiding this comment.
NIT — legacy %p support is in HangDump, not in ArtifactNamingHelper
The sentence "Legacy %p patterns from earlier hang-dump versions continue to work" is technically true but is potentially misleading here because ArtifactNamingHelper.ResolveTemplate only handles {placeholder} tokens. The %p substitution is performed by HangDumpProcessLifetimeHandler after calling ArtifactNamingHelper, as a separate post-processing step:
// First resolve {placeholder} templates, then handle legacy %p pattern for backward compatibility.
.Replace("%p", processId);A reader of the ArtifactNamingHelper entry could wrongly conclude that the helper itself understands %p. Consider moving this sentence to the HangDump entry, or scoping it explicitly:
Legacy
%ppatterns from earlier HangDump versions continue to work (handled by the HangDump extension as a post-processing step after template resolution).
Evangelink
deleted the
docs/glossary-weekly-update-2026-06-09-0a3a5bd036e5a902
branch
Glossary Updates
Scan Type: Full scan (weekly — Monday 2026-06-09)
Terms Added (7 new entries):
{pname},{pid},{asm},{tfm},{time}placeholder tokens for constructing artifact file paths. Used by HangDump and CrashDump extensions.Microsoft.Testing.Extensions.AzureFoundry) that implementsIChatClientProviderbacked by Azure AI Foundry (Azure OpenAI). Reference implementation of theMicrosoft.Testing.Platform.AIabstractions.Microsoft.Testing.Extensions.HtmlReport) for generating self-contained HTML test reports. Was present in the codebase but missing from the glossary.Microsoft.Testing.Platform.AI.IChatClientProvider) that defines the contract for AI provider integrations —IsAvailable,HasToolsCapability,ModelName,CreateChatClientAsync.Microsoft.Testing.Extensions.JUnitReport) that emits JUnit XML test reports in the Jenkins/Surefire schema, compatible with Jenkins, GitLab CI, Azure DevOps, CircleCI, and GitHub Actions. Enabled via--report-junit.IChatClientProviderand leveragesMicrosoft.Extensions.AI. Abstractions-only; requires an AI provider implementation such as AzureFoundry.environmentVariablessection (RFC 002) mirrors the legacy.runsettings<EnvironmentVariables>element and triggers the controller process model to inject variables before spawning the test host.Terms Updated: None
Changes Analyzed:
1b64756)Microsoft.Testing.Extensions.AzureFoundry,Microsoft.Testing.Extensions.JUnitReport,Microsoft.Testing.Platform.AIpackages and associated docs (001-AI-Extensibility.md,002-TestConfig-EnvironmentVariables.md,ArtifactNamingHelper.md)Related Changes:
1b64756:[main] Update dependencies from devdiv/DevDiv/vs-code-coverage (#8945)— introduced new AI extensibility framework, JUnit report extension, and Azure Foundry AI providerdocs/microsoft.testing.platform/001-AI-Extensibility.md: AI-Powered Extensibility designdocs/RFCs/002-TestConfig-EnvironmentVariables.md:testconfig.jsonenvironment variables supportdocs/microsoft.testing.platform/ArtifactNamingHelper.md: Artifact naming template systemAdd this agentic workflows to your repo
To install this agentic workflow, run