You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Deferred from #1088. The hub fix (PR #1107) migrates the reusable validator's unit-test step to native MTP and states the consumer-side prerequisites in WORKFLOW.md D1.6 and the csharp.coverage.codecov assert, but those prerequisites are currently verified by hand only.
What Should Be Detected
A dotnet repo whose test project is MTP-based (Microsoft.Testing.Platform.MSBuild in the graph, directly or transitively, in practice xunit.v3 4.0.0 and later) but which carries no root global.jsontest.runner opt-in. That is the exact drift #1088 exists to prevent finding out about only when a dependabot bump PR goes red, which is how Utilities, PlexCleaner, and NxWitness each found out.
Worth detecting alongside it, since they are the same migration and the same read:
xunit.runner.visualstudio still referenced by an MTP-based test project (the VSTest adapter MTP replaces).
coverlet.collector still referenced by an MTP-based test project. Its VSTest data collector is silently ignored under MTP (MTP0001), so coverage goes dark without failing.
Microsoft.Testing.Extensions.CodeCoverage below the 18.9.0 floor D1.6 states. Below 18.1.0 the failure is loud (a TypeLoadException, zero tests, non-zero exit), but it still writes a well-formed Cobertura file claiming full coverage, so anything keying on the report's presence rather than the exit code reads it as a clean run.
Why It Is Not Trivial
Per the existing mechanics, spec/audit.py currently evaluates no check belonging to a type in spec/project-types.json, and AUDIT.md says so explicitly in two places. Mechanizing a csharp.* check contradicts that standing claim, so this needs AUDIT.md and the two matching statements in spec/audit.py reconciled as part of the same change, not just a new block in audit_repo.
There is also no existing "does this repo have a test project" signal in the registry or the spec. The nearest proxy is the coverage_active derivation, which really means "declares csharp or python at build profile". Detecting a test project means scanning the tree via repo_tree_entries for *Tests*.csproj, which the file-presence dimension already fetches once and can be reused rather than re-fetched.
Notes
The check has parseable logic, so per the fleet's own convention it should be a module-level helper returning (kind, text) tuples with a case table in _selftest, rather than an inline block in audit_repo (neither the dependabot nor the cspell inline blocks have self-test coverage today).
Verdict tier is a judgment call worth making deliberately: a repo not yet migrated is a migration owed rather than drift, per the wording PR Migrate the Hub's Unit-Test Step to Native Microsoft.Testing.Platform #1107 lands in D1.6, so DRIFT is probably right and LETTER probably is not, at least until the fleet rollout completes.
Deferred from #1088. The hub fix (PR #1107) migrates the reusable validator's unit-test step to native MTP and states the consumer-side prerequisites in
WORKFLOW.mdD1.6 and thecsharp.coverage.codecovassert, but those prerequisites are currently verified by hand only.What Should Be Detected
A dotnet repo whose test project is MTP-based (
Microsoft.Testing.Platform.MSBuildin the graph, directly or transitively, in practice xunit.v3 4.0.0 and later) but which carries no rootglobal.jsontest.runneropt-in. That is the exact drift #1088 exists to prevent finding out about only when a dependabot bump PR goes red, which is how Utilities, PlexCleaner, and NxWitness each found out.Worth detecting alongside it, since they are the same migration and the same read:
xunit.runner.visualstudiostill referenced by an MTP-based test project (the VSTest adapter MTP replaces).coverlet.collectorstill referenced by an MTP-based test project. Its VSTest data collector is silently ignored under MTP (MTP0001), so coverage goes dark without failing.Microsoft.Testing.Extensions.CodeCoveragebelow the 18.9.0 floor D1.6 states. Below 18.1.0 the failure is loud (aTypeLoadException, zero tests, non-zero exit), but it still writes a well-formed Cobertura file claiming full coverage, so anything keying on the report's presence rather than the exit code reads it as a clean run.Why It Is Not Trivial
Per the existing mechanics,
spec/audit.pycurrently evaluates no check belonging to a type inspec/project-types.json, andAUDIT.mdsays so explicitly in two places. Mechanizing acsharp.*check contradicts that standing claim, so this needsAUDIT.mdand the two matching statements inspec/audit.pyreconciled as part of the same change, not just a new block inaudit_repo.There is also no existing "does this repo have a test project" signal in the registry or the spec. The nearest proxy is the
coverage_activederivation, which really means "declares csharp or python at build profile". Detecting a test project means scanning the tree viarepo_tree_entriesfor*Tests*.csproj, which the file-presence dimension already fetches once and can be reused rather than re-fetched.Notes
(kind, text)tuples with a case table in_selftest, rather than an inline block inaudit_repo(neither the dependabot nor the cspell inline blocks have self-test coverage today).DRIFTis probably right andLETTERprobably is not, at least until the fleet rollout completes.