Move package compatibility app to tools and add robust test coverage by paulmedynski · Pull Request #4242 · dotnet/SqlClient

paulmedynski · 2026-04-28T12:48:14Z

Summary

Move the package-compatibility smoke app from doc/apps/AzureAuthentication to tools/PackageCompatibility.
Expand package-version override support and generated version constants.
Add a modern xUnit v3 test suite:
- Confirms app behaviour across a variety of scenarios.
- Gives us a good example of how xUnit v3 works.
Add permanent test-documentation guidance to .github/instructions/testing.instructions.md.
Remove unused Azure.Core package version from the local CPM file.

Validation

dotnet test -f net10.0 in tools/PackageCompatibility/test (pass: 8/8)

Notes

Added tools/PackageCompatibility/global.json to force Microsoft.Testing.Platform runner for local xUnit v3 compatibility on net10.

Maintainer checklist

Tests added or updated
Verified against customer repro (if applicable)
Ensure no breaking changes introduced

Copilot

Pull request overview

This PR relocates the package-compatibility smoke app into tools/PackageCompatibility and adds an xUnit v3 test suite to validate CLI behavior, App.Run error paths, and MSBuild package-version override/codegen output.

Changes:

Moved/renamed the compatibility tool and updated namespaces/solution wiring.
Added MSBuild-driven PackageVersions.g.cs generation plus expanded version override support.
Added xUnit v3 tests covering CLI required-args/help output, malformed connection handling, and -p: override propagation.

Reviewed changes

Copilot reviewed 17 out of 21 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
tools/PackageCompatibility/src/PackageCompatibility.csproj	Renames root namespace, adds package references, and grants InternalsVisibleTo to the new test project.
tools/PackageCompatibility/src/App.cs	Updates banner/package reporting and app name; keeps connectivity test behavior.
tools/PackageCompatibility/src/EntryPoint.cs	Updates help text and prints additional resolved package versions.
tools/PackageCompatibility/src/GeneratePackageVersions.targets	Adds MSBuild target + Roslyn task to generate `PackageVersions.g.cs` from resolved package versions.
tools/PackageCompatibility/src/PackageVersions.Partial.cs	Renames the Azure “N/A” constant and aligns namespace with new tool location.
tools/PackageCompatibility/src/SqlClientEventListener.cs	Namespace update to match the new tool location.
tools/PackageCompatibility/test/PackageCompatibility.Test.csproj	New xUnit v3 test project targeting net481/net10.0.
tools/PackageCompatibility/test/EntryPointTests.cs	New tests for required args and `--help` output contracts.
tools/PackageCompatibility/test/AppTests.cs	New tests for malformed connection-string handling and log-events banner ordering.
tools/PackageCompatibility/test/BuildVersionOverrideTests.cs	New hermetic build test validating `-p:` overrides are reflected in generated version constants.
tools/PackageCompatibility/test/ConsoleCollection.cs	Adds a non-parallel xUnit collection for console-mutating tests.
tools/PackageCompatibility/Directory.Build.props	Enables CPM for the tool subtree.
tools/PackageCompatibility/Directory.Packages.props	Tool-local CPM versions + MSBuild override defaults for package versions.
tools/PackageCompatibility/NuGet.config	Adds local `packages/` feed for unpublished nupkgs.
tools/PackageCompatibility/global.json	Pins SDK and forces Microsoft.Testing.Platform runner within the subtree.
tools/PackageCompatibility/README.md	New tool documentation and usage examples.
tools/PackageCompatibility/packages/.gitkeep	Ensures `packages/` exists for the local NuGet feed.
src/Microsoft.Data.SqlClient.slnx	Removes old doc app and adds the tool + test projects to the solution.
doc/apps/AzureAuthentication/README.md	Removes the old doc sample app README.
doc/apps/AzureAuthentication/Directory.Packages.props	Removes old doc sample’s CPM file.
.github/instructions/testing.instructions.md	Adds guidance requiring XML docs on tests/helpers/fixtures.

Comments suppressed due to low confidence (1)

tools/PackageCompatibility/src/PackageCompatibility.csproj:8

The project targets net481 unconditionally. This can break cross-platform builds/solution loading on non-Windows hosts (net48* TFMs aren't supported there). The repo convention is to add netfx TFMs only when building on Windows (e.g., src/Microsoft.Data.SqlClient/tests/UnitTests/Microsoft.Data.SqlClient.UnitTests.csproj:12-19). Consider making net481 conditional on '$(OS)' == 'Windows_NT' (or dropping it if not required).

cheenamalhotra · 2026-04-28T14:02:30Z

@@ -0,0 +1,197 @@
+# PackageCompatibility Tool


How is it different than running Manual Tests in package reference mode? What's missing in Manual tests (package reference) that this tool will cover?

Added an explanation in the README.

But my question was really:

Can this NOT be added to or covered by existing Manual tests? I would avoid creating new projects for individual usecases, and make Manual tests in Package mode cover these cases as well. If it's not POSSIBLE to do so, that's a different thing.

But I don't think its not POSSIBLE to be done within the scope of Manual/Integration testing.

Copilot

Pull request overview

Copilot reviewed 18 out of 22 changed files in this pull request and generated 3 comments.

codecov · 2026-04-28T18:15:36Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.90%. Comparing base (c724554) to head (dfa046c).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4242      +/-   ##
==========================================
- Coverage   66.57%   64.90%   -1.67%     
==========================================
  Files         284      279       -5     
  Lines       43301    66219   +22918     
==========================================
+ Hits        28827    42981   +14154     
- Misses      14474    23238    +8764

Flag	Coverage Δ
CI-SqlClient	`?`
PR-SqlClient-Project	`64.90% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mdaigle

Do you plan to run these on PR/CI builds eventually?

mdaigle · 2026-05-05T19:12:57Z

+/// <see cref="System.Console.Error"/> cannot interfere with one another.
+/// </summary>
+[CollectionDefinition(Name, DisableParallelization = true)]
+public sealed class ConsoleCollection : ICollectionFixture<ConsoleCollectionFixture>


✏️ you could capture and restore the original out and error as part of this fixture's construct/dispose lifecycle to avoid duplicating setup and tear down in each test.

Fixed in 311a570: EntryPointTests now implements IDisposable — constructor captures and redirects Console.Out/Console.Error, Dispose() restores them. No more duplicated setup/teardown in each test method.

…zureAuthentication app - Add LoggingVersion and AbstractionsVersion MSBuild properties (default 1.0.0) - Add PackageVersion entries for Microsoft.Data.SqlClient.Internal.Logging and Microsoft.Data.SqlClient.Extensions.Abstractions - Reference both packages explicitly in the project to surface their resolved versions in the generated PackageVersions class - Emit Logging and Abstractions versions alongside the other package versions in App.cs and EntryPoint.cs startup output - Rename PackageVersions.AzureExtensionsVersion to MicrosoftDataSqlClientExtensionsAzureVersion to match the naming pattern used by all other generated version constants - Update README with new build parameters, help-text examples, and sample output

The primary purpose of this app is to test package compatibility, not specifically Azure authentication, so the name now better reflects its purpose. - git mv doc/apps/AzureAuthentication/ tools/PackageCompatibility/ - Rename AzureAuthentication.csproj to PackageCompatibility.csproj - Update RootNamespace and all source namespaces to Microsoft.Data.SqlClient.Tools.PackageCompatibility - Rename AppName constant to 'Package Compatibility Tester' - Update README title, help-text block, and sample output accordingly

- add explicit Microsoft.SqlServer.Server version support and output - keep PackageCompatibility package/version ordering as currently arranged - update help text and sample output for current package version set - expand README guidance around package compatibility and auth flows

- Added support to specify all SqlClient suite package versions. - Added a modern xUnit3 test suite.

- Fix net481 build: add IsExternalInit polyfill, GetRelativePath helper, and Split overload compat for net481 in BuildVersionOverrideTests - Use PackageVersions constants instead of hard-coded version strings in EntryPointTests.HelpOutputContainsDefaultVersions (fix #1) - Fix WaitForExit deadlock risk on net481: read stdout/stderr async before waiting; check timeout bool and kill/throw on expiry (fix #2) - Fix Directory.Packages.props typo: explcitily -> explicitly (fix #4) - Fix README.md typo: -p:SqlServerVersio= -> -p:SqlServerVersion= (fix #5) - Move WaitForCapturedOutput inside try block so late console output is captured before streams are restored (fix #6) - Use ArgumentList on NET and QuoteArgument helper on net481 so paths with spaces are passed correctly to dotnet build (fix #7) - Fix testing.instructions.md list continuation indents (4-space -> 2-space) so GitHub renders them as wrapped text not code blocks (fix #8)

… running manual tests.

…evels, console fixture - Exclude packages/ in ShouldSkip to avoid copying large local .nupkg files; create an empty packages/ dir in the temp workspace for NuGet.config validity. - Add try/finally cleanup of temp build directories (opt-out via PACKAGECOMPATIBILITY_PRESERVE_TEMP env var). - Demote sub-headings under 'Test Documentation Requirements' from ### to #### so they render as subsections of the ### parent. - Move console capture/restore into EntryPointTests constructor/Dispose to eliminate duplicated setup/teardown in each test method.

Copilot

Pull request overview

Copilot reviewed 18 out of 22 changed files in this pull request and generated 2 comments.

… root - Remove dead 'bool preserveTemp' in BuildAppWithProperties (cleanup is controlled by the caller's ShouldPreserveTempBuildDirectories() call). - Update tools/PackageCompatibility/global.json SDK version from 10.0.107 to 10.0.300 to match the repo-root global.json.

Copilot

Pull request overview

Copilot reviewed 18 out of 22 changed files in this pull request and generated 2 comments.

+A minimal console application that verifies that a set of SqlClient packages can coexist without
+transitive dependency conflicts, API surface mismatches, or broken runtime functionality.  It loads
+assemblies from each package and then opens a `SqlConnection` to confirm that the resolved package
+graph works end-to-end against a real SQL Server instance.


+| SQL Server auth | *(omit or `SqlPassword`)* | `Microsoft.Data.SqlClient` only |
+| Windows / Integrated | `ActiveDirectoryIntegrated` | `Microsoft.Data.SqlClient` + SSPI |
+| Entra ID — default chain | `ActiveDirectoryDefault` | `Microsoft.Data.SqlClient` + `Extensions.Azure` (requires `AzureVersion`) |


Copilot AI review requested due to automatic review settings April 28, 2026 12:48

github-project-automation Bot added this to SqlClient Board Apr 28, 2026

github-project-automation Bot moved this to To triage in SqlClient Board Apr 28, 2026

Copilot started reviewing on behalf of paulmedynski April 28, 2026 12:48 View session

paulmedynski added the Area\Tests Issues that are targeted to tests or test projects label Apr 28, 2026

paulmedynski moved this from To triage to In progress in SqlClient Board Apr 28, 2026

paulmedynski added this to the 7.1.0-preview2 milestone Apr 28, 2026

Copilot AI reviewed Apr 28, 2026

View reviewed changes

cheenamalhotra reviewed Apr 28, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings April 28, 2026 16:52

paulmedynski marked this pull request as ready for review April 28, 2026 16:52

paulmedynski requested a review from a team as a code owner April 28, 2026 16:52

paulmedynski moved this from In progress to In review in SqlClient Board Apr 28, 2026

Copilot started reviewing on behalf of paulmedynski April 28, 2026 17:03 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Comment thread tools/PackageCompatibility/test/BuildVersionOverrideTests.cs

Comment thread tools/PackageCompatibility/test/BuildVersionOverrideTests.cs

Comment thread .github/instructions/testing.instructions.md

paulmedynski enabled auto-merge (squash) May 5, 2026 11:56

mdaigle previously approved these changes May 5, 2026

View reviewed changes

paulmedynski assigned mdaigle and cheenamalhotra May 8, 2026

paulmedynski marked this pull request as draft May 8, 2026 16:48

auto-merge was automatically disabled May 8, 2026 16:48
Pull request was converted to draft

paulmedynski unassigned mdaigle and cheenamalhotra May 8, 2026

paulmedynski added 6 commits May 29, 2026 10:41

- Moved this app from doc/apps to tools/ which seems like a better fit.

a25a78c

- Added support to specify all SqlClient suite package versions. - Added a modern xUnit3 test suite.

Removed unused Azure.Core package version.

85474ac

Added explanation of why this tool is useful, and how it differs from…

cdeb1b1

… running manual tests.

paulmedynski dismissed mdaigle’s stale review via cdeb1b1 May 29, 2026 13:41

paulmedynski force-pushed the dev/paul/auth-app branch from ba7bbbb to cdeb1b1 Compare May 29, 2026 13:41

Copilot AI review requested due to automatic review settings May 29, 2026 13:55

Copilot started reviewing on behalf of paulmedynski May 29, 2026 13:55 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

Comment thread tools/PackageCompatibility/test/BuildVersionOverrideTests.cs

Comment thread tools/PackageCompatibility/global.json

paulmedynski marked this pull request as ready for review May 29, 2026 18:10

paulmedynski requested review from a team and Copilot May 29, 2026 18:10

paulmedynski assigned mdaigle and cheenamalhotra May 29, 2026

Copilot started reviewing on behalf of paulmedynski May 29, 2026 18:10 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

Conversation

paulmedynski commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Notes

Maintainer checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cheenamalhotra Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

paulmedynski Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

cheenamalhotra May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mdaigle left a comment

Choose a reason for hiding this comment

Uh oh!

mdaigle May 5, 2026

Choose a reason for hiding this comment

Uh oh!

paulmedynski May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

paulmedynski commented Apr 28, 2026 •

edited

Loading

codecov Bot commented Apr 28, 2026 •

edited

Loading