Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f238486
Merge | Pack Mds Target, Wire PR/CI Pipelines to Build2.proj (#4033)
benrr101 Mar 19, 2026
de86f7a
Migrate to slnx solution file format (#4070)
paulmedynski Mar 20, 2026
d359511
Eng | Disable ESRP steps on non-official pipelines (#4077)
cheenamalhotra Mar 25, 2026
171765a
Test | Fix Transient Fault handling and other flaky unit tests (#4080)
cheenamalhotra Mar 25, 2026
1013cf3
Added CI test stages for SQL Server 16 and 17. (#4087)
paulmedynski Mar 25, 2026
721859a
Test | Fix Diagnostic Tests and remove Remote Executor (#4078)
cheenamalhotra Mar 26, 2026
f58d7a8
Create stress test pipeline (#3867)
paulmedynski Mar 27, 2026
3a7e5c2
Changed SBOM package version to use build number (#4095)
paulmedynski Mar 30, 2026
b780ad1
Merge | Official Builds From Common Project (#4068)
benrr101 Mar 31, 2026
c85343c
Trim docs using pwsh via dotnet tool to support cross-platform local …
mdaigle Apr 1, 2026
3fcde6a
Revert "Trim docs using pwsh via dotnet tool to support cross-platfor…
mdaigle Apr 1, 2026
e7b0854
Test | Connection pool transaction tests (#3805)
mdaigle Apr 2, 2026
9aad4ec
Add GitHub Actions workflow to automate cherry-pick hotfix PRs (#4106)
paulmedynski Apr 6, 2026
26d20f6
Fix milestone lookup in hotfix workflow (#4148)
paulmedynski Apr 7, 2026
614b410
Update prompt tools. (#4132)
mdaigle Apr 8, 2026
f113008
Official Build Cleanup (#4123)
benrr101 Apr 8, 2026
0824970
Tests | Add collation transcoding tests (#4051)
edwardneal Apr 8, 2026
3877beb
Fix typo in "Received instance info from UDP Client." (#4103)
SimonCropp Apr 8, 2026
c0b0627
Official Build Cleanup 2 (#4155)
benrr101 Apr 9, 2026
78634a3
Merge | Cleanup (#4145)
benrr101 Apr 9, 2026
4b99276
Tests | Introduce RAII SQL object primitives (#4050)
edwardneal Apr 17, 2026
67fa5c2
Add conditional ProjectReference to Microsoft.SqlServer.Server (#4003)
edwardneal Apr 17, 2026
f4e9483
Fix review comments: stale XML doc param, missing ConditionalTheory g…
paulmedynski Apr 20, 2026
0b54d6b
- Updated all version numbers to reflect the 7.0.1 release.
paulmedynski Apr 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"ms-mssql.mssql"
],
"settings": {
"dotnet.defaultSolution": "src/Microsoft.Data.SqlClient.sln"
"dotnet.defaultSolution": "src/Microsoft.Data.SqlClient.slnx"
}
}
},
Expand Down
4 changes: 4 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ When a new issue is created, follow these steps:
- Suggest release note entries for fixes by updating files under `release-notes/` or by using the `release-notes` prompt (instead of editing `CHANGELOG.md` directly).
- Tag reviewers based on `CODEOWNERS` file

## 🌿 Branch Naming
- All branches created by AI agents **must** use the `dev/automation/` prefix (e.g. `dev/automation/fix-connection-timeout`).
- Do **not** create branches directly under `main`, `dev/`, or any other top-level prefix.

## 🧠 Contextual Awareness
- All source code is in `src/Microsoft.Data.SqlClient/src/`. Do NOT add code to legacy `netfx/src/` or `netcore/src/` directories.
- Only `ref/` folders in `netcore/ref/` and `netfx/ref/` remain active for defining the public API surface.
Expand Down
292 changes: 109 additions & 183 deletions .github/instructions/ado-pipelines.instructions.md
Original file line number Diff line number Diff line change
@@ -1,195 +1,121 @@
---
applyTo: "eng/pipelines/**/*.yml"
---
# Azure DevOps Pipelines Guide

## Overview

This repository uses Azure DevOps Pipelines for CI/CD. The pipeline configurations are located in `eng/pipelines/`.

**ADO Organization**: sqlclientdrivers
**ADO Project**: ADO.NET

## Pipeline Structure

```
eng/pipelines/
├── abstractions/ # Abstractions package pipelines
├── azure/ # Azure package pipelines
├── common/ # Shared templates
│ └── templates/
│ ├── jobs/ # Reusable job templates
│ ├── stages/ # Reusable stage templates
│ └── steps/ # Reusable step templates
├── jobs/ # Top-level job definitions
├── libraries/ # Shared variable definitions
├── stages/ # Stage definitions
├── steps/ # Step definitions
├── variables/ # Variable templates
├── akv-official-pipeline.yml # AKV provider official/signing build
├── dotnet-sqlclient-ci-core.yml # Core CI pipeline (reusable)
├── dotnet-sqlclient-ci-package-reference-pipeline.yml # CI with package references
├── dotnet-sqlclient-ci-project-reference-pipeline.yml # CI with project references
├── dotnet-sqlclient-signing-pipeline.yml # Package signing pipeline
├── sqlclient-pr-package-ref-pipeline.yml # PR validation (package ref)
├── sqlclient-pr-project-ref-pipeline.yml # PR validation (project ref)
└── stress-tests-pipeline.yml # Stress testing
```

## Main Pipelines

### CI Core Pipeline (`dotnet-sqlclient-ci-core.yml`)
Reusable core CI pipeline consumed by both project-reference and package-reference CI pipelines. Configurable parameters:

| Parameter | Description | Default |
|-----------|-------------|---------|
| `targetFrameworks` | Windows test frameworks | `[net462, net8.0, net9.0, net10.0]` |
| `targetFrameworksUnix` | Unix test frameworks | `[net8.0, net9.0, net10.0]` |
| `referenceType` | Project or Package reference | Required |
| `buildConfiguration` | Debug or Release | Required |
| `useManagedSNI` | Test with managed SNI | `[false, true]` |
| `testJobTimeout` | Test job timeout (minutes) | Required |
| `runAlwaysEncryptedTests` | Include AE tests | `true` |
| `enableStressTests` | Include stress test stage | `false` |

### CI Reference Pipelines
- `dotnet-sqlclient-ci-project-reference-pipeline.yml` — Full CI using project references (builds from source)
- `dotnet-sqlclient-ci-package-reference-pipeline.yml` — Full CI using package references (tests against published NuGet packages)

### PR Validation Pipelines
- `sqlclient-pr-project-ref-pipeline.yml` — PR validation with project references
- `sqlclient-pr-package-ref-pipeline.yml` — PR validation with package references

These pipelines trigger on pull requests and run a subset of the full CI matrix to provide fast feedback.

### Official/Signing Pipeline (`dotnet-sqlclient-signing-pipeline.yml`)
Signs and publishes NuGet packages. Used for official releases. Requires secure service connections and key vault access for code signing.

### AKV Official Pipeline (`akv-official-pipeline.yml`)
Builds and signs the `Microsoft.Data.SqlClient.AlwaysEncrypted.AzureKeyVaultProvider` add-on package separately from the main driver. Uses 1ES pipeline templates for compliance.

### Stress Tests Pipeline (`stress-tests-pipeline.yml`)
Optional pipeline for long-running stress and endurance testing. Enabled via `enableStressTests` parameter in CI core.

## Build Stages

1. **build_abstractions_package_stage**: Build and pack abstractions
2. **build_sqlclient_package_stage**: Build main driver and AKV packages
3. **build_azure_package_stage**: Build Azure extensions package
4. **stress_tests_stage**: Optional stress testing
5. **run_tests_stage**: Execute all test suites
# Azure DevOps CI/CD Pipeline Guidelines

## Purpose

Rules and conventions for editing the Azure DevOps CI/CD pipelines that build, test, and validate Microsoft.Data.SqlClient. These pipelines live under `eng/pipelines/` (excluding `onebranch/`, which is covered by separate instructions).

**ADO Organization**: sqlclientdrivers | **ADO Project**: ADO.NET

## Pipeline Layout

Two categories of pipelines exist in this repository:

- **CI/PR pipelines** (`eng/pipelines/`) — Build, test, and validate on every push/PR
- **OneBranch pipelines** (`eng/pipelines/onebranch/`) — Official signing/release builds (separate instructions file)

Top-level CI/PR pipeline files:
- `dotnet-sqlclient-ci-core.yml` — Reusable core template; all CI and PR pipelines extend this
- `dotnet-sqlclient-ci-package-reference-pipeline.yml` — CI with Package references (Release)
- `dotnet-sqlclient-ci-project-reference-pipeline.yml` — CI with Project references (Release)
- `sqlclient-pr-package-ref-pipeline.yml` — PR validation with Package references
- `sqlclient-pr-project-ref-pipeline.yml` — PR validation with Project references
- `stress-tests-pipeline.yml` — Stress tests triggered after successful CI-Package runs

Reusable templates are organized under:
- `common/templates/jobs/` — Job templates (`ci-build-nugets-job`, `ci-code-coverage-job`, `ci-run-tests-job`)
- `common/templates/stages/` — Stage templates (`ci-run-tests-stage`)
- `common/templates/steps/` — Step templates (build, test, config, publish)
- `jobs/` — Package-specific CI jobs (pack/test Abstractions, Azure, Logging, stress)
- `stages/` — Package-specific CI stages (build Logging → Abstractions → SqlClient → Azure → verify → stress)
- `libraries/` — Shared variables (`ci-build-variables.yml`)
- `steps/` — SDK install steps

## CI Core Template

`dotnet-sqlclient-ci-core.yml` is the central orchestrator. All CI and PR pipelines extend it with different parameters.

Key parameters:
- `referenceType` (required) — `Package` or `Project`; controls how sibling packages are referenced
- `buildConfiguration` (required) — `Debug` or `Release`
- `testJobTimeout` (required) — test job timeout in minutes
- `targetFrameworks` — Windows test TFMs; default `[net462, net8.0, net9.0, net10.0]`
- `targetFrameworksUnix` — Unix test TFMs; default `[net8.0, net9.0, net10.0]`
- `testSets` — test partitions; default `[1, 2, 3]`
- `useManagedSNI` — SNI variants to test; default `[false, true]`
- `runAlwaysEncryptedTests` — include AE test set; default `true`
- `enableStressTests` — enable stress test stage; default `false`
- `debug` — enable debug output; default `false`
- `dotnetVerbosity` — MSBuild verbosity; default `normal`

## Build Stage Order

Stages execute in dependency order (Package reference mode requires artifacts from prior stages):
1. `generate_secrets` — Generate test secrets
2. `build_logging_package_stage` — Build Logging package
3. `build_abstractions_package_stage` — Build Abstractions (depends on Logging)
4. `build_sqlclient_package_stage` — Build SqlClient + AKV Provider (depends on Abstractions + Logging)
5. `build_azure_package_stage` — Build Azure extensions (depends on Abstractions + Logging + SqlClient)
6. `verify_nuget_packages_stage` — Verify NuGet package metadata
7. `stress_tests_stage` — Optional stress tests
8. `ci_run_tests_stage` — Run MDS and AKV test suites

When adding a new build stage, respect the dependency graph and pass artifact names/versions to downstream stages.

## PR vs CI Pipeline Differences

PR pipelines:
- Trigger on PRs to `dev/*`, `feat/*`, `main`; exclude `eng/pipelines/onebranch/*` paths
- Use reduced TFM matrix: `[net462, net8.0, net9.0]` (excludes net10.0)
- Timeout: 90 minutes
- Package-ref PR disables Always Encrypted tests in Debug config

CI pipelines:
- Trigger on push to `main` (GitHub) and `internal/main` (ADO) with `batch: true`
- Scheduled weekday builds (see individual pipeline files for cron times)
- Full TFM matrix including net10.0

## Test Configuration

### Test Sets
Tests are divided into sets for parallelization:
- `TestSet=1` — First partition of tests
- `TestSet=2` — Second partition
- `TestSet=3` — Third partition
- `TestSet=AE` — Always Encrypted tests

### Test Filters
Tests use category-based filtering. The default filter excludes both `failing` and `flaky` tests:
```
category!=failing&category!=flaky
```

Category values:
- `nonnetfxtests` — Excluded on .NET Framework
- `nonnetcoreapptests` — Excluded on .NET Core
- `nonwindowstests` — Excluded on Windows
- `nonlinuxtests` — Excluded on Linux
- `failing` — Known permanent failures (excluded from all runs)
- `flaky` — Intermittently failing tests (quarantined, run separately)

### Flaky Test Quarantine in Pipelines
Quarantined tests (`[Trait("Category", "flaky")]`) run in **separate pipeline steps** after the main test steps. This ensures:
- Main test runs are **not blocked** by intermittent failures
- Flaky tests are still **monitored** for regression or resolution
- Code coverage is **not collected** for flaky test runs
- Results appear in pipeline output for visibility

The quarantine steps are configured in:
- `eng/pipelines/common/templates/steps/build-and-run-tests-netcore-step.yml`
- `eng/pipelines/common/templates/steps/build-and-run-tests-netfx-step.yml`
- `eng/pipelines/common/templates/steps/run-all-tests-step.yml`

### Test Timeout
All test runs use `--blame-hang-timeout 10m` (configured in `build.proj`). Tests exceeding 10 minutes are killed and reported as failures.

### SNI Testing
The `useManagedSNI` parameter controls testing with:
- Native SNI (`false`) - Windows native library
- Managed SNI (`true`) - Cross-platform managed implementation
Test partitioning:
- Tests split into `TestSet=1`, `TestSet=2`, `TestSet=3` for parallelization
- `TestSet=AE` — Always Encrypted tests (controlled by `runAlwaysEncryptedTests`)

## Variables

### Build Variables (`ci-build-variables.yml`)
Common build configuration:
- Package versions
- Build paths
- Signing configuration

### Runtime Variables
Set via pipeline parameters or UI:
- `Configuration` - Debug/Release
- `Platform` - AnyCPU/x86/x64
- `TF` - Target framework

## Creating Pipeline Changes

### Adding New Test Categories
1. Add category attribute to tests: `[Category("newcategory")]`
2. Update filter expressions in test job templates
3. Document category purpose in test documentation

### Adding New Pipeline Parameters
1. Define parameter in appropriate `.yml` file
2. Add to parameter passing in calling templates
3. Document in this file

### Modifying Build Steps
1. Changes should be made in template files for reusability
2. Test changes locally when possible
3. Submit as PR - validation will run
Test filters — default excludes `failing` and `flaky` categories:
- `failing` — known permanent failures, always excluded
- `flaky` — intermittent failures, quarantined in separate pipeline steps
- `nonnetfxtests` / `nonnetcoreapptests` — platform-specific exclusions
- `nonwindowstests` / `nonlinuxtests` — OS-specific exclusions

## Best Practices
Flaky test quarantine:
- Quarantined tests (`[Trait("Category", "flaky")]`) run in separate steps after main tests
- Main test runs are not blocked by flaky failures
- No code coverage collected for flaky runs
- Configured in `common/templates/steps/build-and-run-tests-netcore-step.yml`, `build-and-run-tests-netfx-step.yml`, and `run-all-tests-step.yml`

### Template Design
- Use templates for reusable definitions
- Pass parameters explicitly (avoid global variables)
- Use descriptive stage/job/step names
SNI testing — `useManagedSNI` controls testing with native SNI (`false`) or managed SNI (`true`)

### Variable Management
- Use template variables for shared values
- Use pipeline parameters for per-run configuration
- Avoid hardcoding versions (use Directory.Packages.props)
Test timeout — `--blame-hang-timeout 10m` (configured in `build.proj`); tests exceeding 10 minutes are killed

### Test Infrastructure
- Ensure tests are properly categorized
- Handle test configuration files properly
- Use test matrix for cross-platform coverage

## Troubleshooting

### Common Issues
1. **Test failures due to missing config**: Ensure `config.json` exists
2. **Platform-specific failures**: Check platform exclusion categories
3. **Timeout issues**: Increase `testJobTimeout` parameter

### Debugging Pipelines
- Enable debug mode via `debug: true` parameter
- Use `dotnetVerbosity: diagnostic` for detailed output
- Check build logs in Azure DevOps

## Security Considerations

- Pipelines use service connections for artifact publishing
- Signing uses secure key vault integration
- Sensitive configuration should use pipeline secrets
- Never commit credentials in pipeline files

## Related Documentation
## Variables

- [BUILDGUIDE.md](../../BUILDGUIDE.md) - Local build instructions
- [Azure DevOps Documentation](https://learn.microsoft.com/azure/devops/pipelines/)
- All CI build variables centralized in `libraries/ci-build-variables.yml`
- Package versions use `-ci` suffix (e.g., `7.0.0.$(Build.BuildNumber)-ci`)
- `assemblyBuildNumber` derived from first segment of `Build.BuildNumber` (16-bit safe)
- `localFeedPath` = `$(Build.SourcesDirectory)/packages` — local NuGet feed for inter-package deps
- `packagePath` = `$(Build.SourcesDirectory)/output` — NuGet pack output

## Conventions When Editing Pipelines

- Always use templates for reusable logic — do not inline complex steps
- Pass parameters explicitly; avoid relying on global variables
- Use descriptive stage/job/step display names
- When adding parameters, define them in the core template and thread through calling pipelines
- When adding test categories, update filter expressions in test step templates
- PR pipelines should run a minimal matrix for fast feedback
- Test changes via PR pipeline first — validation runs automatically
- Enable `debug: true` and `dotnetVerbosity: diagnostic` for troubleshooting
- Never commit credentials or secrets in pipeline files
- Signing and release are handled by OneBranch pipelines — not these CI/PR pipelines
Loading
Loading