Add symbol verification to OneBranch pipelines #4073
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||
|---|---|---|---|---|---|---|---|---|
|
|
@@ -133,6 +133,7 @@ When a new issue is created, follow these steps: | |||||||
| - All source code is in `src/Microsoft.Data.SqlClient/src/`. Do NOT add code to legacy `netfx/src/` or `netcore/src/` directories. | ||||||||
| - Only `ref/` folders in `netcore/ref/` and `netfx/ref/` remain active for defining the public API surface. | ||||||||
| - Check for platform-specific differences using file suffixes (`.netfx.cs`, `.netcore.cs`, `.windows.cs`, `.unix.cs`) and conditional compilation (`#if NETFRAMEWORK`, `#if NET`, `#if _WINDOWS`, `#if _UNIX`). | ||||||||
| - Lines of code, comments, and other text should be a maximum of 100 characters (see `policy/coding-style.md`). | ||||||||
|
There was a problem hiding this comment. This new bullet appears to exceed the repo’s 100-character line guideline (policy/coding-style.md:18-19). Consider wrapping it onto two lines to keep the instructions file consistent with the rule it references.
Suggested change
Contributor
Author
There was a problem hiding this comment. I will fix it here if other changes are requested, otherwise it will be fixed naturally once the agents start adhering to our style rules (other PRs are working on this). |
||||||||
| - Respect API compatibility rules across .NET versions | ||||||||
| - Do not introduce breaking changes without proper justification and documentation | ||||||||
| - Use the `doc/` directory for any new documentation or updates to existing documentation | ||||||||
|
|
||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| ################################################################################# | ||
| # Licensed to the .NET Foundation under one or more agreements. # | ||
| # The .NET Foundation licenses this file to you under the MIT license. # | ||
| # See the LICENSE file in the project root for more information. # | ||
| ################################################################################# | ||
|
|
||
| # Validates that symbols (PDBs) for every published package DLL are available on each configured | ||
| # symbol server using the symchk tool. | ||
| # | ||
| # All packages specified in the 'packages' parameter will be checked against all symbol servers | ||
| # specified in the 'symbolServers' parameter. | ||
|
|
||
| parameters: | ||
| # True to enable debug information and steps. | ||
| - name: debug | ||
| type: boolean | ||
|
|
||
| # The list of symbol servers to verify against. Each entry has a 'name' (friendly display name) | ||
| # and 'url' (the symbol server URL that symchk can query). | ||
| - name: symbolServers | ||
| type: object | ||
| default: | ||
| - name: MSDL (Public) | ||
| url: https://msdl.microsoft.com/download/symbols | ||
| - name: SymWeb (Internal) | ||
| url: https://symweb.azurefd.net | ||
|
|
||
| # The packages whose symbols should be verified. Each entry has: | ||
| # artifactName — pipeline artifact name (from onebranch-variables.yml) | ||
| # packageName — NuGet package name prefix used to locate the .nupkg file | ||
| # dllPath — relative path to the DLL inside the extracted package | ||
| - name: packages | ||
| type: object | ||
| default: [] | ||
|
|
||
| jobs: | ||
| - job: validate_symbols | ||
| displayName: Verify symbols on symbol servers | ||
|
|
||
| pool: | ||
| type: windows | ||
| isCustom: true | ||
| name: ADO-1ES-Pool | ||
| vmImage: ADO-Win25 | ||
|
|
||
| variables: | ||
| extractRoot: $(Build.SourcesDirectory)\symchk_packages | ||
|
|
||
| steps: | ||
| - ${{ if parameters.debug }}: | ||
| - script: SET | ||
| displayName: Print Environment Variables | ||
|
|
||
| # ── Download, extract, and verify symbols for each package ────────────── | ||
|
|
||
| - ${{ each pkg in parameters.packages }}: | ||
| - download: current | ||
| artifact: ${{ pkg.artifactName }} | ||
| patterns: '**/*.nupkg' | ||
| displayName: Download ${{ pkg.packageName }} | ||
|
|
||
paulmedynski marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - ${{ each server in parameters.symbolServers }}: | ||
| - task: PowerShell@2 | ||
| displayName: Verify ${{ pkg.packageName }} on ${{ server.name }} | ||
| inputs: | ||
| pwsh: true | ||
| filePath: eng/pipelines/onebranch/jobs/validate-symbols.ps1 | ||
| arguments: > | ||
| -ArtifactPath "$(Pipeline.Workspace)\${{ pkg.artifactName }}" | ||
| -ExtractPath "$(extractRoot)\${{ pkg.packageName }}" | ||
| -PackageName "${{ pkg.packageName }}" | ||
| -DllPath "${{ pkg.dllPath }}" | ||
| -SymbolServerUrl "${{ server.url }}" | ||
| -SymbolServerName "${{ server.name }}" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| <# | ||
| .SYNOPSIS | ||
| Verifies that symbols (PDBs) for a single DLL are available on a symbol server. | ||
|
|
||
| .DESCRIPTION | ||
| This script is called by the validate-symbols-job.yml Azure Pipelines job, | ||
| once per package per symbol server. It: | ||
| 1. Locates the .nupkg file in the downloaded artifact directory. | ||
| 2. Extracts the package contents (skipped if already extracted). | ||
| 3. Runs symchk.exe to verify that matching PDBs are available on the | ||
| specified symbol server. | ||
|
|
||
| The script exits with a non-zero exit code if verification fails. | ||
|
|
||
| .PARAMETER ArtifactPath | ||
| The directory containing the downloaded pipeline artifact (.nupkg files). | ||
|
|
||
| .PARAMETER ExtractPath | ||
| The directory where the package will be extracted. | ||
|
|
||
| .PARAMETER PackageName | ||
| The NuGet package name prefix used to locate the .nupkg file | ||
| (e.g. "Microsoft.Data.SqlClient"). | ||
|
|
||
| .PARAMETER DllPath | ||
| The relative path to the DLL inside the extracted package | ||
| (e.g. "lib\net8.0\Microsoft.Data.SqlClient.dll"). | ||
|
|
||
| .PARAMETER SymbolServerUrl | ||
| The symbol server URL that symchk can query | ||
| (e.g. "https://msdl.microsoft.com/download/symbols"). | ||
|
|
||
| .PARAMETER SymbolServerName | ||
| A friendly display name for the symbol server, used in log output. | ||
|
|
||
| .PARAMETER MaxRetries | ||
| Maximum number of attempts when symbols are not yet available. The | ||
| first attempt runs immediately; subsequent attempts wait | ||
| RetryIntervalSeconds between them. Defaults to 10 (~5 minutes total | ||
| with default interval). | ||
|
|
||
| .PARAMETER RetryIntervalSeconds | ||
| Seconds to wait between retry attempts (default 30). | ||
|
|
||
| .EXAMPLE | ||
| .\validate-symbols.ps1 ` | ||
| -ArtifactPath "C:\agent\_work\1\drop_SqlClient" ` | ||
| -ExtractPath "C:\agent\_work\1\s\symchk_packages\Microsoft.Data.SqlClient" ` | ||
| -PackageName "Microsoft.Data.SqlClient" ` | ||
| -DllPath "lib\net8.0\Microsoft.Data.SqlClient.dll" ` | ||
| -SymbolServerUrl "https://msdl.microsoft.com/download/symbols" ` | ||
| -SymbolServerName "MSDL (Public)" | ||
| #> | ||
|
|
||
| [CmdletBinding()] | ||
| param( | ||
| [Parameter(Mandatory)] | ||
| [string]$ArtifactPath, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [string]$ExtractPath, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [string]$PackageName, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [string]$DllPath, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [string]$SymbolServerUrl, | ||
|
|
||
| [Parameter(Mandatory)] | ||
| [string]$SymbolServerName, | ||
|
|
||
| [int]$MaxRetries = 10, | ||
|
|
||
| [int]$RetryIntervalSeconds = 30 | ||
| ) | ||
|
|
||
| Set-StrictMode -Version Latest | ||
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| # -- Extract the package (skip if already done) -------------------------------- | ||
|
|
||
| $dllFullPath = Join-Path $ExtractPath $DllPath | ||
|
|
||
| if (-not (Test-Path $dllFullPath)) { | ||
| Write-Host "Extracting $PackageName" | ||
|
|
||
| New-Item -ItemType Directory -Force -Path $ExtractPath | Out-Null | ||
|
|
||
| $nupkg = Get-ChildItem -Path $ArtifactPath -Filter "$PackageName.*.nupkg" ` | ||
| | Where-Object { $_.Name -notlike '*.snupkg' } ` | ||
| | Select-Object -First 1 | ||
|
|
||
| if (-not $nupkg) { | ||
| Write-Host "##vso[task.logissue type=error]No $PackageName nupkg found in $ArtifactPath" | ||
| exit 1 | ||
| } | ||
|
|
||
| Write-Host "Found: $($nupkg.FullName)" | ||
|
|
||
| $zipPath = Join-Path $ExtractPath 'package.zip' | ||
| Copy-Item $nupkg.FullName $zipPath | ||
| Expand-Archive -Path $zipPath -DestinationPath $ExtractPath -Force | ||
| Remove-Item $zipPath | ||
| } | ||
|
|
||
| if (-not (Test-Path $dllFullPath)) { | ||
| Write-Host "##vso[task.logissue type=error]DLL not found after extraction: $dllFullPath" | ||
| exit 1 | ||
| } | ||
|
|
||
| # -- Locate symchk.exe --------------------------------------------------------- | ||
|
|
||
| $symchkCandidates = @( | ||
| "${env:ProgramFiles(x86)}\Windows Kits\10\Debuggers\x64\symchk.exe" | ||
| "${env:ProgramFiles}\Windows Kits\10\Debuggers\x64\symchk.exe" | ||
| ) | ||
|
|
||
| $symchkPath = $null | ||
| foreach ($candidate in $symchkCandidates) { | ||
| if (Test-Path $candidate) { | ||
| $symchkPath = $candidate | ||
| break | ||
| } | ||
| } | ||
|
|
||
| if (-not $symchkPath) { | ||
| Write-Host "##vso[task.logissue type=error]symchk.exe not found. Ensure Debugging Tools for Windows are installed." | ||
| exit 1 | ||
| } | ||
|
|
||
| # -- Verify symbols (with retries for publishing latency) ---------------------- | ||
|
|
||
| $dllLeaf = Split-Path $dllFullPath -Leaf | ||
|
|
||
| Write-Host "Verifying symbols for $dllLeaf on $SymbolServerName ($SymbolServerUrl)" | ||
| Write-Host "Using symchk: $symchkPath" | ||
| Write-Host "Max attempts: $MaxRetries, interval: ${RetryIntervalSeconds}s" | ||
|
|
||
| $symchkArgs = @( | ||
| $dllFullPath, | ||
| "/s", "srv*$SymbolServerUrl", | ||
| "/os" | ||
| ) | ||
|
|
||
| for ($attempt = 1; $attempt -le $MaxRetries; $attempt++) { | ||
| Write-Host "Attempt $attempt of $MaxRetries -- running: symchk $($symchkArgs -join ' ')" | ||
| $output = & $symchkPath @symchkArgs 2>&1 | Out-String | ||
| $symchkExit = $LASTEXITCODE | ||
|
|
||
| Write-Host $output | ||
|
|
||
| $passed = ($symchkExit -eq 0) -and | ||
| ($output -match "FAILED files = 0") -and | ||
| ($output -match "PASSED \+ IGNORED files = [1-9]") | ||
|
|
||
| if ($passed) { | ||
| Write-Host "Symbols verified successfully for $dllLeaf on $SymbolServerName" | ||
| exit 0 | ||
| } | ||
|
|
||
| if ($attempt -lt $MaxRetries) { | ||
| Write-Host "Symbols not yet available. Retrying in $RetryIntervalSeconds seconds..." | ||
| Start-Sleep -Seconds $RetryIntervalSeconds | ||
| } | ||
| } | ||
|
|
||
| Write-Host "##vso[task.logissue type=error]symchk could not verify symbols for $dllLeaf on $SymbolServerName after $MaxRetries attempts." | ||
| exit 1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot has some built-in settings that specify 80 chars max, and it stores it locally (i.e. somewhere in my user directory). It claims that this will ensure that it generates text with our desired line length.