infra: publish Aspire CLI native AOT symbols (Win + Linux + macOS) to MSDL

[radical]

The Aspire CLI ships as a NativeAOT executable but its native debug symbols never reach MSDL/SymWeb. dotnet symbol --symbols against a shipped aspire (any platform) returns nothing, so customers and our own crash triage can't symbolicate stack traces from the CLI binary on any of Windows, Linux, or macOS.

$ dotnet symbol --symbols aspire.exe -o ./syms
Downloading from https://msdl.microsoft.com/download/symbols/
ERROR: Not Found

Root cause

ILC emits a symbol artifact next to the binary at artifacts/bin/Aspire.Cli/<config>/net10.0/<rid>/native/ on every NativeAOT build (aspire.pdb on Windows, aspire.dbg on Linux, aspire.dSYM/ on macOS — per Microsoft.NETCore.Native.targets), but it is then dropped on the floor:

• <CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory> in src/Aspire.Cli/Aspire.Cli.csproj keeps it out of the publish dir.
• The clipack archive (eng/clipack/Common.projitems) only stages the binary itself into the per-RID aspire-cli-<rid>.<zip|tar.gz>.
• build_sign_native.yml only publishes packages/ as the native_archives_<rid> pipeline artifact, so the symbol artifact never reaches the downstream build stage.
• The Windows build stage runs BuildAndTest.yml with /p:SkipNativeBuild=true, so it never produces the symbol artifact locally either.

End result: arcade's symbol-publishing infrastructure has no file to upload, even though it is fully wired up.

The fix

Arcade has two distinct symbol-publishing pipelines with different shape requirements; we use each for the platforms it supports:

• Windows .pdb — loose-file path via FilesToPublishToSymbolServer (Publish.proj GatherPublishItems). Arcade's PrepLoosePdbsForPublish hard-filters loose files to .pdb/.dll, so this path is Windows-only.
• Linux .dbg and macOS .dwarf — wrapped in a NuGet symbol package and routed via arcade's _ExistingSymbolPackage filter. SymbolUploadHelper.AddPackageToRequest opens the .symbols.nupkg with raw ZipFile.Open (SymbolUploadHelper.cs#L273), filters entries by an extension allowlist that includes .dbg/.dwarf/.so/.dylib (SymbolUploadHelper.cs#L37), and indexes with symbol.exe adddirectory. The symbol-server key is computed from the file's intrinsic build-id — ELF .note.gnu.build-id on Linux, Mach-O LC_UUID on macOS — giving SSQP keys <name>.dbg/elf-buildid-sym-<id>/_.debug (Linux) and _.dwarf/mach-uuid-sym-<uuid>/_.dwarf (macOS) that dotnet-symbol resolves on lookup.

dotnet/runtime uses this exact same path for CoreCLR and libraries native symbols:

• liveBuilds.targets#L122-L141 — CoreCLR shared framework RuntimeFiles ItemGroup includes *.pdb;*.dbg;*.dwarf.
• functions.cmake#L362-L431 — defaults dsymutil to its flat-DWARF output mode for macOS, producing flat .dwarf files (libcoreclr.dylib.dwarf etc.) instead of .dSYM bundles.
• Microsoft.DotNet.ILCompiler.pkgproj#L54-L63 — LibPackageExcludes for .dbg/.dwarf/.dSYM with the comment "exclude native symbols from ilc package (they are included in symbols package)".

The macOS lookup side is implemented in MachOKeyGenerator.cs, with the protocol documented in SSQP_Key_Conventions.md.

Three coordinated edits plumb the symbol artifacts from build_sign_native into the build stage's working directory before arcade's -publish runs:

• eng/pipelines/templates/build_sign_native.yml —
  • Windows agents stage aspire.pdb into artifacts/native-symbols-staging/<rid>/.
  • Linux agents pack aspire.dbg into Aspire.Cli.<rid>.<version>.symbols.nupkg using a minimal hand-built zip + nuspec.
  • macOS agents extract the inner Mach-O DWARF from aspire.dSYM/Contents/Resources/DWARF/aspire, ship it as aspire.dwarf in the same .symbols.nupkg shape. The file's LC_UUID matches the binary's, so dotnet-symbol's Mach-O lookup resolves to it.
  • All three platforms publish under the new per-RID pipeline artifact native_symbols_<rid>.
• eng/pipelines/azure-pipelines.yml and azure-pipelines-unofficial.yml — Windows build job adds two DownloadPipelineArtifact@2 tasks: **/aspire.pdb into artifacts/native-symbols/ (consumed by the FilesToPublishToSymbolServer glob), and **/Aspire.Cli.*.symbols.nupkg into artifacts/native-symbol-pkgs/. A subsequent pwsh step copies the symbol packages into packages/<config>/Shipping so arcade's manifest generation picks them up as Symbols assets. The existing **/Aspire.Cli*.nupkg download is tightened to exclude .symbols.nupkg so the same file isn't downloaded twice.
• eng/Publishing.props — project-level FilesToPublishToSymbolServer glob for the Windows pdbs only (the loose-file path is .pdb/.dll-only). Linux and macOS symbol packages flow through arcade's existing .symbols.nupkg routing without needing a separate property.

Why this approach (and not the alternatives)

• The .dSYM directory bundle is not separately published. The Apple-native automatic symbolication path (lldb / atos / Instruments via Spotlight UUID indexing) needs the bundle form and is tracked by Distribute macOS symbols as dSYM, not .dwarf dotnet/runtime#88286. For server-mediated symbolication via dotnet-symbol — the primary CLI crash-triage workflow — the flat .dwarf we ship is the working format that dotnet/runtime itself uses.
• AutoGenerateSymbolPackages stays false. That property controls arcade's managed-PDB → .symbols.nupkg wrapper for shipping NuGet packages, independent of the symbol publishing here.
• Not unsetting <CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>. Globbing directly from bin/<rid>/native/ avoids re-triggering the SymStore race that the comment at eng/clipack/Common.projitems warns about for the managed pdb. Keeps a clean separation between the managed-pdb path (suppressed) and the native-pdb path (this PR).
• Hand-built .symbols.nupkg rather than a NuGet Pack invocation. Arcade's SymbolUploadHelper opens the package with raw ZipFile.Open, not NuGet OPC validation, so OPC compliance is unnecessary. A minimal .nuspec + symbol payload zip is sufficient and avoids adding a NuGet Pack task to the build_sign_native job (and the cross-platform tooling that would require).

Surprises and call-outs

• CI structurally required build_sign_native → pipeline artifact → build job download for symbols to be on disk when -publish runs. A Publishing.props glob alone could not pick them up — the files aren't on the publishing agent without this plumbing.
• This change cannot be PR-validated through GitHub Actions: azure-pipelines-public.yml does not run build_sign_native. Verified end-to-end on internal AzDO build 2985850 for the Windows path (an earlier revision): both 🟣Stage native AOT pdb steps succeed; the Windows build job's -publish log emits Uploading 'PdbArtifacts/Windows/native_symbols_win_<arch>/aspire.pdb' to the BAR with distinct relative paths per RID, keeping the same-named pdbs apart on MSDL via RelativePDBPath keying. A fresh AzDO build is in flight to exercise the Linux + macOS paths.
• Path shape for the Linux .dbg is documented in Microsoft.NETCore.Native.targets (NativeOutputPath = $(OutputPath)native\, NativeSymbolExt = .dbg on Linux, StripSymbols=true default on non-Windows so debug info is split into the .dbg sidecar via objcopy --only-keep-debug + --add-gnu-debuglink).
• macOS payload shape (aspire.dSYM/Contents/Resources/DWARF/aspire) confirmed locally on macOS NativeAOT publish; dwarfdump --uuid showed the inner file's UUID matches the binary's, which is the contract MachOFileKeyGenerator relies on for symbol-server lookup.

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17567

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17567"

[joperezr]

Assuming we have done a dry run and this works.

[github-actions]

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

• Tests / Hosting.Docker / Hosting.Docker (ubuntu-latest) - Failed step 'Run tests (Linux/macOS)' will be retried because the job log matched a configurable test-retry pattern: MCR registry request blocked (rate limiting)

[github-actions]

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

• Tests / Hosting-1 / Hosting-1 (windows-latest) - Job-level runner or infrastructure failure matched the transient allowlist.

[davidfowl]

While you're here, we also want to upload symbols for aspire-managed.exe

[radical]

aspire-managed.exe already ships with embedded PDBs — the repo defaults to DebugType=embedded (Directory.Build.props:23), so each .dll carries its portable PDB inside the PE debug directory (entry type 17). That's the alternative to MSDL upload, not a precursor; debuggers find the symbols directly in the binary without a server round-trip.

Verified against ~/.aspire/versions/13.4.0_9c260c29a6.../: 4 Aspire-owned assemblies have both an embedded PDB (debug-dir type 17) and a CodeView record (type 2) — aspire-managed.dll, Aspire.Dashboard.dll, Aspire.Hosting.RemoteHost.dll, Aspire.TypeSystem.dll. Runtime BCL .dlls use DebugType=portable + MSDL (CodeView-only), which is the route this PR sets up for the NativeAOT CLI binary (no embedded option exists for native AOT).

[DamianEdwards]

aspire-managed.exe already ships with embedded PDBs — the repo defaults to DebugType=embedded (Directory.Build.props:23), so each .dll carries its portable PDB inside the PE debug directory (entry type 17). That's the alternative to MSDL upload, not a precursor; debuggers find the symbols directly in the binary without a server round-trip.

We should consider changing this for the bundle arguably, i.e. optimize for distribution/layout size rather than ease of debuggability. For NuGet packages I think embedded debug symbols is generally the right trade-off, but we don't expect end-users to be debugging the managed host or other parts of the bundle, e.g. dashboard, DCP, etc., and even if they do, they can download the symbols from the symbol store.