Skip to content

Add TypeScript API compatibility check#17103

Merged
sebastienros merged 6 commits into
mainfrom
sebastienros/sebros-typescript-api-compat
May 19, 2026
Merged

Add TypeScript API compatibility check#17103
sebastienros merged 6 commits into
mainfrom
sebastienros/sebros-typescript-api-compat

Conversation

@sebastienros
Copy link
Copy Markdown
Contributor

@sebastienros sebastienros commented May 14, 2026

Description

Adds a PR-side TypeScript API compatibility check for the ATS surface used to generate polyglot AppHost SDKs. This prevents pull requests from introducing undeclared breaking changes after the baseline has been established, while still allowing intentional breaks through explicit suppressions with traceability.

The implementation adds a dedicated TypeScriptApiCompat tool, a reusable CI workflow wired into PR validation, tests covering parsing/comparison/suppressions/reporting, and documentation for baseline resets and suppression format. The check compares the target-branch ATS baseline against fresh aspire sdk dump --format ci output from the PR so baseline edits in the PR cannot hide breaks.

Fixes #17041

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • No
    • No

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 14, 2026 22:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a TypeScript ATS API compatibility checker and wires it into PR validation to detect undeclared breaking changes against the target-branch baseline.

Changes:

  • Adds a new TypeScriptApiCompat tool for parsing ATS surfaces, comparing compatibility, applying suppressions, emitting reports, and GitHub annotations.
  • Adds unit coverage for parsing, comparison, suppressions, and runner/report behavior.
  • Adds CI workflow integration and documentation for baselines and suppressions.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tools/TypeScriptApiCompat/TypeScriptApiCompatRunner.cs Coordinates loading, comparison, suppression, reporting, annotations, and exit codes.
tools/TypeScriptApiCompat/TypeScriptApiCompat.csproj Defines the new compatibility checker tool project.
tools/TypeScriptApiCompat/Program.cs Adds command-line options and invokes the runner.
tools/TypeScriptApiCompat/GitHubAnnotationWriter.cs Emits GitHub Actions error annotations for failures.
tools/TypeScriptApiCompat/CommandLineOptions.cs Defines parsed command-line option values.
tools/TypeScriptApiCompat/AtsSurfaceParser.cs Parses ATS CI dump text into compatibility model objects.
tools/TypeScriptApiCompat/AtsSurface.cs Defines ATS surface records and surface loading.
tools/TypeScriptApiCompat/AtsCompatibilityComparer.cs Implements breaking-change detection between baseline and current surfaces.
tools/TypeScriptApiCompat/ApiCompatSuppression.cs Loads and validates suppression files.
tools/TypeScriptApiCompat/ApiCompatResult.cs Applies suppressions and classifies result state.
tools/TypeScriptApiCompat/ApiCompatReport.cs Produces Markdown compatibility reports.
tools/TypeScriptApiCompat/ApiCompatDiagnostic.cs Defines compatibility diagnostics and suppression keys.
tests/Infrastructure.Tests/TypeScriptApiCompat/TypeScriptApiCompatTests.cs Adds tests for parser, comparer, suppressions, and runner output.
tests/Infrastructure.Tests/Infrastructure.Tests.csproj References the new tool from infrastructure tests.
docs/ci/typescript-api-compat.md Documents baseline behavior, breaking-change kinds, and suppression format.
Aspire.slnx Adds the new tool project to the solution.
.github/workflows/typescript-api-compat.yml Adds reusable PR-side TypeScript API compatibility workflow.
.github/workflows/tests.yml Adds the compatibility workflow to PR test validation and final results.

Comment thread tools/TypeScriptApiCompat/AtsCompatibilityComparer.cs Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 14, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17103

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17103"

sebastienros and others added 2 commits May 14, 2026 15:47
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@sebastienros sebastienros requested a review from joperezr May 14, 2026 23:50
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sebastienros and others added 2 commits May 15, 2026 11:51
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🎬 CLI E2E Test Recordings — 85 recordings uploaded (commit 488de91)

View all recordings
Status Test Recording
AddPackageInteractiveWhileAppHostRunningDetached ▶️ View Recording
AddPackageWhileAppHostRunningDetached ▶️ View Recording
AgentCommands_AllHelpOutputs_AreCorrect ▶️ View Recording
AgentInitCommand_DefaultSelection_InstallsSkillOnly ▶️ View Recording
AgentInitCommand_MigratesDeprecatedConfig ▶️ View Recording
AspireAddPackageVersionToDirectoryPackagesProps ▶️ View Recording
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost ▶️ View Recording
AspireInitWithExistingAppHostDirRecreatesMissingNuGetConfigAndPreservesFiles ▶️ View Recording
AspireInitWithSolutionFileGeneratesAppHostThatBuildsAgainstChannelHive ▶️ View Recording
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps ▶️ View Recording
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent ▶️ View Recording
Banner_DisplayedOnFirstRun ▶️ View Recording
Banner_DisplayedWithExplicitFlag ▶️ View Recording
Banner_NotDisplayedWithNoLogoFlag ▶️ View Recording
CertificatesClean_RemovesCertificates ▶️ View Recording
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate ▶️ View Recording
CertificatesTrust_WithUntrustedCert_TrustsCertificate ▶️ View Recording
ConfigSetGet_CreatesNestedJsonFormat ▶️ View Recording
CreateAndRunAspireStarterProject ▶️ View Recording
CreateAndRunAspireStarterProjectWithBundle ▶️ View Recording
CreateAndRunEmptyAppHostProject ▶️ View Recording
CreateAndRunJavaEmptyAppHostProject ▶️ View Recording
CreateAndRunJsReactProject ▶️ View Recording
CreateAndRunPythonReactProject ▶️ View Recording
CreateAndRunTypeScriptEmptyAppHostProject ▶️ View Recording
CreateAndRunTypeScriptStarterProject ▶️ View Recording
CreateJavaAppHostWithViteApp ▶️ View Recording
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain ▶️ View Recording
DashboardRunWithOtelTracesReturnsNoTraces ▶️ View Recording
DeployK8sBasicApiService ▶️ View Recording
DeployK8sWithExternalHelmChart ▶️ View Recording
DeployK8sWithGarnet ▶️ View Recording
DeployK8sWithMongoDB ▶️ View Recording
DeployK8sWithMySql ▶️ View Recording
DeployK8sWithPostgres ▶️ View Recording
DeployK8sWithRabbitMQ ▶️ View Recording
DeployK8sWithRedis ▶️ View Recording
DeployK8sWithSqlServer ▶️ View Recording
DeployK8sWithValkey ▶️ View Recording
DeployTypeScriptAppToKubernetes ▶️ View Recording
DescribeCommandResolvesReplicaNames ▶️ View Recording
DescribeCommandShowsRunningResources ▶️ View Recording
DetachFormatJsonProducesValidJson ▶️ View Recording
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance ▶️ View Recording
DoListStepsShowsPipelineSteps ▶️ View Recording
DocsCommand_RendersInteractiveMarkdownFromLocalSource ▶️ View Recording
DoctorCommand_DetectsDeprecatedAgentConfig ▶️ View Recording
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain ▶️ View Recording
DoctorCommand_WithSslCertDir_ShowsTrusted ▶️ View Recording
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted ▶️ View Recording
GlobalMigration_HandlesCommentsAndTrailingCommas ▶️ View Recording
GlobalMigration_HandlesMalformedLegacyJson ▶️ View Recording
GlobalMigration_PreservesAllValueTypes ▶️ View Recording
GlobalMigration_SkipsWhenNewConfigExists ▶️ View Recording
GlobalSettings_MigratedFromLegacyFormat ▶️ View Recording
InitTypeScriptAppHost_AugmentsExistingViteRepoAtRoot ▶️ View Recording
InteractiveCSharpInitCreatesExpectedFiles ▶️ View Recording
InvalidAppHostPathWithComments_IsHealedOnRun ▶️ View Recording
LatestCliCanStartStableChannelAppHost ▶️ View Recording
LatestCliCanStartStableChannelTypeScriptAppHost ▶️ View Recording
LegacySettingsMigration_AdjustsRelativeAppHostPath ▶️ View Recording
LogLevelTrace_ProducesTraceEntriesInCliLogFile ▶️ View Recording
LogsCommandShowsResourceLogs ▶️ View Recording
OtelLogsReturnsStructuredLogsFromStarterAppCore ▶️ View Recording
PsCommandListsRunningAppHost ▶️ View Recording
PsFormatJsonOutputsOnlyJsonToStdout ▶️ View Recording
PublishWithConfigureEnvFileUpdatesEnvOutput ▶️ View Recording
PublishWithDockerComposeServiceCallbackSucceeds ▶️ View Recording
PublishWithoutOutputPathUsesAppHostDirectoryDefault ▶️ View Recording
ResourceCommand_FailsWhenInteractionServiceIsRequired ▶️ View Recording
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput ▶️ View Recording
RestoreGeneratesSdkFiles ▶️ View Recording
RestoreGeneratesSdkFiles_WithConfiguredToolchain ▶️ View Recording
RestoreRefreshesGeneratedSdkAfterAddingIntegration ▶️ View Recording
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes ▶️ View Recording
RunFromParentDirectory_UsesExistingConfigNearAppHost ▶️ View Recording
SecretCrudOnDotNetAppHost ▶️ View Recording
SecretCrudOnTypeScriptAppHost ▶️ View Recording
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels ▶️ View Recording
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets ▶️ View Recording
StopAllAppHostsFromAppHostDirectory ▶️ View Recording
StopNonInteractiveSingleAppHost ▶️ View Recording
StopWithNoRunningAppHostExitsSuccessfully ▶️ View Recording
UnAwaitedChainsCompileWithAutoResolvePromises ▶️ View Recording
UpdateProjectChannelToStable_TypeScript_PicksUpStablePackages ▶️ View Recording

📹 Recordings uploaded automatically from CI run #25948640209

Copy link
Copy Markdown
Member

@mitchdenny mitchdenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if some of these mgiht be good to implement as roslyn analyzers?

@IEvangelist
Copy link
Copy Markdown
Member

Built the tool locally and ran it through its paces.

Build

  • dotnet build tools/TypeScriptApiCompat/TypeScriptApiCompat.csproj — clean.
  • dotnet test tests/Infrastructure.Tests --filter-class "*.TypeScriptApiCompatTests" — 5/5 pass.

Dynamic scenarios

Using src/Aspire.Hosting/api/Aspire.Hosting.ats.txt as a synthetic baseline:

Scenario Result
Identical baseline + current exit 0, "No undeclared TypeScript API compatibility breaks were found."
Removed a handle type (BeforeStartEvent) exit 1, 3 diagnostics fired: 1 handle-removed + 2 capability-removed for that handle's members (correctly cascaded)
Same break + matching suppression file (all 3 keys) exit 0, "Suppressed diagnostics: 3"
Malformed suppression line (missing <url>) exit 1, line-pointed error + breaks still reported
Identical baseline + an UNUSED suppression exit 1, "Unused suppressions" diagnostic — verifies stale-entry pressure works

Code review

Walked parser robustness (CRLF/LF/BOM handled via ReplaceLineEndings), suppression matching (exact Kind|PackageName|Symbol key, duplicate detection emits clear errors), comparison logic (additive vs breaking classifications, parameter order/insertions), output handling (encoding, directory creation), GitHub annotation escaping (per spec), CI workflow security (no pull_request_target with PR-controlled inputs; proper quoting), and file enumeration (symlink safety, standard excludes).

No bugs to flag. The mergeable-baseline design (--baseline-suppressions-root allows merged intentional breaks to become "unused" without blocking unrelated PRs) is documented and the tool behaves consistently with that contract.

LGTM. Nice tool.

@sebastienros sebastienros merged commit c1bdb4a into main May 19, 2026
297 checks passed
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 13.4 milestone May 19, 2026
@sebastienros sebastienros deleted the sebastienros/sebros-typescript-api-compat branch May 19, 2026 15:00
@aspire-repo-bot
Copy link
Copy Markdown
Contributor

⚠️ Documentation drafting was attempted but the draft PR could not be confirmed.

See the workflow run for details: https://github.com/microsoft/aspire/actions/runs/26103895726

Added a new TypeScript API compatibility section to src/frontend/src/content/docs/extensibility/multi-language-integration-authoring.mdx. The section explains what the new CI check does, which ATS surface changes are classified as breaking, the suppression file format and location, and a concrete example entry — giving integration authors the information they need to work with intentional breaking changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add TypeScript generated API compatibility validation

4 participants