Skip to content

CLI E2E tests that use bare bash (non-Docker) fail with "aspire: command not found" outside PR context #15930

@radical

Description

@radical

Description

Several CLI E2E tests in Aspire.Cli.EndToEnd.Tests are structurally broken when run outside of a PR context — specifically in the quarantine (tests-quarantine.yml) and outerloop (tests-outerloop.yml) scheduled workflows. These tests fail immediately with bash: aspire: command not found because no CLI binary is available on PATH.

Root Cause

The non-Docker CLI E2E tests guard their CLI installation behind IsRunningInCI, which checks for GITHUB_PR_NUMBER and GITHUB_PR_HEAD_SHA environment variables:

// CliE2ETestHelpers.cs
internal static bool IsRunningInCI =>
    !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_NUMBER")) &&
    !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("GITHUB_PR_HEAD_SHA"));

When isCI is false (as in scheduled runs with no PR context), the CLI setup block is skipped entirely — but the test still proceeds to call aspire new, which fails:

if (isCI)
{
    await auto.InstallAspireCliFromPullRequestAsync(prNumber, counter);
    await auto.SourceAspireCliEnvironmentAsync(counter);
    await auto.VerifyAspireCliVersionAsync(commitSha, counter);
}
// No else — no fallback to GA release or any other install method
await auto.AspireNewAsync(ProjectName, counter, useRedisCache: false); // 💥

Docker-based tests don't have this problem because DetectDockerInstallMode() falls back to DockerInstallMode.GaRelease, which downloads the GA release via get-aspire-cli.sh inside the container.

Affected Tests

These quarantined tests consistently fail with aspire: command not found in scheduled runs:

Test Quarantine Issue
KubernetesPublishTests.CreateAndPublishToKubernetes #15870
DockerDeploymentTests.CreateAndDeployToDockerCompose #15882
DockerDeploymentTests.CreateAndDeployToDockerComposeInteractive #15871

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions