Quarantine tests

[radical]

Task

Quarantine flaky test(s):

Tests to quarantine:

• Method: Aspire.Hosting.Testing.Tests.TestingBuilderTests.CanOverrideLaunchProfileViaArgsAdHocBuilder
  • Issue: Flaky test: Aspire.Hosting.Testing.Tests.TestingBuilderTests.CanOverrideLaunchProfileViaArgsAdHocBuilder #9712

Background for Quarantined tests

• Tests that are flaky and don't fail deterministically are marked with the QuarantinedTest attribute. The full type name for this is Aspire.TestUtilities.QuarantinedTest.

• Such tests are not run as part of the regular tests workflow (tests.yml).

  • Instead they are run in the Outerloop workflow (tests-outerloop.yml).

• The QuarantinedTest attribute takes an optional reason parameter, typically a GitHub issue URL

• The pattern is consistent: [QuarantinedTest("https://github.com/dotnet/aspire/issues/XXXXX")]

• The attribute is placed right before the test method, after [Fact] or other test attributes

• Tests are quarantined by adding this attribute, which makes them run in the outerloop workflow instead of regular tests

Running tests

(1) Build from the root with build.sh.
(2) If that produces errors, fix those errors and build again. Repeat until the build is successful.
(3) To then run tests, use a command similar to this dotnet.sh test tests/Aspire.Seq.Tests/Aspire.Seq.Tests.csproj (using the path to whatever projects are applicable to the change).
- On Windows use dotnet.cmd instead. These two commands can be found in the root of the repository.