Skip to content

[MTP Integration Analysis] Retry accepts negative numeric values and can crash before the first attempt #8432

@Evangelink

Description

@Evangelink

Summary

Microsoft.Testing.Extensions.Retry accepts negative integer values for --retry-failed-tests, --retry-failed-tests-max-percentage, and --retry-failed-tests-max-tests. A negative retry count can skip the retry loop entirely and then crash when the orchestrator indexes an empty exitCodes list.

Evidence

  • src/Platform/Microsoft.Testing.Extensions.Retry/RetryCommandLineOptionsProvider.cs:71-83
    • if (commandOption.Name == RetryFailedTestsOptionName && !int.TryParse(arguments[0], out int _))
    • if (commandOption.Name == RetryFailedTestsMaxPercentageOptionName && !int.TryParse(arguments[0], out int _))
    • if (commandOption.Name == RetryFailedTestsMaxTestsOptionName && !int.TryParse(arguments[0], out int _))
  • src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs:168-170
    • while (attemptCount < userMaxRetryCount + 1)
  • src/Platform/Microsoft.Testing.Extensions.Retry/RetryOrchestrator.cs:418-453
    • if (exitCodes[^1] != (int)ExitCode.Success)
    • return exitCodes[^1];

Why this is a real issue

Passing --retry-failed-tests -1 satisfies the current validation because it is still an integer. That makes userMaxRetryCount + 1 equal 0, so the while loop never executes, exitCodes stays empty, and the later exitCodes[^1] access throws instead of returning a validation error. The same missing range validation also lets negative threshold values through, which produces nonsensical retry-threshold behavior instead of rejecting the command line up front.

Suggested resolution

Validate these options as non-negative integers before execution starts. For --retry-failed-tests, require >= 0; for threshold options, require values in a documented range (for example >= 0, and likely <= 100 for percentages). Add acceptance coverage for negative values so the failure stays in validation instead of surfacing as an orchestrator crash.

Related issues

Metadata

Metadata

Labels

area/mtp-extensionsMTP extensions (TrxReport, Retry, HtmlReport, ...).type/automationCreated or maintained by an agentic workflow.

Type

No type
No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions