[MTP Reporters Analysis] AzureDevOps live publishing drops dequeued results on publish failure

[Evangelink]

Summary

AzureDevOpsTestResultsPublisher permanently loses a batch of live test results when PublishTestResultsAsync fails. The code dequeues results before publishing them and only logs on failure; it never requeues or preserves the failed batch for the forced final flush.

Evidence

• src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsTestResultsPublisher.cs:336-355

  List<AzureDevOpsTestCaseResult> batch = [];
  while (batch.Count < _options.BatchSize && _pendingResults.TryDequeue(out AzureDevOpsTestCaseResult? result))
  {
      batch.Add(result);
  }
  ...
  await _client.PublishTestResultsAsync(_publishConfiguration, CurrentRunId.Value, batch, cancellationToken).ConfigureAwait(false);
  _lastFlushTime = _clock.UtcNow;

• On failure it only logs in src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsTestResultsPublisher.cs:357-359

  catch (Exception ex) when (ex is not OperationCanceledException)
  {
      _logger.LogWarning($"{AzureDevOpsResources.AzureDevOpsLivePublishingPublishResultsFailed} {ex.Message}");
  }

Why this is a real issue

If Azure DevOps rejects one request or the network fails after the batch has been dequeued, those results are gone from _pendingResults. The publisher keeps running and the final forced flush has nothing left to resend, so the live run can finish with missing test results even though the extension only logged a warning.

Suggested resolution

Keep batches durable until the publish call succeeds:

• requeue the batch on failure, or
• only remove items from the pending buffer after a successful publish.

A max-retry / poison-batch strategy would still be better than silently dropping the data.

Related issues

• Brainstorm on ideas for more AzDO extensions in MTP #5951