Skip to content

[WIP] Fix AzureDevOpsReport suppressing failures without reporting stack frame#8525

Draft
Copilot wants to merge 1 commit into
mainfrom
copilot/fix-suppressing-failures-azuredevopsreport
Draft

[WIP] Fix AzureDevOpsReport suppressing failures without reporting stack frame#8525
Copilot wants to merge 1 commit into
mainfrom
copilot/fix-suppressing-failures-azuredevopsreport

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 23, 2026

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.


This section details on the original issue you should resolve

<issue_title>[MTP Reporters Analysis] AzureDevOpsReport suppresses failures without a usable local stack frame</issue_title>
<issue_description>## Summary
AzureDevOpsReport suppresses failures entirely when it cannot resolve a stack frame to a local repo file. The current implementation only emits ##vso[task.logissue ...] when it finds a usable source path; otherwise it silently returns without reporting anything.

Evidence

  • src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs:167-177
    string? line = GetErrorText(testDisplayName, explanation, exception, severity, _fileSystem, _logger, _targetFrameworkMoniker, annotationSuffix);
    if (line is null)
    {
        ...
        return;
    }
  • src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs:193-200
    if (exception is null || exception.StackTrace is null)
    {
        ...
        return null;
    }
  • src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs:274-316
    else
    {
        ...
        continue;
    }
    ...
    if (!fileSystem.ExistFile(fullPath))
    {
        ...
        continue;
    }
    ...
    logger.LogTrace("No stack trace line matched criteria, no failure line was reported.");
    return null;

Why this is a real issue

There are multiple concrete failure shapes that hit this path:

  • the framework sets Explanation but no exception,
  • an exception exists but has no stack trace,
  • stack frames point outside the repo root,
  • deterministic/source-generated/external paths do not exist on disk on the current machine.

In all of those cases, the reporter produces no Azure DevOps annotation at all, even though it still has a test display name and an error message that could be logged.

Suggested resolution

Add a fallback task.logissue path that emits at least the message/title when no source location can be resolved. sourcepath / linenumber should be treated as optional enhancements, not as a prerequisite for reporting the failure.

Related issues

Comments on the Issue (you are @copilot in this section)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MTP Reporters Analysis] AzureDevOpsReport suppresses failures without a usable local stack frame

2 participants