Skip to content

Latest commit

 

History

History
57 lines (45 loc) · 1.71 KB

steps-desync.md

File metadata and controls

57 lines (45 loc) · 1.71 KB
title description keywords author ms.author ms.date ms.topic ms.assetid ms.service ms.custom
Test report steps are out of sync
Understand what causes test step order discrepancies in test reports. See how to reconcile test report step order in App Center Test.
test cloud, steps, skipped
lucen-ms
lucen
05/11/2022
troubleshooting
0b5ef2ec-0962-47ca-9279-f149f5bca569
vs-appcenter
test

Test report steps are out of sync

[!INCLUDE Retirement announcement for App Center]

App Center Test creates test reports that synchronize each test and test step across devices used. To organize the reports, Test relies on the filenames and order of the screenshots it takes.

If your test suite runs on multiple devices, and allows devices to take different paths through the test code, test steps might appear out of order, skipped, or duplicated in the report. The report summary doesn't count tests with these symptoms as failures.

The following pseudocode examples demonstrate this issue and a workaround.

Pseudocode to reproduce the issue

This example creates discrepancies in the test report if both of the following conditions apply:

  • The test is running on multiple devices.
  • At least one device takes each code path.
if(bool)
{
    // code to run if true
    app.Screenshot("True");
} else 
{
    // code to run if false
    app.Screenshot("False");
};

Pseudocode workaround

In the following pseudocode, since both code paths create the same screenshot names and sequence, the test report can reconcile both paths to the same test step.

if(bool)
{
    // code to run if true
    app.Screenshot("Result");
} else 
{
    // code to run if false
    app.Screenshot("Result");
};