Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion content/en/synthetics/browser_tests/test_results.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ In the **History** section, you can see three graphs:
- The **Time-to-interactive by location and device** graph displays the amount of time until a page can be interacted with in seconds. For more information about uptime monitoring, see the [Website Uptime Monitoring with SLOs][14] guide.
- The **Test duration by location and device** graph displays the amount of time in minutes each location and device takes to complete in a given time interval.

{{< img src="synthetics/browser_tests/history.png" alt="The History and Sample Runs section in the Test Details page" style="width=80%" >}}
<!-- PLACEHOLDER IMAGE — replace history_2.png with a new screenshot of the Test History section on the redesigned Test Details Page before release. Tracked in DOCS-13939. -->
{{< img src="synthetics/browser_tests/history_2.png" alt="The History and Sample Runs section in the Test Details page" style="width=80%" >}}

## Sample results

Expand Down Expand Up @@ -157,6 +158,23 @@ If your test reaches the maximum execution time, the timeout message indicates t

A test result is considered `FAILED` if it does not satisfy its assertions or if a step failed for another reason. You can troubleshoot failed runs by looking at their screenshots, checking for potential [errors](#errors-and-warnings) at the step level, and looking into [resources][17] and [backend traces](#backend-traces) generated by their steps.

### AI failure summaries

<!-- DRAFT — feature not yet publicly available. Do not merge until release. Tracked in DOCS-13939. -->

When a browser test run fails, Datadog generates an AI failure summary to help you identify the cause and next steps for investigation. Each summary includes:

- A short explanation of what failed, grounded in run data such as network errors, assertions, and screenshots.
- A classification of the failure as either a **true failure** (a real problem with your application) or a **test misconfiguration** (an issue with the test setup).
- Suggested next steps for troubleshooting.

AI failure summaries appear on the test run details page for any failing browser test run. Treat them as a starting point for investigation, not as authoritative root cause analysis, because LLM-generated content can contain inaccuracies.

To share feedback on the accuracy or usefulness of a summary, click the feedback button in the summary panel.

<!-- PLACEHOLDER IMAGE — replace ai_failure_summary.png with a real screenshot of the summary panel on a failing browser test run before release. -->
{{< img src="synthetics/browser_tests/test_results/ai_failure_summary.png" alt="AI failure summary panel on a failing browser test run" style="width:100%" >}}

### Compare screenshots

To help during the investigation, click **Compare Screenshots** to receive side-by-side screenshots of the failed result and the last successful execution. The comparison helps you to spot any differences that could have caused the test to fail.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ To reduce alert fatigue, Synthetic tests can be retried when a test run fails. I

With a fast retry, Datadog runs a Synthetic test multiple times before transitioning the test's monitor to alert and sending you a notification. For more information about monitors associated with your Synthetic tests, see [Use Synthetic Test Monitors][3].

{{< img src="synthetics/guide/synthetics_test_retries/fast_retries.png" alt="Failed test runs with fast retries" style="width:100%;">}}
<!-- PLACEHOLDER IMAGE — replace fast_retries_2.png with a new screenshot of fast retries on the redesigned Test Details Page before release. Tracked in DOCS-13939. -->
{{< img src="synthetics/guide/synthetics_test_retries/fast_retries_2.png" alt="Failed test runs with fast retries" style="width:100%;">}}


## Group evaluations
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ further_reading:

Click on a mobile app test on the [**Synthetic Tests** page][11] to see the Test Details page. The Test Details page contains all the information relating to your test, including test properties, test history, sample runs, and test runs.

{{< img src="mobile_app_testing/test_details_2.png" alt="The Mobile App Test Details page" style="width=80%" >}}
<!-- PLACEHOLDER IMAGE — replace test_details_3.png with a new screenshot of the redesigned Mobile App Test Details page before release. Tracked in DOCS-13939. -->
{{< img src="mobile_app_testing/test_details_3.png" alt="The Mobile App Test Details page" style="width=80%" >}}

Test runs appear in a test details page after a Synthetic mobile app test executes. [Sample results](#sample-results) correlate to the latest passed and failed test executions over a time interval and in a specific number of locations and devices.

Expand All @@ -38,7 +39,8 @@ In the **Properties** section, you can see the test ID, test creation and edit d

In the **History** section, you can see the **Global Uptime** graph, which displays the total uptime of all test locations in a given time interval. The global uptime takes into consideration the [alert conditions][4] configured for a test.

{{< img src="mobile_app_testing/history.png" alt="The History graph displays global uptime" style="width=80%" >}}
<!-- PLACEHOLDER IMAGE — replace history_2.png with a new screenshot of the History graph on the redesigned Test Details Page before release. Tracked in DOCS-13939. -->
{{< img src="mobile_app_testing/history_2.png" alt="The History graph displays global uptime" style="width=80%" >}}

## Sample results

Expand Down Expand Up @@ -79,7 +81,8 @@ Video replay

Every executed test step contains a screenshot of the step action, step action name, step ID, and step duration.

{{< img src="mobile_app_testing/screenshot-and-action.png" alt="Screenshots and actions in the Sample Runs section of the test details" style="width:100%" >}}
<!-- PLACEHOLDER IMAGE — replace screenshot-and-action_2.png with a new screenshot of the Sample Runs screenshots and actions on the redesigned Test Details Page before release. Tracked in DOCS-13939. -->
{{< img src="mobile_app_testing/screenshot-and-action_2.png" alt="Screenshots and actions in the Sample Runs section of the test details" style="width:100%" >}}

Screenshots can be disabled by checking "Do not capture screenshot for this step" in the **Advanced Options** section.

Expand All @@ -100,7 +103,8 @@ Click the video icon ▶️ for a failed test run on the [Mobile Application tes
Results of steps contain the XML representation of the screen being tested, as well as the ability to hover over the screenshot or the XML to highlight specific elements of the app.
Click an element to view additional attributes:

{{< img src="mobile_app_testing/xml_inspector.png" alt="Screenshot of a test result under the Sample Runs section, showing the XML inspector" style="width=80%" >}}
<!-- PLACEHOLDER IMAGE — replace xml_inspector_2.png with a new screenshot of the XML inspector on the redesigned Test Details Page before release. Tracked in DOCS-13939. -->
{{< img src="mobile_app_testing/xml_inspector_2.png" alt="Screenshot of a test result under the Sample Runs section, showing the XML inspector" style="width=80%" >}}

## Crash reports

Expand Down
Binary file added static/images/mobile_app_testing/history_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading