Skip to content

Commit 7fce4ac

Browse files
committed
Bug 1882087 - Format ui-test failure messages for treeherder. r=aaronmt,tthibaud,android-reviewers,aryx
Example: https://treeherder.mozilla.org/logviewer?job_id=450541238&repo=try ``` TEST-UNEXPECTED-FAIL | failure | https://console.firebase.google.com/project/moz-fenix/testlab/histories/bh.66b7091e15d53d45/matrices/8658376039685060316 | 1 test cases failed, 589 passed ``` That's an unconventional and suboptimal error message, but it's the only information that I see readily available; hopefully someone can improve on this in the future. Differential Revision: https://phabricator.services.mozilla.com/D204207
1 parent 54ebf80 commit 7fce4ac

File tree

3 files changed

+16
-0
lines changed

3 files changed

+16
-0
lines changed

mobile/android/android-components/automation/taskcluster/androidTest/parse-ui-test.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,12 @@ def main():
6262
**matrix_result
6363
)
6464
)
65+
if matrix_result["outcome"] != "success":
66+
# write failures to test log in format known to treeherder logviewer
67+
sys.stdout.write(
68+
f"TEST-UNEXPECTED-FAIL | {matrix_result['outcome']} | {matrix_result['webLink']} | {matrix_result['axes'][0]['details']}\n"
69+
)
70+
6571
print("---\n")
6672
print("# References & Documentation\n")
6773
print(

mobile/android/fenix/automation/taskcluster/androidTest/parse-ui-test.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,11 @@ def main():
6060
f"| {matrix_result['matrixId']} | {matrix_result['outcome']}"
6161
f"| [Firebase Test Lab]({matrix_result['webLink']}) | {axis['details']}\n"
6262
)
63+
if matrix_result["outcome"] != "success":
64+
# write failures to test log in format known to treeherder logviewer
65+
sys.stdout.write(
66+
f"TEST-UNEXPECTED-FAIL | {matrix_result['outcome']} | {matrix_result['webLink']} | {axis['details']}\n"
67+
)
6368
print("---\n")
6469
print("# References & Documentation\n")
6570
print(

mobile/android/focus-android/automation/taskcluster/androidTest/parse-ui-test.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,11 @@ def main():
6060
f"| {matrix_result['matrixId']} | {matrix_result['outcome']}"
6161
f"| [Firebase Test Lab]({matrix_result['webLink']}) | {axis['details']}\n"
6262
)
63+
if matrix_result["outcome"] != "success":
64+
# write failures to test log in format known to treeherder logviewer
65+
sys.stdout.write(
66+
f"TEST-UNEXPECTED-FAIL | {matrix_result['outcome']} | {matrix_result['webLink']} | {axis['details']}\n"
67+
)
6368
print("---\n")
6469
print("# References & Documentation\n")
6570
print(

0 commit comments

Comments
 (0)