Bug 1189826 - Don't recommend retriggering if we have full set of runs

If we already have 6 runs for base / new, more won't really help, even if we have low/medium confidence.
mozilla · Jul 31, 2015 · ed2e392 · ed2e392
1 parent de99cbb
commit ed2e392
Showing 1 changed file with 13 additions and 3 deletions.
diff --git a/ui/partials/perf/comparetable.html b/ui/partials/perf/comparetable.html
@@ -56,10 +56,20 @@
           {{compareResult.newRuns.length}}</span>
       </td>
       <td class="test-warning">
+        <!-- Show a hint that people should trigger more runs if:
+             * we only have 1 result for either base/new (can't draw any
+               strong conclusions from one result)
+             * we have less than 6 runs for either base/new AND our
+               confidence level isn't 'high' (6 runs is our gold standard
+               for confidence -- if we've done that many and we're still
+               not confident in a change, more are unlikely to help)
+          -->
         <span ng-if="compareResult.originalRuns.length && compareResult.newRuns.length &&
-                     (compareResult.originalRuns.length < 2 ||
-                      compareResult.newRuns.length < 2 ||
-                      compareResult.confidenceText !== 'high')"
+                     ((compareResult.originalRuns.length < 2 ||
+                       compareResult.newRuns.length < 2) ||
+                      (compareResult.originalRuns.length < 6 &&
+                       compareResult.newRuns.length < 6 &&
+                       compareResult.confidenceText !== 'high'))"
                 class="glyphicon glyphicon-warning-sign text-warning"
                 tooltip="More base / new runs recommended for increased confidence in comparison"
                 tooltip-placement="left"