Fix JSON generations on new flakiness dashboard

https://bugs.webkit.org/show_bug.cgi?id=123723

Reviewed by Andreas Kling.

Add is_flaky column on results table so that this column can be used to find flaky tests on a given builder
efficiently without having to through results for all tests in PHP. This column is updated in report.php
when a new build is added. Because is_flaky depends on the preceding and succeeding results, we must update
is_flaky flag on results for builds immediately before and after the new build as well.

To see why, suppose we had two consecutive results [PASS] [PASS]. If we were to insert [FAIL] result between
the two, those two results may also turn into flaky results if they were surrounded by [FAIL]. Similarly,
if we had [PASS] [FAIL] and the second result was marked flaky, inserting new [FAIL] must unmark it.


* init-database.sql: Added is_flaky column to results table with an index. Also added an index on
build_revisions.time as many queries filter results by this quantity. Also set the work_mem to 50MB avoid
disk thrashing while sorting results in various queries.

* public/api/failing-tests.php: Handle builder ids as well as names. Call generate() with failure types.
No longer generates *-failing.json since it's a subset of *-wrongexpectations.json to save time.

* public/api/report.php: Rewritten. Calls update_flakiness_after_inserting_build to update is_flaky flags
on the newly added results.
(store_results): Added.
(main): Added.

* public/include/test-results.php:
(ResultsJSONWriter):
(ResultsJSONWriter::add_results_for_test): Renamed from add_results_for_test_if_matches.
(ResultsJSONGenerator::generate): Takes the failure type. Instead of generating JSONs for all failure types
at once, generate one JSON for the specified type. We generate the list of test ids based on the failure type
and query results based on that. This dramatically cuts down the time spent in PHP.
(ResultsJSONGenerator::latest_build): Added.
(ResultsJSONGenerator::write_jsons): Takes single writer now.
(update_flakiness_for_build): Added.
(update_flakiness_after_inserting_build): Added.

* public/index.html:
(TestResultsView._populateBuilderPane): Emulate *-failing.json upon *-wrongexpectations.json.
(TestResultsView.fetchFailingTestsForBuilder): Ditto.

* public/main.css: Minor style tweaks.
(.testResults): Extend the border that wraps the test results as needed.
(.tooltip): Don't wrap text inside tooltips.


Canonical link: https://commits.webkit.org/141918@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@158565 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Websites/test-results/ChangeLog

@@ -1,3 +1,51 @@
+2013-11-04  Ryosuke Niwa  <rniwa@webkit.org>
+
+        Fix JSON generations on new flakiness dashboard
+        https://bugs.webkit.org/show_bug.cgi?id=123723
+
+        Reviewed by Andreas Kling.
+
+        Add is_flaky column on results table so that this column can be used to find flaky tests on a given builder
+        efficiently without having to through results for all tests in PHP. This column is updated in report.php
+        when a new build is added. Because is_flaky depends on the preceding and succeeding results, we must update
+        is_flaky flag on results for builds immediately before and after the new build as well.
+
+        To see why, suppose we had two consecutive results [PASS] [PASS]. If we were to insert [FAIL] result between
+        the two, those two results may also turn into flaky results if they were surrounded by [FAIL]. Similarly,
+        if we had [PASS] [FAIL] and the second result was marked flaky, inserting new [FAIL] must unmark it.
+
+
+        * init-database.sql: Added is_flaky column to results table with an index. Also added an index on
+        build_revisions.time as many queries filter results by this quantity. Also set the work_mem to 50MB avoid
+        disk thrashing while sorting results in various queries.
+
+        * public/api/failing-tests.php: Handle builder ids as well as names. Call generate() with failure types.
+        No longer generates *-failing.json since it's a subset of *-wrongexpectations.json to save time.
+
+        * public/api/report.php: Rewritten. Calls update_flakiness_after_inserting_build to update is_flaky flags
+        on the newly added results.
+        (store_results): Added.
+        (main): Added.
+
+        * public/include/test-results.php:
+        (ResultsJSONWriter):
+        (ResultsJSONWriter::add_results_for_test): Renamed from add_results_for_test_if_matches.
+        (ResultsJSONGenerator::generate): Takes the failure type. Instead of generating JSONs for all failure types
+        at once, generate one JSON for the specified type. We generate the list of test ids based on the failure type
+        and query results based on that. This dramatically cuts down the time spent in PHP.
+        (ResultsJSONGenerator::latest_build): Added.
+        (ResultsJSONGenerator::write_jsons): Takes single writer now.
+        (update_flakiness_for_build): Added.
+        (update_flakiness_after_inserting_build): Added.
+
+        * public/index.html:
+        (TestResultsView._populateBuilderPane): Emulate *-failing.json upon *-wrongexpectations.json.
+        (TestResultsView.fetchFailingTestsForBuilder): Ditto.
+
+        * public/main.css: Minor style tweaks.
+        (.testResults): Extend the border that wraps the test results as needed.
+        (.tooltip): Don't wrap text inside tooltips.
+
 2013-10-26  Ryosuke Niwa  <rniwa@webkit.org>
 
         Make new bug link in flakiness dashboard configurable

Websites/test-results/init-database.sql

@@ -41,6 +41,7 @@ CREATE TABLE build_revisions (
     PRIMARY KEY (repository, build));
 CREATE INDEX revision_build_index ON build_revisions(build);
 CREATE INDEX revision_repository_index ON build_revisions(repository);
+CREATE INDEX revision_time_index ON build_revisions(time);
 
 CREATE TABLE tests (
     id serial PRIMARY KEY,
@@ -55,6 +56,10 @@ CREATE TABLE results (
     expected varchar(64) NOT NULL,
     actual varchar(64) NOT NULL,
     modifiers varchar(64) NOT NULL,
-    time integer);
+    time integer,
+    is_flaky boolean);
 CREATE INDEX results_test ON results(test);
 CREATE INDEX results_build ON results(build);
+CREATE INDEX results_is_flaky ON results(is_flaky);
+
+SET work_mem='50MB';

Websites/test-results/public/api/failing-tests.php

@@ -5,20 +5,25 @@
 
 function main() {
     require_existence_of($_GET, array('builder' => '/^[A-Za-z0-9 \(\)\-_]+$/'));
-    $builder_name = $_GET['builder'];
+    $builder_param = $_GET['builder'];
 
     $db = connect();
-    $builder_row = $db->select_first_row('builders', NULL, array('name' => $builder_name));
-    if (!$builder_row)
-        exit_with_error('BuilderNotFound');
+    $builder_row = $db->select_first_row('builders', NULL, array('name' => $builder_param));
+    if (!$builder_row) {
+        $builder_row = $db->select_first_row('builders', NULL, array('id' => $builder_param));
+        if (!$builder_row)
+            exit_with_error('BuilderNotFound');
+    }
     $builder_id = $builder_row['id'];
 
     $generator = new ResultsJSONGenerator($db, $builder_id);
 
-    if ($generator->generate())
-        exit_with_success();
+    if (!$generator->generate('wrongexpectations'))
+        exit_with_error('ResultsWithWrongExpectationsNotFound', array('builderId' => $builder_id));
+    else if (!$generator->generate('flaky'))
+        exit_with_error('FlakyResultsNotFound', array('builderId' => $builder_id));
     else
-        exit_with_error('ResultsNotFound');
+        exit_with_success();
 }
 
 main();

Websites/test-results/public/api/report.php

@@ -3,73 +3,87 @@
 require_once('../include/json-shared.php');
 require_once('../include/test-results.php');
 
-$db = connect();
-
-require_existence_of($_POST, array(
-    'master' => '/[A-Za-z0-9\.]+/',
-    'builder_name' => '/^[A-Za-z0-9 \(\)\-_]+$/',
-    'build_number' => '/^[0-9]+?$/',
-    'build_slave' => '/^[A-Za-z0-9\-_]+$/',
-    'revisions' => '/^.+?$/',
-    'start_time' => '/^[0-9]+(\.[0-9]+)?$/',
-    'end_time' => '/^[0-9]+(\.[0-9]+)?$/'));
-$master = $_POST['master'];
-$builder_name = $_POST['builder_name'];
-$build_number = intval($_POST['build_number']);
-
-if (!array_key_exists('file', $_FILES) or !array_key_exists('tmp_name', $_FILES['file']) or count($_FILES['file']['tmp_name']) <= 0)
-    exit_with_error('ResultsJSONNotIncluded');
-
-$revisions = json_decode(str_replace('\\', '', $_POST['revisions']), TRUE);
-foreach ($revisions as $repository_name => $revision_data) {
-    require_format('repository_name', $repository_name, '/^\w+$/');
-    require_existence_of($revision_data, array(
-        'revision' => '/^[a-z0-9]+$/',
-        'timestamp' => '/^[a-z0-9\-\.:TZ]+$/',
-    ), 'revision');
+function store_results($db, $master, $builder_name, $build_number, $start_time, $end_time, $revisions, $json_path) {
+    $test_results = fetch_and_parse_test_results_json($json_path);
+    if (!$test_results)
+        exit_with_error('InvalidResultsJSON');
+
+    $builder_id = add_builder($db, $master, $builder_name);
+    if (!$builder_id)
+        exit_with_error('FailedToInsertBuilder', array('master' => $master, 'builderName' => $builder_name));
+
+    $build_id = add_build($db, $builder_id, $build_number);
+    if (!$build_id)
+        exit_with_error('FailedToInsertBuild', array('builderId' => $builder_id, 'buildNumber' => $build_number));
+
+    foreach ($revisions as $repository_name => $revision_data) {
+        $repository_id = $db->select_or_insert_row('repositories', NULL, array('name' => $repository_name));
+        if (!$repository_id)
+            exit_with_error('FailedToInsertRepository', array('name' => $repository_name));
+
+        $revision_data = array(
+            'repository' => $repository_id,
+            'build' => $build_id,
+            'value' => $revision_data['revision'],
+            'time' => array_get($revision_data, 'timestamp'));
+        $db->select_or_insert_row('build_revisions', NULL, array('repository' => $repository_id, 'build' => $build_id), $revision_data, 'value')
+            or exit_with_error('FailedToInsertRevision', array('name' => $repository_name, 'data' => $revision_data));
+    }
+
+    $slave_id = add_slave($db, $_POST['build_slave']);
+    if (!store_test_results($db, $test_results, $build_id, $start_time, $end_time, $slave_id))
+        exit_with_error('FailedToStoreResults', array('buildId' => $build_id));
+
+    return $build_id;
 }
 
-$test_results = fetch_and_parse_test_results_json($_FILES['file']['tmp_name']);
-if (!$test_results)
-    exit_with_error('InvalidResultsJSON');
-
-$start_time = float_to_time($_POST['start_time']);
-$end_time = float_to_time($_POST['end_time']);
-
-$builder_id = add_builder($db, $master, $builder_name);
-if (!$builder_id)
-    exit_with_error('FailedToInsertBuilder', array('master' => $master, 'builderName' => $builder_name));
-
-$build_id = add_build($db, $builder_id, $build_number);
-if (!$build_id)
-    exit_with_error('FailedToInsertBuild', array('builderId' => $builder_id, 'buildNumber' => $build_number));
-
-foreach ($revisions as $repository_name => $revision_data) {
-    $repository_id = $db->select_or_insert_row('repositories', NULL, array('name' => $repository_name));
-    if (!$repository_id)
-        exit_with_error('FailedToInsertRepository', array('name' => $repository_name));
-
-    $revision_data = array(
-        'repository' => $repository_id,
-        'build' => $build_id,
-        'value' => $revision_data['revision'],
-        'time' => array_get($revision_data, 'timestamp'));
-    $db->select_or_insert_row('build_revisions', NULL, array('repository' => $repository_id, 'build' => $build_id), $revision_data, 'value')
-        or exit_with_error('FailedToInsertRevision', array('name' => $repository_name, 'data' => $revision_data));
+function main() {
+    require_existence_of($_POST, array(
+        'master' => '/[A-Za-z0-9\.]+/',
+        'builder_name' => '/^[A-Za-z0-9 \(\)\-_]+$/',
+        'build_number' => '/^[0-9]+?$/',
+        'build_slave' => '/^[A-Za-z0-9\-_]+$/',
+        'revisions' => '/^.+?$/',
+        'start_time' => '/^[0-9]+(\.[0-9]+)?$/',
+        'end_time' => '/^[0-9]+(\.[0-9]+)?$/'));
+    $master = $_POST['master'];
+    $builder_name = $_POST['builder_name'];
+    $build_number = intval($_POST['build_number']);
+    $start_time = float_to_time($_POST['start_time']);
+    $end_time = float_to_time($_POST['end_time']);
+
+    $revisions = json_decode(str_replace('\\', '', $_POST['revisions']), TRUE);
+    foreach ($revisions as $repository_name => $revision_data) {
+        require_format('repository_name', $repository_name, '/^\w+$/');
+        require_existence_of($revision_data, array(
+            'revision' => '/^[a-z0-9]+$/',
+            'timestamp' => '/^[a-z0-9\-\.:TZ]+$/',
+        ), 'revision');
+    }
+
+    if (!array_key_exists('file', $_FILES) or !array_key_exists('tmp_name', $_FILES['file']) or count($_FILES['file']['tmp_name']) <= 0)
+        exit_with_error('ResultsJSONNotIncluded');
+    $json_path = $_FILES['file']['tmp_name'];
+
+    $db = connect();
+    $build_id = store_results($db, $master, $builder_name, $build_number, $start_time, $end_time, $revisions, $json_path);
+    @ob_end_clean();
+    ignore_user_abort();
+    ob_start();
+
+    echo_success();
+
+    header('Connection: close');
+    header('Content-Length: ' . ob_get_length());
+
+    @ob_end_flush();
+    flush();
+    if (function_exists('fastcgi_finish_request'))
+        fastcgi_finish_request();
+
+    update_flakiness_after_inserting_build($build_id);
 }
 
-$slave_id = add_slave($db, $_POST['build_slave']);
-if (!store_test_results($db, $test_results, $build_id, $start_time, $end_time, $slave_id))
-    exit_with_error('FailedToStoreResults', array('buildId' => $build_id));
-
-echo_success();
-
-@ob_end_flush();
-flush();
-if (function_exists('fastcgi_finish_request'))
-    fastcgi_finish_request();
-
-$generator = new ResultsJSONGenerator($db, $builder_id);
-$generator->generate();
+main();
 
 ?>