From 8cbee37ba6a36969c7645032667db170dda6bd23 Mon Sep 17 00:00:00 2001 From: Michelle Shepardson Date: Tue, 10 Sep 2019 10:52:59 -0700 Subject: [PATCH] Updating READMEs. --- CONTRIBUTING.md | 23 ++++++ README.md | 142 ++++++++++++++++++++++++++++++++----- TEST_INFRA_README.md | 134 ---------------------------------- build_test_update.md | 3 +- cmd/entomologist/README.md | 4 +- 5 files changed, 153 insertions(+), 153 deletions(-) delete mode 100644 TEST_INFRA_README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d002fd465..36eef509a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -31,3 +31,26 @@ information on using pull requests. This project follows [Google's Open Source Community Guidelines](https://opensource.google.com/conduct/). + +## Source Code Headers + +Every file containing source code must include copyright and license +information. This includes any JS/CSS files that you might be serving out to +browsers. (This is to help well-intentioned people avoid accidental copying that +doesn't comply with the license.) + +Apache header: + + Copyright 2019 the TestGrid Authors. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md index 17892783a..f870c87f7 100644 --- a/README.md +++ b/README.md @@ -8,25 +8,135 @@ We're in the middle of open-sourcing TestGrid! You can check this repo or [this issue](https://github.com/kubernetes/test-infra/issues/10409) for more updates as we continue. -## Source Code Headers +We have a short [video] from the testgrid session at the 2018 Kubernetes +contributor summit. -Every file containing source code must include copyright and license -information. This includes any JS/CSS files that you might be serving out to -browsers. (This is to help well-intentioned people avoid accidental copying that -doesn't comply with the license.) +The video demos power features of testgrid, including: +* Sorting +* Filtering +* Graphing +* Grouping +* Dashboard groups +* Summaries +* Clustered Failures -Apache header: +Please have a look! - Copyright 2019 the TestGrid Authors. +## Contributing - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at +If you want to modify TestGrid beyond adding new tests or dashboards, see +[Contributing](CONTRIBUTING.md) and [Updating Testgrid](build_test_update.md). - https://www.apache.org/licenses/LICENSE-2.0 +## Configuration - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. +If you need to add a new test that you want TestGrid to display, or otherwise change what is shown +on https://testgrid.k8s.io, see [Testgrid Configuration](config.md). + +Updates to the config are automatically tested and pushed to production. + +## Using the client + +Here are some quick tips and clarifications for using the TestGrid site! + +### Tab Statuses + +TestGrid assigns dashboard tabs a status based on recent test runs. + + * **PASSING**: No failures found in recent (`num_columns_recent`) test runs. + * **FAILING**: One or more consistent failures in recent test runs. + * **FLAKY**: The tab is neither PASSING nor FAILING. There is at least one + recent failed result that is not a consistent failure. + +### Summary Widget + +You can get a small widget showing the status of your dashboard tab, based on +the tab statuses above! For example: + +`sig-testing-misc#bazel`: [![sig-testing-misc/bazel](https://testgrid.k8s.io/q/summary/sig-testing-misc/bazel/tests_status?style=svg)](https://testgrid.k8s.io/sig-testing-misc#bazel) + +Inline it with: + +``` + +[![/](https://testgrid.k8s.io/q/summary///tests_status?style=svg)](https://testgrid.k8s.io/#) +``` + +### Customizing Test Result Sizes + +Change the size of the test result rectangles. + +The three sizes are Standard, Compact, and Super Compact. You can also specify +`width=X` in the URL (X > 3) to customize the width. For small widths, this may +mean the date and/or changelist, or other custom headers, are no longer +visible. + +### Filtering Tests + +You can repeatedly add filters to include/exclude test rows. Under **Options**: + +* **Include/Exclude Filter by RegEx**: Specify a regular expression that + matches test names for rows you'd like to include/exclude. +* **Exclude non-failed Tests**: Omit rows with no failing results. + +### Grouping Tests + +Grouped tests are summarized in a single row that is collapsible/expandable by +clicking on the test name (shown as a triangle on the left). Under **Options**: + +* **Group by RegEx Mask**: Specify a regular expression to mask a portion of + the test name. Any test names that match after applying this mask will be + grouped together. +* **Group by Target**: Any tests that contain the same target will be + grouped together. +* **Group by Hierarchy Pattern**: Specify a regular expression that matches + one or more parts of the tests' names and the tests will be grouped + hierarchically. For example, if you have these tests in your dashboard: + + ```text + /test/dir1/target1 + /test/dir1/target2 + /test/dir2/target3 + ``` + + By specifying regular expression "\w+", the tests will be organized into: + + ```text + ▼test + ▼dir1 + target1 + ▼dir2 + target2 + target3 + ``` + +### Sorting Tests + +Under **Options** + +* **Sort by Failures**: Tests with more recent failures will appear before + other tests. +* **Sort by Flakiness**: Tests with a higher flakiness score will appear + before tests with a lower flakiness score. The flakiness score, which is not + reported, is based on the number of transitions from passing to failing (and + vice versa) with more weight given to more recent transitions. +* **Sort by Name**: Sort alphabetically. + +## Clustered Failures + +You can display identified clustered failures in your test results grid in a +dashboard tab. Select the ***Display Clustered Failures List*** toggle button to +render a list/table of identified failure clusters at the bottom of the browser. + +Clusters can be grouped by: +* test status +* test status and error message + +The clustered failures table shows the test status, error message (if grouped by +error message), and area of the clusters. The clusters are sorted by area in +descending order. + +Selecting a row highlights the cells belonging to that cluster. Multiple row +selection (with multiple cluster highlighting) is supported. To de-select a row, +click on the selected row again. + +[video]: https://www.youtube.com/watch?v=jm2l2SLq_yE diff --git a/TEST_INFRA_README.md b/TEST_INFRA_README.md deleted file mode 100644 index 231b52dcc..000000000 --- a/TEST_INFRA_README.md +++ /dev/null @@ -1,134 +0,0 @@ -# Testgrid - -The Kubernetes Testgrid instance is available at https://testgrid.k8s.io - -We have a short [video] from the testgrid session at the 2018 contributor summit. - -The video demos power features of testgrid, including: -* Sorting -* Filtering -* Graphing -* Grouping -* Dashboard groups -* Summaries -* Clustered Failures - -Please have a look! - -## Using the client - -Here are some quick tips and clarifications for using the TestGrid site! - -### Tab Statuses - -TestGrid assigns dashboard tabs a status based on recent test runs. - - * **PASSING**: No failures found in recent (`num_columns_recent`) test runs. - * **FAILING**: One or more consistent failures in recent test runs. - * **FLAKY**: The tab is neither PASSING nor FAILING. There is at least one - recent failed result that is not a consistent failure. - -### Summary Widget - -You can get a small widget showing the status of your dashboard tab, based on -the tab statuses above! For example: - -`sig-testing-misc#bazel`: [![sig-testing-misc/bazel](https://testgrid.k8s.io/q/summary/sig-testing-misc/bazel/tests_status?style=svg)](https://testgrid.k8s.io/sig-testing-misc#bazel) - -Inline it with: - -``` - -[![/](https://testgrid.k8s.io/q/summary///tests_status?style=svg)](https://testgrid.k8s.io/#) -``` - -### Customizing Test Result Sizes - -Change the size of the test result rectangles. - -The three sizes are Standard, Compact, and Super Compact. You can also specify -`width=X` in the URL (X > 3) to customize the width. For small widths, this may -mean the date and/or changelist, or other custom headers, are no longer -visible. - -### Filtering Tests - -You can repeatedly add filters to include/exclude test rows. Under **Options**: - -* **Include/Exclude Filter by RegEx**: Specify a regular expression that - matches test names for rows you'd like to include/exclude. -* **Exclude non-failed Tests**: Omit rows with no failing results. - -### Grouping Tests - -Grouped tests are summarized in a single row that is collapsible/expandable by -clicking on the test name (shown as a triangle on the left). Under **Options**: - -* **Group by RegEx Mask**: Specify a regular expression to mask a portion of - the test name. Any test names that match after applying this mask will be - grouped together. -* **Group by Target**: Any tests that contain the same target will be - grouped together. -* **Group by Hierarchy Pattern**: Specify a regular expression that matches - one or more parts of the tests' names and the tests will be grouped - hierarchically. For example, if you have these tests in your dashboard: - - ```text - /test/dir1/target1 - /test/dir1/target2 - /test/dir2/target3 - ``` - - By specifying regular expression "\w+", the tests will be organized into: - - ```text - ▼test - ▼dir1 - target1 - ▼dir2 - target2 - target3 - ``` - -### Sorting Tests - -Under **Options** - -* **Sort by Failures**: Tests with more recent failures will appear before - other tests. -* **Sort by Flakiness**: Tests with a higher flakiness score will appear - before tests with a lower flakiness score. The flakiness score, which is not - reported, is based on the number of transitions from passing to failing (and - vice versa) with more weight given to more recent transitions. -* **Sort by Name**: Sort alphabetically. - -## Clustered Failures - -You can display identified clustered failures in your test results grid in a -dashboard tab. Select the ***Display Clustered Failures List*** toggle button to -render a list/table of identified failure clusters at the bottom of the browser. - -Clusters can be grouped by: -* test status -* test status and error message - -The clustered failures table shows the test status, error message (if grouped by -error message), and area of the clusters. The clusters are sorted by area in -descending order. - -Selecting a row highlights the cells belonging to that cluster. Multiple row -selection (with multiple cluster highlighting) is supported. To de-select a row, -click on the selected row again. - -## Configuration - -If you need to add a new test that you want TestGrid to display, or otherwise change what is shown -on https://testgrid.k8s.io, see [Testgrid Configuration](config.md). - -Updates to the config are automatically tested and pushed to production. - -## Contributing - -If you want to modify TestGrid beyond adding new tests or dashboards, see [Updating Testgrid](build_test_update.md). - -[video]: https://www.youtube.com/watch?v=jm2l2SLq_yE diff --git a/build_test_update.md b/build_test_update.md index d1f55452e..5647fc07c 100644 --- a/build_test_update.md +++ b/build_test_update.md @@ -3,7 +3,8 @@ If you need to add a new test that you want TestGrid to display, see [TestGrid Configuration](config.md). If you're looking to develop for TestGrid, welcome! Note that most of the inner workings of TestGrid -are not open source. This is a [known issue](https://github.com/kubernetes/test-infra/issues/10409). +are not open source yet. Follow [this issue](https://github.com/kubernetes/test-infra/issues/10409) +for updates. ## YAML configuration and config.proto diff --git a/cmd/entomologist/README.md b/cmd/entomologist/README.md index cc21a2290..bfa7cb774 100644 --- a/cmd/entomologist/README.md +++ b/cmd/entomologist/README.md @@ -23,7 +23,7 @@ Targets in Pull Requests won't be pinned. Entomologist can be configured with a caching proxy, such as [ghProxy](https://github.com/kubernetes/test-infra/tree/master/ghproxy), to minimize API token usage. -Entomologist is writing multiple [issue_state.proto](https://github.com/kubernetes/test-infra/blob/master/testgrid/issue_state/issue_state.proto) +Entomologist is writing multiple [issue_state.proto](https://github.com/GoogleCloudPlatform/testgrid/blob/master/issue_state/issue_state.proto) files to Google Cloud Storage (GCS). TestGrid consumes the information placed there. ## Required Flags @@ -32,4 +32,4 @@ files to Google Cloud Storage (GCS). TestGrid consumes the information placed th For example, `--repos=kubernetes/test-infra,kubernetes/kubernetes` - `--output`: The location of the issue_state.proto file that Entomologist will write to - `--config`: The location of the config.proto file that this TestGrid is using -- `--gcs-credentials-file`: GCS requires credentials to write to a GCS location \ No newline at end of file +- `--gcs-credentials-file`: GCS requires credentials to write to a GCS location