Skip to content

Commit

Permalink
documentation: update trigger phrases
Browse files Browse the repository at this point in the history
- `test-me-please` replaced with `/test`.
- `build-me-please` replace with `/build`.
- Other triggers as-is but prefixed with `/`.
- Unified race detection trigger phrases with the rest of the document:
  singular race detection jobs will expose their unique trigger phrases
  directly from the PR checks.

Signed-off-by: Nicolas Busseneau <nicolas@isovalent.com>
  • Loading branch information
nbusseneau authored and jibi committed Sep 27, 2021
1 parent b416482 commit 0e1613c
Show file tree
Hide file tree
Showing 4 changed files with 67 additions and 79 deletions.
4 changes: 2 additions & 2 deletions Documentation/contributing/development/contributing_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -212,8 +212,8 @@ Getting a pull request merged

#. As you submit the pull request as described in the section :ref:`submit_pr`.
One of the reviewers will start a CI run by replying with a comment
``test-me-please`` as described in :ref:`trigger_phrases`. If you are a
core team member, you may trigger the CI run yourself.
``/test`` as described in :ref:`trigger_phrases`. If you are a core team
member, you may trigger the CI run yourself.

#. Hound: basic ``golang/lint`` static code analyzer. You need to make the
puppy happy.
Expand Down
3 changes: 2 additions & 1 deletion Documentation/contributing/release/backports.rst
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,8 @@ Running the CI Against the Pull Request

To validate a cross-section of various tests against the PRs, backport PRs
should be validated in the CI by running all CI targets. This can be triggered
by adding a comment to the PR with exactly the text ``test-backport-x.x``, where ``x.x`` is the target version.
by adding a comment to the PR with exactly the text ``/test-backport-x.x``,
where ``x.x`` is the target version as described in :ref:`trigger_phrases`.
The comment must not contain any other characters.

After the Backports are Merged
Expand Down
137 changes: 62 additions & 75 deletions Documentation/contributing/testing/ci.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,48 +40,48 @@ After you don't need to run tests on your branch, please remove the branch from
(net-next, 49, 419 are possible values right now), ``--k8s_version`` for k8s
version. If you want to run only one ``It`` block, you need to prepend it
with a test suite and create a regex, e.g
``test-only --focus="K8sDatapathConfig.*Check connectivity with automatic direct nodes routes" --k8s_version=1.18 --kernel_version=net-next``
``/test-only --focus="K8sDatapathConfig.*Check connectivity with automatic direct nodes routes" --k8s_version=1.18 --kernel_version=net-next``
will run specified test in 1.18 Kubernetes cluster running on net-next nodes.
Kubernetes version defaults to 1.21, kernel version defaults to 4.19.

+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8s"`` | Runs all kubernetes tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sConformance"`` | Runs all k8s conformance tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sChaos"`` | Runs all k8s chaos tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sDatapathConfig"`` | Runs all k8s datapath configuration tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sDemos"`` | Runs all k8s demo tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sKubeProxyFreeMatrix"`` | Runs all k8s kube-proxy free matrix tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sFQDNTest"`` | Runs all k8s fqdn tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sHealthTest"`` | Runs all k8s health tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sHubbleTest"`` | Runs all k8s Hubble tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sIdentity"`` | Runs all k8s identity tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sIstioTest"`` | Runs all k8s Istio tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sKafkaPolicyTest"`` | Runs all k8s Kafka tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sPolicyTest"`` | Runs all k8s policy tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sServicesTest"`` | Runs all k8s services tests |
+------------------------------------------------+-------------------------------------------+
| ``test-only --focus="K8sUpdates"`` | Runs k8s update tests |
+------------------------------------------------+-------------------------------------------+


Running Runtime test suite is still done via ``test-focus`` command.

+---------------------------------------+-------------------------------------------+
| ``test-focus Runtime`` | Runs all runtime tests |
+---------------------------------------+-------------------------------------------+
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8s"`` | Runs all kubernetes tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sConformance"`` | Runs all k8s conformance tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sChaos"`` | Runs all k8s chaos tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sDatapathConfig"`` | Runs all k8s datapath configuration tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sDemos"`` | Runs all k8s demo tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sKubeProxyFreeMatrix"`` | Runs all k8s kube-proxy free matrix tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sFQDNTest"`` | Runs all k8s fqdn tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sHealthTest"`` | Runs all k8s health tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sHubbleTest"`` | Runs all k8s Hubble tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sIdentity"`` | Runs all k8s identity tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sIstioTest"`` | Runs all k8s Istio tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sKafkaPolicyTest"`` | Runs all k8s Kafka tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sPolicyTest"`` | Runs all k8s policy tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sServicesTest"`` | Runs all k8s services tests |
+-------------------------------------------------+-------------------------------------------+
| ``/test-only --focus="K8sUpdates"`` | Runs k8s update tests |
+-------------------------------------------------+-------------------------------------------+


Running Runtime test suite is still done via ``/test-focus`` command.

+----------------------------------------+-------------------------------------------+
| ``/test-focus Runtime`` | Runs all runtime tests |
+----------------------------------------+-------------------------------------------+



Expand Down Expand Up @@ -127,11 +127,10 @@ environment variable in the Jenkins job. That version ID will be used in Cilium

Changes to this image are made via contributions to the packer-ci-build
repository. Authorized GitHub users can trigger builds with a GitHub comment on
the PR containing the trigger phrase ``build-me-please``. In case that a new box
needs to be rebased with a different branch than master, authorized developers
can run the build with custom parameters. To use a different Cilium branch in
the `job`_ go to *Build with parameters* and a base branch can be set as the
user needs.
the PR containing the trigger phrase ``/build``. In case that a new box needs to
be rebased with a different branch than master, authorized developers can run
the build with custom parameters. To use a different Cilium branch in the `job`_
go to *Build with parameters* and a base branch can be set as the user needs.

This box will need to be updated when a new developer needs a new dependency
that is not installed in the current version of the box, or if a dependency that
Expand Down Expand Up @@ -184,58 +183,46 @@ them all at once:
+------------------+--------------------------+
| PR target branch | Trigger required PR jobs |
+==================+==========================+
| master | test-me-please |
| master | /test |
+------------------+--------------------------+
| v1.10 | test-backport-1.10 |
| v1.10 | /test-backport-1.10 |
+------------------+--------------------------+
| v1.9 | test-backport-1.9 |
| v1.9 | /test-backport-1.9 |
+------------------+--------------------------+
| v1.8 | test-backport-1.8 |
| v1.8 | /test-backport-1.8 |
+------------------+--------------------------+

For ``master`` PRs: on top of ``test-me-please``, one may use
``test-missed-k8s`` to trigger all non-required K8s versions on Kernel 4.9 as
per the `Cilium CI matrix`_.
For ``master`` PRs: on top of ``/test``, one may use ``/test-missed-k8s`` to
trigger all non-required K8s versions on Kernel 4.9 as per the `Cilium CI
matrix`_.

For all PRs: one may manually retrigger a specific job (e.g. in case of a flake)
with the individual trigger featured directly in the PR check's name (e.g. for
``K8s-1.20-kernel-4.9 (test-1.20-4.9)``, use ``test-1.20-4.9``).
``K8s-1.20-kernel-4.9 (test-1.20-4.9)``, use ``/test-1.20-4.9``).

For a full list of Jenkins PR jobs, see `Jenkins
For a full list of Jenkins PR jobs, see `Jenkins (PR tab)
<https://jenkins.cilium.io/view/PR/>`_. Trigger phrases are configured within
each job's build triggers advanced options.

There are some feature flags based on Pull Requests labels, the list of labels
are the following:

- ``area/containerd``: Enable containerd runtime on all Kubernetes test.
- ``ci/net-next``: Run tests on net-next kernel. This causes the
``test-me-please`` target to only run on the net-next kernel. It is purely
for testing on a different kernel, to merge a PR it must pass the CI
without this flag.

- ``ci/net-next``: Run tests on net-next kernel. This causes the ``/test``
target to only run on the net-next kernel. It is purely for testing on a
different kernel, to merge a PR it must pass the CI without this flag.

Testing with race condition detection enabled
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want to run test suite with race condition detection enabled, enter following trigger phrases. These jobs are not required to merge.

+------------------------------------------------------------------------------------+------------------------+
| Jenkins Job | Trigger Phrase |
+====================================================================================+========================+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-Ginkgo-Tests-Kernel-Race-Detection | test-race-4.19 |
+------------------------------------------------------------------------------------+------------------------+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-K8s-1.16-net-next-Race-Detection | test-race-net-next |
+------------------------------------------------------------------------------------+------------------------+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-K8s-1.21-kernel-4.9-Race-Detection | test-race-4.9 |
+------------------------------------------------------------------------------------+------------------------+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-K8s-GKE-Race-Detection | test-race-gke |
+------------------------------------------------------------------------------------+------------------------+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-Kubernetes-Upstream-Race-Detection | test-race-upstream-k8s |
+------------------------------------------------------------------------------------+------------------------+
| https://jenkins.cilium.io/view/PR/job/Cilium-PR-Runtime-4.9-Race-Detection | test-race-runtime |
+------------------------------------------------------------------------------------+------------------------+
Optional non-required Jenkins are available for running the test suite with race
condition detection enabled, and may be triggered using the trigger phrase
``/test-race``.

For a full list of Jenkins PR jobs with race detection enabled, see `Jenkins
(Race Detection tab) <https://jenkins.cilium.io/view/Race%20Detection/>`_.
Trigger phrases are configured within each job's build triggers advanced
options.

Using Jenkins for testing
~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
2 changes: 1 addition & 1 deletion test/get-gh-comment-info.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import argparse

parser = argparse.ArgumentParser()
parser.add_argument('ghcomment', type=str) # this is for test-me-please phrases
parser.add_argument('ghcomment', type=str) # used for `/test` trigger phrases
parser.add_argument('--focus', type=str, default="")
parser.add_argument('--kernel_version', type=str, default="")
parser.add_argument('--k8s_version', type=str, default="")
Expand Down

0 comments on commit 0e1613c

Please sign in to comment.