Test infrastructure for the Kubernetes project.
Clone or download
Latest commit 38bfb6c Jan 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Add labels in github template metadata Jan 4, 2019
boskos Remove deprecated automanaged tag from some go rules Jan 10, 2019
config fix security config Jan 18, 2019
def Update deps Nov 29, 2018
docs update dep.md Jan 18, 2019
experiment Bump to gcr.io/k8s-testimages/kubekins-e2e:v20190117-9a712c1e6-(maste… Jan 17, 2019
gcsweb Run hack/update-bazel.sh Feb 26, 2018
ghproxy Revert rules_docker update due to broken docker_push Jan 10, 2019
gopherage Update bazel workspace and npm dependencies Jan 9, 2019
greenhouse Add / update a bunch of OWNERS files Jan 4, 2019
gubernator Make Gubernator store webhook event GUIDs and display on timeline page. Jan 17, 2019
hack Switch hack/update-codegen.sh to use go modules Jan 18, 2019
images Bump to gcr.io/k8s-testimages/kubekins-e2e:v20190117-9a712c1e6-(maste… Jan 17, 2019
jenkins eliminate maybe_upload_podscpec Jan 16, 2019
jobs Update ubuntu/gke image for ci-kubernetes-node-kubelet-conformance - … Jan 18, 2019
kettle Update K8s versions Jan 7, 2019
kubetest Merge pull request #10807 from lubinsz/pr_kubetest Jan 18, 2019
label_sync Add Label for Product Security Team (PST) Jan 18, 2019
logexporter Make Michelle, Shyam approvers for their respective areas Jan 14, 2019
maintenance Upload the oncall page to a GCS bucket with no TTL Jan 17, 2019
metrics Add / update a bunch of OWNERS files Jan 4, 2019
pkg Remove deprecated automanaged tag from some go rules Jan 10, 2019
planter Add / update a bunch of OWNERS files Jan 4, 2019
prow Merge pull request #10569 from nikhita/enable-tide-for-new-repos Jan 19, 2019
robots Revert rules_docker update due to broken docker_push Jan 10, 2019
scenarios revert 1.10 verify image bump Jan 14, 2019
testgrid Merge pull request #10837 from shyamjvs/add-eks-conformance Jan 19, 2019
triage Update bazel workspace and npm dependencies Jan 9, 2019
velodrome Add / update a bunch of OWNERS files Jan 4, 2019
vendor Run hack/update-deps.sh to switch to go mod Jan 18, 2019
.bazelrc bazelrc set --build_tests_only for test Oct 30, 2018
.gitignore Add node_modules to gitignore. Oct 15, 2018
BUILD.bazel Change golint to use go_repository() rule instead of vendor Jan 18, 2019
CONTRIBUTING.md Update CONTRIBUTING.md Mar 25, 2018
LICENSE Update boilerplate to remove "All rights reserved" (#458) Aug 26, 2016
OWNERS Make Michelle, Shyam approvers for their respective areas Jan 14, 2019
README.md Add/update jobs for the 1.13 release. Nov 7, 2018
SECURITY_CONTACTS add SECURITY_CONTACTS file May 24, 2018
WORKSPACE Change golint to use go_repository() rule instead of vendor Jan 18, 2019
code-of-conduct.md Revert "move COC so github will recognize it" Sep 25, 2018
go.mod Run hack/update-deps.sh to switch to go mod Jan 18, 2019
go.sum Run hack/update-deps.sh to switch to go mod Jan 18, 2019
package.json Update bazel workspace and npm dependencies Jan 9, 2019
pylintrc Address nits. Jul 14, 2017
tsconfig.json Rework spyglass lenses to be isolated properly. Nov 27, 2018
yarn.lock Update bazel workspace and npm dependencies Jan 9, 2019

README.md

Kubernetes Test Infrastructure

Go Report Card GoDoc

The test-infra repository contains a collection of tools for testing Kubernetes and displaying Kubernetes tests results. See also CONTRIBUTING.md.

See the architecture diagram for an overview of how the different services interact.

Viewing test results

E2E Testing

Our e2e testing uses kubetest to build/deploy/test kubernetes clusters on various providers. Please see those documents for additional details about this tool as well as e2e testing generally.

Anyone can reconfigure our CI system with a test-infra PR that updates the appropriate files. Detailed instructions follow:

CI Job management

Create a new job

Bootstrap is deprecated, Please utilize the podutils to create new prowjobs.

Create a PR in this repo to add/update/remove a job or suite. Specifically you'll need to do the following:

  • Add the job to the appropriate section in config/jobs

    • Directory Structure:
      • In general for jobs for github.com/org/repo use config/jobs/org/repo/filename.yaml
      • For Kubernetes repos we also allow config/jobs/kubernetes/sig-foo/filename.yaml
      • We use basename of the config name as a key in the prow configmap, so the name of your config file need to be unique across the config subdir
    • Type of jobs:
      • Presubmit jobs run on unmerged code in PRs
      • Postsubmit jobs run after merging code
      • Periodic job run on a timed basis
      • You can find more prowjob definitions at how-to-add-new-jobs
    • A simple sample job uses podutil looks like:
      - name: foo-repo-presubmit-test
        decorate: true
        spec:
          containers:
          - image: gcr.io/k8s-testimages/kubekins-e2e:latest-master
            command:
            - /path/to/cmd
            args:
            - positional
            - --and
            - flags
  • Add the job name to the test_groups list in testgrid/config.yaml

    • Also the group to at least one dashboard_tab

The configs need to be sorted and kubernetes must be in sync with the security repo, or else presubmit will fail. You can run the script below to keep them valid:

hack/update-config.sh

Local testing

docker run your image locally, and mount in the repos you depend on.

Release branch jobs & Image validation jobs

Release branch jobs and image validation jobs are defined in test_config.yaml. We test different master/node image versions against multiple k8s branches on different features.

Those jobs are using channel based versions, current supported testing map is:

  • k8s-dev : master
  • k8s-beta : release-1.13
  • k8s-stable1 : release-1.12
  • k8s-stable2 : release-1.11
  • k8s-stable3 : release-1.10

Our build job will generate a ci/(channel-name) file pointer in gcs.

After you update test_config.yaml, please run

bazel run //experiment:generate_tests -- --yaml-config-path=experiment/test_config.yaml

to regenerate the job configs.

We are moving towards making more jobs to fit into the generated config.

Presubmit will tell you if you forget to do any of this correctly.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Update an existing job

Largely similar to creating a new job, except you can just modify the existing entries rather than adding new ones.

Update what a job does by editing its definition in config/jobs.

Update where the job appears on testgrid by changing testgrid/config.yaml.

Delete a job

The reverse of creating a new job: delete the appropriate entries in config/jobs and testgrid/config.yaml.

Merge your PR and @k8s-ci-robot will deploy your change automatically.

Building and testing the test-infra

We use Bazel to build and test the code in this repo. The commands bazel build //... and bazel test //... should be all you need for most cases. If you modify Go code, run ./hack/update-bazel.sh to keep BUILD.bazel files up-to-date.

Contributing Test Results

The Kubernetes project encourages organizations to contribute execution of e2e test jobs for a variety of platforms (e.g., Azure, rktnetes). For information about how to contribute test results, see Contributing Test Results.

Other Docs