Kata Containers tests, CI, and metrics
Clone or download
chavafg Merge pull request #1067 from GabyCT/topic/skipoci
test: Skip oci_call_test for Debian
Latest commit 7467874 Jan 18, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ci Merge pull request #1040 from GabyCT/topic/jenkinsfire Jan 15, 2019
Dockerfiles/BuildTest tests: Add docker integration build test. Feb 27, 2018
arch ci: move all aarch64-specific lib files under .ci/aarch64 Aug 9, 2018
cmd ci: Add installation scripts for debian. Dec 19, 2018
conformance/posixfs tests: Add posix filesystem conformance test Oct 25, 2018
data data: add rpm packages public signing key Oct 18, 2018
functional functional: Add test checking for container state Nov 29, 2018
integration test: Skip oci_call_test for Debian Jan 17, 2019
lib CI: If runtime config is not found, set sane defaults Dec 12, 2018
metrics test: Modify soak test for Debian Jan 16, 2019
pentest security: Add security tests Sep 24, 2018
sanity tests: Add sanity test after running integration and functional tests Oct 5, 2018
stress ci: memory-tested image vish/stress is single-arch image for amd64 Sep 5, 2018
vendor ci: memory-tested image vish/stress is single-arch image for amd64 Sep 5, 2018
.gitignore cmd: Add log parser Mar 2, 2018
.pullapprove.yml CI: Require doc team signoff for doc changes Jan 24, 2018
.travis.yml travis: add retry for bash install on osx Jan 11, 2019
CODE_OF_CONDUCT.md CoC: Add Code of Conduct Dec 4, 2017
CONTRIBUTING.md Docs : Add minimal contribution guidelines Jan 30, 2018
Gopkg.lock ci: memory-tested image vish/stress is single-arch image for amd64 Sep 5, 2018
Gopkg.toml vendor: add github.com/BurntSushi/toml May 30, 2018
LICENSE Initial commit Nov 13, 2017
Makefile Merge pull request #617 from lifupan/master Dec 22, 2018
README.md docs: Document how to run the tests Nov 29, 2018
bundle.go functional: The terminal needs to be disabled Nov 29, 2018
command.go tests: Add docker run stdin integration test. Feb 16, 2018
config.go test: add vsock test Aug 21, 2018
container.go functional: Add test checking for container state Nov 29, 2018
log.go tests: Add initial docker integration tests Jan 22, 2018
process.go tests: implement and use ShimRunning Jun 4, 2018
rand.go tests: Add initial docker integration tests Jan 22, 2018
versions.yaml cri-o: build go-md2man from sources Dec 4, 2018

README.md

Kata Containers tests

This repository contains various types of tests and utilities (called "content" from now on) for testing the Kata Containers code repositories.

Getting the code

$ go get -d github.com/kata-containers/tests

Test Content

We provide several tests to ensure Kata-Containers run on different scenarios and with different container managers.

  1. Functional tests
  2. Integration tests to ensure compatibility with:
  3. Network tests
  4. Stability tests
  5. Metrics

Note: The unit tests of the different Kata Containers components are stored in each repository along with the source code they test.

CI Content

This repository contains a number of scripts that run from under a "CI" (Continuous Integration) system.

Centralised scripts

The CI scripts in this repository are used to test changes to the content of this repository. These scripts are also used by the other Kata Containers code repositories.

The advantages of this approach are:

  • Functionality is defined once.

    • Easy to make changes affecting all code repositories centrally.
  • Assurance that all the code repositories are tested in this same way.

CI setup

WARNING:

The CI scripts perform a lot of setup before running content under a CI. Some of this setup runs as the root user and could break your developer's system. See Developer Mode.

Detecting a CI system

The strategy to check if the tests are running under a CI system is to see if the CI variable is set to the value true. For example, in shell syntax:

if [ "$CI" = true ]; then
    : # Assumed to be running in a CI environment
else
    : # Assumed to NOT be running in a CI environment
fi

Breaking Compatibility

In case the patch you submit breaks the CI because it needs to be tested together with a patch from another kata-containers repository, you have to specify which repository and which pull request it depends on.

Using a simple tag Depends-on: in your commit message will allow the CI to run properly. Notice that this tag is parsed from the latest commit of the pull request.

For example:

	Subsystem: Change summary

	Detailed explanation of your changes.

	Fixes: #nnn

	Depends-on:github.com/kata-containers/runtime#999

	Signed-off-by: <contributor@foo.com>

In this example, we tell the CI to fetch the pull request 999 from the runtime repository and use that rather than the master branch when testing the changes contained in this pull request.

Developer Mode

Developers need a way to run as much test content as possible locally, but as explained in CI Setup, running all the content in this repository could be dangerous.

The recommended approach to resolve this issue is to set the following variable to any non-blank value before using any content from this repository:

export KATA_DEV_MODE=true

Setting this variable has the following effects:

  • Disables content that might not be safe for developers to run locally.
  • Ignores the effect of the CI variable being set (for extra safety).

You should be aware that setting this variable provides a safe subset of functionality; it is still possible that PRs raised for code repositories will still fail under the automated CI systems since those systems are running all possible tests.

Run the Kata Containers tests

Requirements to run Kata Containers tests

You need to install the following to run Kata Containers tests:

Prepare an environment

The recommended method to set up Kata Containers is to use the official and latest stable release. You can find the official documentation to do this in the Kata Containers installation user guides.

To try the latest commits of Kata use the CI scripts, which build and install from the kata-containers repositories, with the following steps:

Warning: This may replace/delete packages and configuration that you already have. Please use these steps only on a testing environment.

Add the $GOPATH/bin directory to the PATH:

$ export PATH=${GOPATH}/bin:${PATH}

Clone the kata-container/tests repository:

$ go get -d github.com/kata-containers/tests

Go to the tests repo directory:

$ cd $GOPATH/src/github.com/kata-containers/tests

Execute the setup script:

$ .ci/setup.sh

Limitation: If the script fails for a reason and it is re-executed, it will execute all steps from the beginning and not from the failed step.

Run the tests

If you have already installed the Kata Containers packages and a container manager (i.e. Docker or Kubernetes), and you want to execute the content for all the tests, run the following:

$ export RUNTIME=kata-runtime
$ export KATA_DEV_MODE=true
$ sudo -E PATH=$PATH make test

You can also execute a single test suite. For example, if you want to execute the docker integration tests, run the following:

$ sudo -E PATH=$PATH make docker