Skip to content

Latest commit

 

History

History
44 lines (29 loc) · 3.1 KB

conformance-testing.md

File metadata and controls

44 lines (29 loc) · 3.1 KB
Raw

Conformance Testing - 1.8+

Overview

With such a wide array of Kubernetes distributions available, conformance tests help ensure that a Kubernetes cluster meets the minimal set of features. They are a subset of end-to-end (e2e) tests that should pass on any Kubernetes cluster.

A conformance-passing cluster provides the following guarantees:

  • Best practices: Your Kubernetes is properly configured. This is useful to know whether you are running a distribution out of the box or handling your own custom setup.

  • Predictability: All your cluster behavior is well-documented. Available features in the official Kubernetes documentation can be taken as a given. Unexpected bugs should be rare, because distribution-specific issues are weeded out during the conformance tests.

  • Interoperability: Workloads from other conforming clusters can be ported into your cluster, or vice versa. This standardization of Kubernetes is a key advantage of open source software, and allows you to avoid vendor lock-in.

Individual Kubernetes distributions may offer additional features beyond conformance testing, but if you change distributions, these features can't be expected to be provided.

NOTE: Kubernetes documentation also describes the concept of node conformance tests. Although they are useful, these tests are more component-focused than system-wide. They validate only the behavior of a specific node, not cluster behavior as a whole.

See the official documentation for Kubernetes's existing conformance tests.

Integration with Sonobuoy

Sonobuoy's plugin architecture enables you to integrate conformance test results into your reporting. The e2e tests can be configured with the plugin. The default configuration runs the basic set of conformance tests against a local provider.

To customize the set of tests that are run as part of the report, the following environmental variables can be set in the plugin-specific YAML config:

Variable Default Value Description
E2E_FOCUS "Conformance" The test suite to run.

NOTE: Because the real conformance suite can take up to an hour to run, the quickstart example's e2e config specifies just a single test, "Pods should be submitted and removed".
E2E_SKIP "Alpha|Disruptive|Feature|Flaky|Kubectl" Which subset of tests to skip
E2E_PROVIDER "local" The platform that the cluster is running on

NOTE: The length of time it takes to run conformance can vary based on the size of your cluster---the timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy config.json.