Skip to content

Latest commit

 

History

History
50 lines (33 loc) · 3.35 KB

conformance-testing.md

File metadata and controls

50 lines (33 loc) · 3.35 KB
Raw

Conformance Testing - 1.7+

Overview

With such a wide array of Kubernetes distributions available, conformance tests help ensure that any self-described "Kubernetes" actually meets the minimal set of features. They are a subset of e2e (end-to-end) tests that should pass on any Kubernetes cluster.

A conformance-passing cluster gives administrators and users the following guarantees:

  • Best practices: Your Kubernetes is properly configured. This is useful to know whether you are running a distribution out of the box or handling your own custom setup.

  • Predictability: All of your cluster behavior is well-documented. (1) Available features in the official Kubernetes documentation can be taken as a given. (2) Unexpected bugs should be rare, as distribution-specific issues are weeded out during the conformance tests.

  • Interoperability: Workloads from other conforming clusters can be easily ported into your cluster (or vice versa). This standardization of Kubernetes is a key advantage of open source software, and allows you to avoid vendor lock-in.

Kubernetes distributions may offer additional features beyond what is covered by Conformance Testing, but these are not expected to carry over in case of a switch.


NOTE: Kubernetes documentation also describes the concept of node conformance tests. While useful, these tests are more component-focused than systemic. They only validate the behavior of a specific node, rather than cluster behavior as a whole.


See the official documentation of Kubernetes's existing conformance tests.

Integration with Sonobuoy

Sonobuoy's plugin architecture enables you to integrate conformance test results into your reporting. The e2e tests can be configured via the plugin mechanism and are set up by default to run the basic set of conformance tests against a local provider.

To customize the set of tests that will be run as part of the report, the following environmental variables can be set in the plugin-specific YAML config:

Variable Default Value Description
E2E_FOCUS "Conformance" The test suite to run.

NOTE: Because the real conformance suite can take up to half an hour to run, the quickstart example's e2e config specifies just a single test, "Pods should be submitted and removed".
E2E_SKIP "Alpha|Disruptive|Feature|Flaky|Kubectl" Which subset of tests to skip
E2E_PROVIDER "local" The platform that the cluster is running on

NOTE: The length of time it takes to run conformance can vary based on the size of your cluster---the timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy config.json.