Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

Commit

Permalink
Add in better descriptions of the existing test specs
Browse files Browse the repository at this point in the history
Currently, we have very well documented reel.Handler instances.  In
fact, documentation could be guaranteed via API in the near future.

However, reel.Handlers are really just the building blocks for
composing larger test cases.  As such, we need to be more pedantic
about documenting our Ginkgo test specs.

Since Ginkgo does not appear to utilize or enforce any interface, this
is currently nearly impossible to do programmatically.  This change is
a low cost addition to README.md which explains in plain english what
each Ginkgo test spec actually tests.  It also provides hints for
well-aligned future work, but makes no commitment to delivering such
tests.

Signed-off-by: Ryan Goulding <rgouldin@redhat.com>
  • Loading branch information
ryandgoulding committed Feb 22, 2021
1 parent a5d3c6e commit d7299e3
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 1 deletion.
54 changes: 54 additions & 0 deletions README.md
Expand Up @@ -52,6 +52,60 @@ multus|The multus test suite is used to test SR-IOV network connectivity between
operator|The operator test suite is designed basic Kubernetes Operator functionality.|4.4.3
container|The container test suite is designed to test container functionality and configuration|4.4.3

Further information about the current offering for each test spec is included below.

### diagnostic tests

The `diagnostic` test spec issues commands to poll environment information which can be appended to the claim file.
This information is necessary to ensure a properly spec'd environment is used, and allows the claim to be reproduced.
As of today, the `diagnostic` test suite just polls `Node` information for all `Node`s in the cluster. Future
iterations may consider running `lshw` or similar types of diagnostic tests.

### generic tests

The `generic` test spec tests:
1) `Default` network connectivity between containers.
2) That CNF container images are RHEL based.

To test `Default` network connectivity, a [test partner pod](https://github.com/redhat-nfvpe/cnf-certification-test-partner)
is installed on the network. The test partner pod is instructed to issue ICMPv4 requests to each container listed in
the [test configuration](./test-network-function/generic_test_configuration.yml), and vice versa. The test asserts that
the test partner pod receives the correct number of replies, and vice versa.

In the future, other networking protocols aside from ICMPv4 should be tested.

### multus tests

Similar to the `generic` test spec, the `multus` test spec is utilized for CNFs that utilize multiple network
interfaces. As of today, the `multus` test suite just tests that
a [test partner pod](https://github.com/redhat-nfvpe/cnf-certification-test-partner) can successfully ping the secondary
interface of the CNF containers. Since SR-IOV is often utilized, and the secondary interface of a CNF cannot be
accessed in user space, the test is unidirectional.

### operator tests

Currently, the `operator` test spec is limited to a single test case called `OPERATOR_STATUS`. `OPERATOR_STATUS` just
checks that the `CSV` corresponding to the CNF Operator is properly installed.

In the future, tests surrounding `Operational Lifecycle Management` will be added.

### container tests

The `container` test spec has the following test cases:

Test Name|Description
---|---
`HOST_NETWORK_CHECK`|Ensures that the CNF pods do not utilize host networking. Note: This test can be disabled for infrastructure CNFs that should utilize host networking.
`HOST_PORT_CHECK`|Ensures that the CNF pods do not utilize host ports.
`HOST_PATH_CHECK`|Ensures that the CNF pods do not utilize the host filesystem.
`HOST_IPC_CHECK`|Ensures that the CNF pods do not utilize host IPC namespace to access or control host processes.
`HOST_PID_CHECK`|Ensures that the CNF pods do not utilize host PID namespace to access or control host processes.
`CAPABILITY_CHECK`|Ensures that the CNF SCC is not configured to allow `NET_ADMIN` or `SYS_ADMIN`.
`ROOT_CHECK`|Ensure that the CNF pods are not run as `root`.
`PRIVILEGE_ESCALATION`|Ensure that the CNF SCC is not configured to allow privileged escalation.

In the future, we are considering additional tests to ensure aspects such as un-alteration of the container image.

## Performing Tests

Currently, all available tests are part of the "CNF Certification Test Suite" test suite, which serves as the entrypoint
Expand Down
2 changes: 1 addition & 1 deletion version.json
@@ -1,3 +1,3 @@
{
"tag": "v1.0.0"
"tag": "v1.0.2"
}

0 comments on commit d7299e3

Please sign in to comment.