Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions main.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@ include::modules/cnf-best-practices-user-plane-cnfs.adoc[leveloffset=+2]

include::modules/cnf-best-practices-developer-guide.adoc[leveloffset=+1]

include::modules/cnf-best-practices-refactoring.adoc[leveloffset=+2]

include::modules/cnf-best-practices-cnf-security.adoc[leveloffset=+2]

include::modules/cnf-best-practices-avoid-accessing-resource-on-host.adoc[leveloffset=+3]

include::modules/cnf-best-practices-avoid-mounting-host-directories-as-volumes.adoc[leveloffset=+3]

include::modules/cnf-best-practices-avoid-the-host-network-namespace.adoc[leveloffset=+3]

include::modules/cnf-best-practices-linux-capabilities.adoc[leveloffset=+2]

include::modules/cnf-best-practices-openshift-operations.adoc[leveloffset=+2]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[id="cnf-best-practices-avoid-accessing-resource-on-host"]
= Avoid accessing resource on host

It is not recommended for an application to access following resources on the host.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[id="cnf-best-practices-avoid-mounting-host-directories-as-volumes"]
= Avoid mounting host directories as volumes

It is not necessary to mount host `/sys/` or host `/dev/` directories as a volume in a pod in order to use a network device such as SR-IOV VF. The moving of a network interface into the pod network namespace is done automatically by CNI. Mounting the whole `/sys/` or `/dev/` directory in the container will overwrite the network device descriptor inside the container which causes `device not found` or `no such file or directory` error.

Network interface statistics can be queried inside the container using the same `/sys/` path as was done when running directly on the host. When running network interfaces in containers, relevant `/sys/` statistics interfaces are available inside the container, such as `/sys/class/net/net1/statistics/`, `/proc/net/tcp` and `/proc/net/tcp6`.

For running DPDK applications with SR-IOV VF, device specs (in case of vfio-pci) are automatically attached to the container via the Device Plugin. There is no need to mount the /dev/ directory as a volume in the container as the application can find device specs under `/dev/vfio/` in the container.
12 changes: 12 additions & 0 deletions modules/cnf-best-practices-avoid-the-host-network-namespace.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[id="cnf-best-practices-avoid-the-host-network-namespace"]
= Avoid the host network namespace

Application pods must avoid using `hostNetwork`. Applications may not use the host network, including `nodePort` for network communication. Any networking needs beyond the functions provided by the pod network and ingress/egress proxy must be serviced via a MULTUS connected interface.

.CNF requirement
[IMPORTANT]
====
Applications may not use `NodePorts` or the `hostNetwork`.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-service-type[access-control-service-type]
====
64 changes: 64 additions & 0 deletions modules/cnf-best-practices-cnf-security.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
[id="cnf-best-practices-cnf-security"]
= CNF security

In OCP, it is possible to run privileged containers that have all of the root capabilities on a host machine, allowing the ability to access resources which are not accessible in ordinary containers. This, however, increases the security risk to the whole cluster. Containers should only request those privileges they need to run their legitimate functions. No containers will be allowed to run with full privileges without an exception.

The general guidelines are:

. Only ask for the necessary privileges and access control settings for your application.

. If the function required by your CNF can be fulfilled by OCP components, your application should not be requesting escalated privilege to perform this function.

. Avoid using any host system resource if possible.

. Leveraging read only root filesystem when possible.

.CNF requirement
[IMPORTANT]
====
Only ask for the necessary privileges and access control settings for your application

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-security-context-non-root-user-check[access-control-security-context-non-root-user-check]
====

.CNF requirement
[IMPORTANT]
====
If the function required by your CNF can be fulfilled by OCP components, your application should not be
requesting escalated privilege to perform this function.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-security-context-privilege-escalation[access-control-security-context-privilege-escalation]
====

.CNF requirement
[IMPORTANT]
====
Avoid using any host system resource.

See test cases link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-ipc[access-control-pod-host-ipc],
link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-pid[access-control-pod-host-pid]
====

.CNF requirement
[IMPORTANT]
====
Do not mount host directories for device access.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-path[access-control-pod-host-path]
====

.CNF requirement
[IMPORTANT]
====
Do not use host network namespace.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-namespace[access-control-namespace]
====

.CNF requirement
[IMPORTANT]
====
CNFs may not modify the platform in any way.

See test cases link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#platform-alteration-base-image[platform-alteration-base-image], link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#platform-alteration-sysctl-config[platform-alteration-sysctl-config]
====
107 changes: 1 addition & 106 deletions modules/cnf-best-practices-developer-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,109 +16,4 @@ This document is mainly for the developers of CNFs, who need to build high-perfo
[id="cnf-best-practices-non-goal"]
== Non-goal

This is not a guide on how to build CNF’s functionality.

[id="cnf-best-practices-refactoring"]
== Refactoring

NFs should break their software down into the smallest set of microservices as possible. Running monolithic applications inside of a container is not the operating model to be in.

It is hard to move a 1000LB boulder. However, it is easy when that boulder is broken down into many pieces (pebbles). All containerized network functions (CNFs) should break apart each piece of the functions/services/processes into separate containers. These containers will still be within kubernetes pods and all of the functions that perform a single task should be within the same namespace.

There is a link:https://martinfowler.com/microservices/[quote] from Lewis and Fowler that describes this best:

[quote, Lewis and Fowler]
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.These services are built around business capabilities and independently deployable by fully automated deployment machinery.

[id="cnf-best-practices-cnf-security"]
== CNF security

In OCP, it is possible to run privileged containers that have all of the root capabilities on a host machine, allowing the ability to access resources which are not accessible in ordinary containers. This, however, increases the security risk to the whole cluster. Containers should only request those privileges they need to run their legitimate functions. No containers will be allowed to run with full privileges without an exception.

The general guidelines are:

. Only ask for the necessary privileges and access control settings for your application.

. If the function required by your CNF can be fulfilled by OCP components, your application should not be requesting escalated privilege to perform this function.

. Avoid using any host system resource if possible.

. Leveraging read only root filesystem when possible.

.CNF requirement
[IMPORTANT]
====
Only ask for the necessary privileges and access control settings for your application

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-security-context-non-root-user-check[access-control-security-context-non-root-user-check]
====

.CNF requirement
[IMPORTANT]
====
If the function required by your CNF can be fulfilled by OCP components, your application should not be
requesting escalated privilege to perform this function.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-security-context-privilege-escalation[access-control-security-context-privilege-escalation]
====

.CNF requirement
[IMPORTANT]
====
Avoid using any host system resource.

See test cases link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-ipc[access-control-pod-host-ipc],
link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-pid[access-control-pod-host-pid]
====

.CNF requirement
[IMPORTANT]
====
Do not mount host directories for device access.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-pod-host-path[access-control-pod-host-path]
====

.CNF requirement
[IMPORTANT]
====
Do not use host network namespace.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-namespace[access-control-namespace]
====

.CNF requirement
[IMPORTANT]
====
CNFs may not modify the platform in any way.

See test cases link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#platform-alteration-base-image[platform-alteration-base-image], link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#platform-alteration-sysctl-config[platform-alteration-sysctl-config]
====

[id="cnf-best-practices-avoid-accessing-resource-on-host"]
=== Avoid accessing resource on host

It is not recommended for an application to access following resources on the host.

[id="cnf-best-practices-avoid-mounting-host-directories-as-volumes"]
=== Avoid mounting host directories as volumes

It is not necessary to mount host `/sys/` or host `/dev/` directories as a volume in a pod in order to use a network device such as SR-IOV VF. The moving of a network interface into the pod network namespace is done automatically by CNI. Mounting the whole `/sys/` or `/dev/` directory in the container will overwrite the network device descriptor inside the container which causes `device not found` or `no such file or directory` error.

Network interface statistics can be queried inside the container using the same `/sys/` path as was done when running directly on the host. When running network interfaces in containers, relevant `/sys/` statistics interfaces are available inside the container, such as `/sys/class/net/net1/statistics/`, `/proc/net/tcp` and `/proc/net/tcp6`.

For running DPDK applications with SR-IOV VF, device specs (in case of vfio-pci) are automatically attached to the container via the Device Plugin. There is no need to mount the /dev/ directory as a volume in the container as the application can find device specs under `/dev/vfio/` in the container.

[id="cnf-best-practices-avoid-the-host-network-namespace"]
=== Avoid the host network namespace

Application pods must avoid using `hostNetwork`. Applications may not use the host network, including `nodePort` for network communication. Any networking needs beyond the functions provided by the pod network and ingress/egress proxy must be serviced via a MULTUS connected interface.

.CNF requirement
[IMPORTANT]
====
Applications may not use `NodePorts` or the `hostNetwork`.

See test case link:https://github.com/test-network-function/cnf-certification-test/blob/main/CATALOG.md#access-control-service-type[access-control-service-type]
====

This is not a guide on how to build CNF functionality.
11 changes: 11 additions & 0 deletions modules/cnf-best-practices-refactoring.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[id="cnf-best-practices-refactoring"]
= Refactoring

NFs should break their software down into the smallest set of microservices as possible. Running monolithic applications inside of a container is not the operating model to be in.

It is hard to move a 1000LB boulder. However, it is easy when that boulder is broken down into many pieces (pebbles). All containerized network functions (CNFs) should break apart each piece of the functions/services/processes into separate containers. These containers will still be within kubernetes pods and all of the functions that perform a single task should be within the same namespace.

There is a link:https://martinfowler.com/microservices/[quote] from Lewis and Fowler that describes this best:

[quote, Lewis and Fowler]
The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.These services are built around business capabilities and independently deployable by fully automated deployment machinery.