Skip to content
Branch: master
Find file History
dcbw and s1061123 CRD: interfaceRequest -> interface (v1 spec conformance)
Change the Network Attachment Selection Annotation long-form
interface name request JSON key from 'interfaceRequest' to
'interface' to conform with the V1 NPWG spec.
Latest commit 5ef87bf Nov 8, 2018
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
npwg-demo-1 CRD: interfaceRequest -> interface (v1 spec conformance) Nov 9, 2018
README.md
clusterrole.yml Fix example files (#171 and #183) Nov 5, 2018
cni-configuration.conf
crd.yml
flannel-conf.yml Update kubernetes.cni.cncf.io -> k8s.cni.cncf.io Jul 24, 2018
macvlan-conf.yml Update kubernetes.cni.cncf.io -> k8s.cni.cncf.io Jul 24, 2018
multus-ptp-portmap.conf
multus-with-flannel.yml
net-resource-sample-pod.yaml
sample-pod.yml [docs] Updates kubernetes.v1 to k8s.v1 in the sample files Jul 25, 2018
sriov-net.yaml parse kubelet checkpoint file for pod devices Oct 10, 2018

README.md

Example Configurations & Pod Specs

In the ./examples folder some example configurations are provided for using Multus, especially with CRDs, and doubly so in reference to their usage with the defacto standard for CRDs as proposed by the Network Plumbing Working Group.

Examples overview

Generally, the examples here show a setup using Multus with CRD support. The examples here demonstrate a setup with Multus as the meta-plugin used by Kubernetes, and delegating to either Flannel (which will be the default pod network), or to macvlan. The CRDs are intended to be alignment with the defacto standard.

It is expected that aspects of your own setup will vary, at least in part, from some of what's demonstrated here. Namely, the IP address spaces, and likely the host ethernet interface names used in the macvlan part of the configuration.

More specifically, these examples show:

  • Multus configured, using CNI a .conf file, with CRD support, specifying that we will use a "default network".
  • A resource definition with a daemonset that places the .conf on each node in the cluster.
  • A CRD defining the "networks" @ network-attachment-definitions.k8s.cni.cncf.io
  • CRD objects containing the configuration for both Flannel & macvlan.

Quick-start instructions

  • Compile Multus and place binaries into (typically) /opt/cni/bin/
    • Refer to the primary README.md for more details on compilation.
  • Allow system:node access to enable Multus to pull CRD objects.
    • See "RBAC configuration section below for details."
  • Create the Flannel + Multus setup with the daemonset provided
    • As in: kubectl create -f multus-with-flannel.yml
    • Optionally, verify that the /etc/cni/net.d/*.conf exists on each node.
  • Create the CRDs
    • Create the CRD itself, kubectl create -f crd.yml
    • Create the network attachment configurations (i.e. CNI configurations packed into CRD objects)
      • kubectl create -f flannel-conf.yml
      • kubectl create -f macvlan-conf.yml
      • Verify the CRD objects are created with: kubectl get networks
  • Spin up an sample pod
    • kubectl create -f sample-pod.yml
    • Verify that it has multiple interfaces with:
      • kubectl exec -it samplepod -- ip a

RBAC configuration

You'll need to enable the system:node users access to the API endpoints that will deliver the CRD objects to Multus.

Using these examples, you'll first create a cluster role with the provided sample:

kubectl create -f clusterrole.yml

You will then create a clusterrolebinding for each hostname in the Kubernetes cluster. Replace HOSTNAME below with the host name of a node, and then repeat for all hostnames in the cluster.

kubectl create clusterrolebinding multus-node-HOSTNAME \
    --clusterrole=multus-crd-overpowered \
    --user=system:node:HOSTNAME

CNI Configuration

A sample cni-configuration.conf is provided, typically this file is placed in /etc/cni/net.d/. It must be the first file alphabetically in this folder in order for the Kubelet to honor its use. However, if you opt to use the provided Flannel + Multus YAML file, this will deploy a configuration (packed inside a daemonset therein) on each node in your Kubernetes cluster.

Other considerations

Primarily in this setup one thing that one should consider are the aspects of the macvlan-conf.yml, which is likely specific to the configuration of the node on which this resides.

Passing down device information

Some CNI plugins require specific device information which maybe pre-allocated by K8s device plugin. This could be indicated by providing k8s.v1.cni.cncf.io/resourceName annotaton in its network attachment definition CRD. The file examples/sriov-net.yaml shows an example on how to define a Network attachment definition with specific device allocation information. Multus will get allocated device information and make them available for CNI plugin to work on.

In this exmaple (shown below), it is expected that an SRIOV Device Plugin making a pool of SRIOV VFs available to the K8s with intel.com/sriov as their resourceName. Any device allocated from this resource pool will be passed down by Multus to the sriov-cni plugin in deviceID field. This is up to the sriov-cni plugin to capture this information and work with this specific device information.

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: sriov-net-a
  annotations:
    k8s.v1.cni.cncf.io/resourceName: intel.com/sriov
spec:
  config: '{
  "type": "sriov",
  "vlan": 1000,
  "ipam": {
    "type": "host-local",
    "subnet": "10.56.217.0/24",
    "rangeStart": "10.56.217.171",
    "rangeEnd": "10.56.217.181",
    "routes": [{
      "dst": "0.0.0.0/0"
    }],
    "gateway": "10.56.217.1"
  }
}'

The net-resource-sample-pod.yaml is an exmaple Pod manifest file that requesting a SRIOV device from a host which is then configured using the above network attachement definition.

For further information on how to configure SRIOV Device Plugin and SRIOV-CNI please refer to the links given above.

You can’t perform that action at this time.