Provision Contrail CNI for Kubernetes with High Availability (HA)

mprasannajuniper edited this page Dec 7, 2017 · 4 revisions

This Wiki Provides steps to provision Contrail Controller HA with Contrail pods replicated on 3 nodes.

Refer to the following Wiki to complete the prerequisites for each node:

Provision Contrail CNI for Kubernetes


Two Yaml files for following topologies are proposed:

  1. Contrail pods present on Kubernetes master and slave nodes
  2. Contrail pods present only on slave nodes and Kubernetes master node hosting only Kubernetes pods

Steps to provision Contrail with HA

  1. Git Clone the contrail-docker repository
git clone https://github.com/Juniper/contrail-docker.git -b R4.0
  1. Go to directory contrail-docker/kubernetes/manifests/
cd contrail-docker/kubernetes/manifests/
  1. Edit single yaml file using your favorite editor, in this case we are using vim
  • For ubuntu 16.04 host OS with Contrail pods on Kuberntes Master and Slave nodes
vim contrail-host-ubuntu-ha.yaml
  • For ubuntu 16.04 host OS with Contrail pods on Kuberntes slave nodes only
vim contrail-host-ubuntu-ha-independent-k8s.yaml
  1. Edit the below variables in configmap and change it according to your setup. Mandatory variables which has to be changed are: config_nodes controller_nodes analytics_nodes analyticsdb_nodes api_server. Please refer to input variable wiki for more details
data:
global-config: |-
  [GLOBAL]
  cloud_orchestrator = kubernetes
  sandesh_ssl_enable = False
  enable_config_service = True
  enable_control_service = True
  enable_webui_service = True
  introspect_ssl_enable = False
  config_nodes = 10.84.24.52,10.84.24.53,10.84.14.144
  controller_nodes = 10.84.24.52,10.84.24.53,10.84.14.144
  analytics_nodes = 10.84.24.52,10.84.24.53,10.84.14.144
  analyticsdb_nodes = 10.84.24.52,10.84.24.53,10.84.14.144
agent-config: |-
  [AGENT]
  compile_vrouter_module = True
  # Optional ctrl_data_network, if different from management
  # ctrl_data_network = "192.168.10.0/24"
kubemanager-config: |-
  [KUBERNETES]
  cluster_name = k8s-default
  cluster_project = {'domain': 'default-domain', 'project': 'default'}
  cluster_network = {}
  service_subnets = 10.96.0.0/12
  pod_subnets = 10.32.0.0/12
  api_server = 10.84.14.144
kubernetes-agent-config: |-
  [AGENT]
  1. Label slave nodes where you want to bring up Contrail pods with following label. In case of Contrail pods on master and slave nodes both, label 2 slave nodes. In case of Contrail pods only on slave nodes, label 3 slave nodes.
Label the node as opencontrail.org/controller=true using the below command
kubectl label node <node-name> opencontrail.org/controller=true
  1. On the nodes where Contrail controller is present, ensure that /etc/hosts has entries for all controller nodes.

  2. [OPTIONAL] If it is Contrail pods on Kubernetes master setup and it is desired to run Contrail Agent on master, then uncomment the below lines under contrail-agent daemonset

#tolerations:
#- key: node-role.kubernetes.io/master
#  operator: Exists
#  effect: NoSchedule
  1. Deploy contrail using either of the below commands
kubectl apply -f contrail-host-ubuntu-ha.yaml

OR

kubectl apply -f contrail-host-ubuntu-ha-independent-k8s.yaml
  1. Verifying contrail status on all contrail containers Look for contrail pod name using the below command
kubectl get pods -n kube-system
contrail-analytics-9m545         1/1       Running   1          23h
contrail-analyticsdb-cpdjn       1/1       Running   1          23h
contrail-controller-gd5vl        1/1       Running   1          23h
contrail-kube-manager-82fcq      1/1       Running   1          23h
contrail-vrouter-agent-vwmbk     1/1       Running   1          23h

Check the contrail-status for all the above pods, using below command

kubectl exec -it <contrail-pod-name> -n kube-system -- contrail-status   

A one liner for above one would be

for pod_name in `kubectl get pods -n kube-system -o wide | grep contrail | awk '{print $1}'`; do kubectl exec -it  $pod_name -n kube-system -- contrail-status ; done

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.