Magma Orchestrator (v1.4.0) on Docker Swarm Cluster

[edaspb]

Step-by-Step Installation Notes

Links:
Magma Platform: https://github.com/magma/magma
Docker Swarm: https://docs.docker.com/engine/install/debian/

Originally Orchestrator enviroment is designed for deployment on AWS platform.
This guide provides steps and describes how to deploy Orchestrator on your local cloud.

Prerequisites

1. Built and published Orchestrator Containers. See https://magma.github.io/magma/docs/orc8r/deploy_build
2. At least one VM with Debian 10.5 or Ubuntu Server 20.04 installed. It is recommended two or more VMs.

## 0. Dependency Installation and Download Magma source code ##
This step has to be done on both VMs:

apt install apt-transport-https software-properties-common ca-certificates gnupg curl -y
wget https://github.com/magma/magma/archive/refs/tags/v1.4.0.tar.gz
tar xfz v1.4.0.tar.gz && mv magma-1.4.0 magma

## 1. PostgresDB installation and configuration ##
All steps are executed on orc8r1 server only.

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -

sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'
apt update
apt install postgresql postgresql-contrib -y

echo "listen_addresses = '0.0.0.0/0'" >> /etc/postgresql/13/main/postgresql.conf
echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/13/main/pg_hba.conf
systemctl start postgresql && systemctl enable postgresql

## 2. Docker Swarm Installation ##

All steps are executed on both servers, orc8r1 and orc8r2

2.1 apt install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
2.2 curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
2.3 add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
2.4 apt update
2.5 apt install docker-ce docker-ce-cli containerd.io -y
2.6 systemctl start docker && systemctl enable docker

## 3. Docker Swarm Activation ##
On orc8r1 VM. (Please don't forget to change IP address):

3.1 docker swarm init --advertise-addr 10.202.6.68
orc8r1 became Master Swarm node
3.2 Copy output string from previous command (something like "docker swarm join --token XXXXX")
Then, on orc8r2 VM:
3.3 Apply the command which you've copied on step 3.2

On orc8r1, label nodes:

3.4 docker node update --label-add controller=true orc8r1
3.5 docker node update --label-add metrics=true orc8r2

## 4. Metrics node preconfiguration ##
On orc8r2:

4.1 mkdir -p /magma
4.2 cp -r magma/orc8r/cloud/docker/metrics-configs /magma/
4.3 chmod -R 777 /magma

## 5. Controller node preconfiguration ##
On orc8r1:

5.1 mkdir -p /magma/certs
5.2 mkdir -p /magma/fluentd/conf
5.3 cp magma/orc8r/cloud/docker/fluentd/conf/fluent.conf /magma/fluentd/conf/

## 6. Create Certificate chain ##
On orc8r1:

6.1 wget https://raw.githubusercontent.com/edaspb/Magma-Orchastrator-in-a-Docker-Swarm/1.4.0/scripts/certs.sh
6.2 chmod +x certs.sh
6.3 ./certs.sh yourdomain.com
6.4 chmod 644 /magma/certs/controller.key

## 7. Roll out Controller and Metrics stack ##
On orc8r1:

7.1 wget https://raw.githubusercontent.com/edaspb/Magma-Orchastrator-in-a-Docker-Swarm/1.4.0/compose/docker-controller.yml
7.2 wget https://raw.githubusercontent.com/edaspb/Magma-Orchastrator-in-a-Docker-Swarm/1.4.0/compose/docker-metrics.yml

Please modify docker-compose-controller.yml and docker-metrics.yml according to your repo/images/lables names.

7.3 docker stack deploy --compose-file docker-controller.yml magma
7.4 docker stack deploy --compose-file docker-metrics.yml magma

## 8. Create NMS Certificate ##
On orc8r1:

8.1 export cntrl_con=magma_controller.1.$(docker service ps -f 'name=magma_controller.1' magma_controller -q --no-trunc | head -n1)
8.2 docker exec -it $cntrl_con bash -c "envdir /var/opt/magma/envdir /var/opt/magma/bin/accessc add-admin -duration 3650 -cert /var/opt/magma/bin/admin_operator admin_operator"
8.3 docker exec -it $cntrl_con bash -c "openssl pkcs12 -export -out /var/opt/magma/bin/admin_operator.pfx -inkey /var/opt/magma/bin/admin_operator.key.pem -in /var/opt/magma/bin/admin_operator.pem"
(press Enter twice)
8.4 for certfile in admin_operator.pem admin_operator.key.pem admin_operator.pfx; do docker cp ${cntrl_con}:/var/opt/magma/bin/${certfile} /magma/certs/${certfile}; done

## 9 Roll out NMS stack ##
On orc8r1:

9.1 wget https://raw.githubusercontent.com/edaspb/Magma-Orchastrator-in-a-Docker-Swarm/1.4.0/compose/docker-nms.yml

Please modify docker-nms.yml according to your repo/images/lables names.

On orc8r2:

9.2 mkdir -p /magma/magmalte
9.3 cp -r magma/nms/app/packages/fbcnms-magma-api /magma/magmalte/
9.4 cp -r magma/nms/app/packages/magmalte/app /magma/magmalte/
9.5 cp -r magma/nms/app/packages/magmalte/scripts /magma/magmalte/
9.6 cp -r magma/nms/app/packages/magmalte/server /magma/magmalte/
9.7 cp -r magma/nms/app/packages/magmalte/grafana /magma/magmalte/
9.8 cp -r magma/nms/app/packages/magmalte/alerts /magma/magmalte/
9.9 mkdir -p /magma/docker_ssl_proxy
9.10 cp magma/nms/app/packages/magmalte/docker/docker_ssl_proxy/* /magma/docker_ssl_proxy/

On orc8r1:
9.11 docker stack deploy --compose-file docker-nms.yml magma

Wait few minutes...

On orc8r2:

9.12 export nms_con=$(docker ps -f 'name=magma_magmalte.1' --format "{{.Names}}")
9.13 docker exec -it $nms_con yarn setAdminPassword master your@email.com YourPassNot1234

That is all, after all these steps you will have your Orchestrator up and running.

## 10 DNS ##

At least the following records must be configured:

yourdomain.com
master.yourdomain.com
api.yourdomain.com
bootstrapper-controller.yourdomain.com
controller.yourdomain.com
fluentd.yourdomain.com

11 AGW Control Proxy /etc/magma/control_proxy.yml Config file example

cloud_address: controller.**yourdomain.com**
cloud_port: 7443

bootstrap_address: bootstrapper-controller.**yourdomain.com**
bootstrap_port: 7444

fluentd_address: fluentd.**yourdomain.com**
fluentd_port: 24224

[karthiksubraveti]

@edaspb this is great. Thanks for adding this. @jaredmullane @wallyrb should we add this under the new documentation changes ?

[hcgatewood]

@karthiksubraveti I'm unclear if this is something we want to directly support. Perhaps the best path forward is to include a link to this Discussion as a starting point for people interested in deploying to Docker Swarm (as an "alternative" deployment method)