diff --git a/README.md b/README.md index d2f5429..5f1b507 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,16 @@ # SMARTER Demo Deployment Instructions [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/smarter)](https://artifacthub.io/packages/search?repo=smarter) -## This demo makes the following assumptions about your environment + +The demo can be deployed by using the terraform script on this repository [Terraform](terraform) and following the [readme](terraform/README.md). It is also described on the section "Deploy using terraform" below. + +## Deploy using terraform + +If you have an AWS account, a terraform script is available on this repository at [Terraform](terraform) and a [readme](terraform/README.md) describes how to use it. This script will allocate an AWS EC2 Graviton instance, install k3s and helm and install all the charts needed to run this demo. The only missing part is one or more edge nodes that the user needs to provide. + +## Step by step deployment + +### This demo makes the following assumptions about your environment if deployed using helm charts In this guide we assume you have done the following: - You should have an installed InfluxDB and Grafana instance in a separate kubernetes cluster (cloud or local). @@ -28,7 +37,6 @@ In this guide we assume you have done the following: - You must be able to reach your edge node via IP on ports `22`(ssh) and `2520`(Webserver) from your dev machine for parts of this demo to work - The node must be able to reach your k3s-edge-server and cloud-data-node via IP -## Deploy demo - To deploy the base system components common to all edge nodes, as well as the demo applications, we opt to use **Helm v3**. To install helm on the device which you are managing your k3s edge cluster with, you can follow the guide [here](https://helm.sh/docs/intro/install/#from-script). - Ensure in your environment that your kubeconfig is set properly. As a quick sanity check you can run: ```bash diff --git a/terraform/.gitignore b/terraform/.gitignore new file mode 100644 index 0000000..d9ba4ec --- /dev/null +++ b/terraform/.gitignore @@ -0,0 +1,4 @@ +.terraform.lock.hcl +.terraform +ssh +terraform.tfstate diff --git a/terraform/README.md b/terraform/README.md new file mode 100644 index 0000000..ffb7db3 --- /dev/null +++ b/terraform/README.md @@ -0,0 +1,100 @@ +# Terraform script to install smarter on AWS EC2 + +This script installs SMARTER example using helm charts into one AWS EC2 instance. + +This figure shows the components of the application and where they reside. +![SMARTER](SMARTER_example.png) + +It assumes that the environment variables AWS\_ACCESS\_KEY\_ID, AWS\_SECRET\_ACCESS\_KEY and AWS\_SESSION\_TOKEN are set correctly so Terraform can access AWS. +Set the following variables to correct values: +region (provider "aws"): AWS region to allocate an EC2 instance on. + +Required variables: + +* letsencrypt\_email + +Optional variables: + +* deployment\_name: Prefix to apply to object names. +* AWS\_EC2\_instance\_type: instance type to be used +* AWS\_VPC\_subnet\_id: subnet_id use the default of the VPC if this is not defined + +## Running + +An template.tfvars is provided that can be copied so all the variables are set in this file and referenced by the option -var-file="smarter-variables.tfvars" where smarter-variables.tfvars is the name of the file used to set the variables. Commented variables are ignored. + +### Run the following commands if using the smarter-variables.tfvars optionn + +``` +terraform init +# optional: terraform plan -var-file="smarter-variables.tfvars" +terraform apply -var-file="smarter-variables.tfvars" +``` + +### Run the following commands if setting the variables on the command line + +``` +terraform init +# optional: terraform plan -var "letsencrypt_email=" +terraform apply -var "letsencrypt_email=" +``` + +## Checking status of installation + +Please observe that the full installation of k3s, helm charts in the EC2 instance can take up to 15min (expected around 10min) with various parts of the system being available at different times. If it is desired to follow the installation the command below will print the current log and follow it + +```bash +ssh -i ssh/-prod-k3s.pem ubuntu@ "tail -f /var/log/cloud-init-output.log" +``` + +## Outputs + +Terraform will output the name of EC2 instance allocated and password/ID generated by Terraform. + +Grafana web interface can be accessed by https://grafana.\.sslip.io with user admin and password \. + +A ssh directory will be created locally containing a private/public SSH key that can be used to access the instance using the following command: + +```bash +ssh -i ssh/-prod-k3s.pem ubuntu@ +``` + +K3s cloud access on the instance (running the cloud containers) can be achieved by setting KUBECONFIG to /etc/rancher/k3s/k3s.yaml. It should be already be set for the ubuntu user at the end of the installation. +K3s edge, that manages the edge devices and applications running on them, can be accessed by setting KUBECONFIG as $(pwd)/k3s.yaml.\, that also will be available at the end of the installation. + +Helm was used to install charts and can be used to manage them by setting the correct KUBECONFIG. + +The edge devices can be installed (Raspberry pi4 for example) by running the following script. The script will install a k3s agent and configure that agent to be a node for k3s edge running on the EC2 instance. + +``` +wget https://grafana..sslip.io//k3s/k3s-start.sh. | bash -s - +``` + +Token and k3s.yaml file can be accessed by: + +``` +wget https://grafana..sslip.io//k3s/token. | bash -s - +wget https://grafana..sslip.io//k3s/k3s.yaml. | bash -s - +``` + +# Troubleshooting + +## AWS authentication + +Use the AWS credentials provided in the "Get credentials for ProjAdmins" page. +Terraform expects the following environment variables: AWS\_ACCESS\_KEY\_ID, AWS\_SECRET\_ACCESS\_KEY and AWS\_SESSION\_TOKEN. + +## Networking + +If an error is reported about "default subnet not found", a subnet was not defined as default for the VPC. A subnet can be set using the AWS\_VPC\_subnet\_id variable. + +## Debugging information + +Log in to the EC2 machine using the ssh command + +```bash +ssh -i ssh/-prod-k3s.pem ubuntu@ +``` + +Please take a look at the log at /var/log/cloud-init-output.log and /var/log/cloud-init.log at the EC2 machine to determine where the program failed +The script that is executed is called part-002 diff --git a/terraform/SMARTER_example.png b/terraform/SMARTER_example.png new file mode 100644 index 0000000..0c2dca8 Binary files /dev/null and b/terraform/SMARTER_example.png differ diff --git a/terraform/k3s/main.tf b/terraform/k3s/main.tf new file mode 100644 index 0000000..a2a3e08 --- /dev/null +++ b/terraform/k3s/main.tf @@ -0,0 +1,143 @@ +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "~> 4.16" + } + } + + required_version = ">= 1.2.0" +} + +data "aws_ami" "ubuntu" { + most_recent = true + + filter { + name = "name" + #arm64 + values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-arm64-server-*"] + #x86_64 + #values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"] + } + owners = ["099720109477"] +} + +resource "random_string" "agent_token" { + length = 24 + special = false +} + +resource "random_string" "k3s_edge_id" { + length = 24 + special = false +} + +resource "aws_iam_instance_profile" "instance_profile" { + name = "${var.deployment_name}-InstanceProfile" + role = var.iam_role_name + count = var.iam_role_name == null ? 0 : 1 +} + +data "cloudinit_config" "userData" { + part { + content = < /tmp/variables +curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE=${var.kubeconfig_mode} K3S_TOKEN=${random_string.agent_token.result} sh - +echo "----- updating ubuntu" +apt-get update -y && apt-get upgrade -y && apt-get install awscli git -y +echo "----- Adding helm" +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash +echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> /home/ubuntu/.profile +echo "export KUBECONFIG=/etc/rancher/k3s/k3s.yaml" >> /home/ubuntu/.bashrc +export ADVERTISE_IP=$(curl http://169.254.169.254/latest/meta-data/public-ipv4) +export PUBLIC_HOSTNAME=$(curl http://169.254.169.254/latest/meta-data/public-hostname | cut -d '.' -f 1 | sed 's/^ec2-//') +export LOCAL_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4) +echo "----- Wating for k3s to start" +until [ -f /etc/rancher/k3s/k3s.yaml ] +do + sleep 5 +done +echo "----- Adding smarter-cloud to k3s" +sudo su - ubuntu bash -c "helm repo add smarter https://smarter-project.github.io/documentation;helm install my-smartercloud smarter/smarter-cloud --set email=${var.letsencrypt_email} --set host=grafana --set domain=$PUBLIC_HOSTNAME.sslip.io --set prometheus.grafana.adminPassword=${random_string.k3s_edge_id.result} --wait" +echo "----- Checking if TLS certificate was generated" +until [ ! -z "$(kubectl get secret/my-smartercloud-grafana-tls 2>/dev/null)" ] +do + echo "Certificate not generated yet, wait 5 seconds and test again" + sleep 5 +done +echo "----- Adding smarter-edge to k3s" +sudo su - ubuntu bash -c "helm install my-smartercloud-edge smarter/smarter-k3s-edge --set configuration.externalHostIP=$ADVERTISE_IP --set configuration.hostIP=$LOCAL_IP --set configuration.port=6444 --set configuration.portHTTP=80 --set configuration.id='${random_string.k3s_edge_id.result}' --set configuration.smarter_demo_labels=true --set configuration.host=grafana --set configuration.domain=$PUBLIC_HOSTNAME.sslip.io --set configuration.traefik=true --set configuration.certificateID=my-smartercloud-grafana-tls --set configuration.wwwpath=/k3s/ --wait" +echo "----- Waiting for k3s.yaml from k3s-edge" +until [ -f /home/ubuntu/k3s.yaml.${random_string.k3s_edge_id.result} ] +do + sudo su - ubuntu bash -c "wget --no-check-certificate https://grafana.$PUBLIC_HOSTNAME.sslip.io/k3s/k3s.yaml.${random_string.k3s_edge_id.result}" + if [ -z "$(grep 'kind: Config' /home/ubuntu/k3s.yaml.${random_string.k3s_edge_id.result})" ] + then + echo "Received a file but it is not a k3s.yaml file, removing" + rm /home/ubuntu/k3s.yaml.${random_string.k3s_edge_id.result} + fi + sleep 5 +done +echo "----- Adding smarter-edge to k3s-edge" +sudo su - ubuntu bash -c "export KUBECONFIG=/home/ubuntu/k3s.yaml.${random_string.k3s_edge_id.result};helm install --create-namespace --namespace smarter my-smartercloud-edge smarter/smarter-edge --wait;helm install --create-namespace --namespace smarter --set global.domain=$(curl http://169.254.169.254/latest/meta-data/public-hostname | cut -d '.' -f 2-) --set smarter-fluent-bit.fluentd.host=$(curl http://169.254.169.254/latest/meta-data/public-hostname | cut -d '.' -f 1) my-smartercloud-demo smarter/smarter-demo --wait" +echo "----- Finished installing" +EOF + content_type = "text/x-shellscript" + } + + part { + content = var.manifest_bucket_path == "" ? "" : <