Skip to content

Latest commit

 

History

History
354 lines (230 loc) · 17.2 KB

08-workload.md

File metadata and controls

354 lines (230 loc) · 17.2 KB
Raw

Deploy a Basic Workload using the Fruit Smoothie Ratings Application

The application consists of a web frontend, an API service and a MongoDB database.

Because the infrastructure has been deployed in a private AKS cluster setup with private endpoints for the container registry and other components, you will need to perform the application container build and the publishing to the Container Registry from the Dev Jumpbox in the Hub VNET, connecting via the Bastion Host service. If your computer is connected to the hub network, you may be able to just use that as well. The rest of the steps can be performed on your local machine by using AKS Run commands which allow access into private clusters using RBAC. This will help with improving security and will provide a more user-friendly way of editing YAML files.

Connecting to the Bastion Host

  1. Use Bastion Host to connect to the jumpbox.
  2. Enter the username and password. If you have used a public key, then select upload private key (corresponding to the public key) to connect.
  3. Once you connect ensure you permit the site to read the content of your clipboard

  • Clone it on the jumpbox.

    git clone https://github.com/Azure/AKS-Landing-Zone-Accelerator

  • Run the script below to install the required tools (Az CLI, Docker, Kubectl, Helm etc). Navigate to "AKS-Landing-Zone-Accelerator/Scenarios/AKS-Secure-Baseline-PrivateCluster/Terraform/04-Network-Hub" folder.

    cd AKS-Landing-Zone-Accelerator/Scenarios/AKS-Secure-Baseline-PrivateCluster/Terraform/04-Network-Hub
chmod +x script.sh
sudo ./script.sh

  • Login to Azure

    TENANTID=<tenant id>
az login -t $TENANTID --debug

  • Ensure you are connected to the correct subscription

    az account set --subscription <subscription id>

Connect the Container Registry Private link to the Hub network

Since the Container registry can only be accessed via private link, we need to connect it to the network where jumpbox or whichever computer we are using to create the container images resides. We already added the container registry to the spoke network where the cluster resides using terraform.

  1. Go to Azure portal

  2. Find the Private DNS zone created for acr. This should be in the landing zone resource group (ESLZ-SPOKE for example)

    Location of private link for acr

  3. Click on Virtual network links in the left blade under Settings

  4. Click on + Add in the in the top left of the next screen

  5. enter a name for the link eg hub_to_acr

  6. Select the hub virtual network for the Virtual network field

  7. Click on OK at the bottom

Provide yourself Access to Create Secrets in your Key vault

  1. Go to the Azure portal and find your Key Vault. This should be in the landing zone resource group (ESLZ-SPOKE for example)
  2. You should see your pod-identity-example managed identity as well as the azurekeyvaultsecrets identity. The pod identity will provide pods access to the pull secrets from the keyvault. The azurekeyvaultsecrets identity will be used by the keyvault driver. If either of these are missing, perhaps you are missing a step.
  3. Click on Access policies under Settings in the left bladeadd access policy
  4. Select the required access policies add access policy
  5. Under Select principal click on the None selected link and select the user group(s) you created for this to provide you and everyone in the group access to the Key vault
  6. Click Select at the bottom of the the screen
  7. Important: Click Save at the top of the next screen to save the changes add access policy

Build Container Images

Clone the required repos to the Dev Jumpbox:

  1. The Ratings API repo
cd ..
git clone https://github.com/MicrosoftDocs/mslearn-aks-workshop-ratings-api.git
  1. The Ratings Web repo
git clone https://github.com/MicrosoftDocs/mslearn-aks-workshop-ratings-web.git

Navigate to each of the application code directories, build and tag the containers with the name of your Azure Container Registry and push the images to ACR. // Make sure it is the correct ACR

# enter the name of your ACR below
SPOKERG=<resource group name for spoke>
ACRNAME=<ACR NAME>
cd mslearn-aks-workshop-ratings-api
sudo docker build . -t $ACRNAME.azurecr.io/ratings-api:v1
cd ../mslearn-aks-workshop-ratings-web
sudo docker build . -t $ACRNAME.azurecr.io/ratings-web:v1

Log into ACR

sudo az acr login -n $ACRNAME

Push the images into the container registry. Ensure you are logged into the Azure Container Registry, you should show a successful login from the command above.

sudo docker push $ACRNAME.azurecr.io/ratings-api:v1
sudo docker push $ACRNAME.azurecr.io/ratings-web:v1

To verify they have been pushed run the following commands:

az acr repository show -n $ACRNAME --image ratings-api:v1
az acr repository show -n $ACRNAME --image ratings-web:v1

Create the secret in key vault. You may use anything you'd like for the username and password for the MongoDB database but this needs to match what you will use when you create the helm chart in the next steps.

Note: Passwords with special characters in a connection string might break the connection because of wrong encoding.

Note: Ensure you have access to create passwords in keyvault by going to the Key vault in Azure Portal, clicking on Access Policies and Add Access Policy. Don't forget to hit "Save" after adding yourself or user group to Key vault access

# update keyvault name, username and password before running the command below
KEYVAULTNAME=<key vault name>
PGUSERNAME=<postgres db user name>
PGPASSWORD=<postgres db password>
az keyvault secret set --name mongodburi --vault-name $KEYVAULTNAME --value "mongodb://$PGUSERNAME:$PGPASSWORD@ratings-mongodb.ratingsapp:27017/ratingsdb"

The following Steps can be performed using AKS Run Commands from your local machine provided you have the correct permissions.

Deploy the database into the cluster

The following steps can be performed using AKS Run Commands from your local machine provided you have the correct permissions.

Ensure the AKS run commands are working as expected.

# create environment variable for cluster and its resource group name
ClusterRGName=<cluster resource group name>
ClusterName=<AKS cluster name>
az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl get nodes"

On the Kubernetes cluster, create a namespace for the Ratings Application.

az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl create namespace ratingsapp"

The MongoDB backend application is installed using Helm. Your username and password must be the same username and password using in the connection string secret that was created in Key vault in the previous step.

az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "helm repo add bitnami https://charts.bitnami.com/bitnami && helm install ratings bitnami/mongodb --namespace ratingsapp --set auth.username=$PGUSERNAME,auth.password=$PGPASSWORD,auth.database=ratingsdb"

Deploy the workload into the cluster

We need to deploy the key vaults secret add on somewhere before this step, it is in the original markdown file but not this new one

In this section, you will be manipulating some of the deployment yaml files, replacing some entries related with Azure Key Vault, Azure Container Registry and Microsoft Entra ID references like ClientID, TenantID etc.

All files will be under the following folder: "Scenarios/AKS-Secure-Baseline-PrivateCluster/Apps/RatingsApp"

You will have to carefully update the following files:

Deploy workload

Navigate to "Scenarios/AKS-Secure-Baseline-PrivateCluster/Apps/RatingsApp" folder.

  1. Updating api-secret-provider-class.yaml

    Update the "api-secret-provider-class.yaml" file to reflect the correct value for the following items:

    • Key Vault name
    • Client ID for the AKS Key Vault Add-on
    • Tenant ID for the subscription.

    If you don't have the Client ID, you can find it by going to the Key vault and clicking on Access Policies in the left blade. Find the identity that starts with "azurekeyvaultsecrets-name of your aks cluster", then look for the resource by searching for the name in the search bar at the top. When you click on the resource, you will find the Client ID on the right side of the screen.

    Deploy the edited yaml file.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f api-secret-provider-class.yaml -n ratingsapp" --file api-secret-provider-class.yaml

  2. Updating 1-ratings-api-deployment.yaml

    Update the "1-ratings-api-deployment.yaml" file to reflect the correct name for the Azure Container Registry. Deploy the file.

       az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 1-ratings-api-deployment.yaml -n ratingsapp" --file 1-ratings-api-deployment.yaml

  3. Ensure the ratings-api deployment was successful.

    If you don't get a running state then it is likely that the pod was unable to get the secret from Key vault. This may be because the username and password of the db doesn't match the connection string that was created in Key vault or because the proper access to the Key vault wasn't granted to the azuresecret identity.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl get pods -n ratingsapp"

    You can troubleshoot container creation issues by running

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl describe pod <pod name> -n ratingsapp"
az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl logs <pod name> -n ratingsapp"

  4. Updating 2-ratings-api-service.yaml

    Deploy the "2-ratings-api-service.yaml" file.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 2-ratings-api-service.yaml -n ratingsapp" --file 2-ratings-api-service.yaml

  5. Updating 3a-ratings-web-deployment.yaml

    Update the "3a-ratings-web-deployment.yaml" file to reflect the correct name for the Azure Container Registry. Deploy the file.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 3a-ratings-web-deployment.yaml -n ratingsapp" --file 3a-ratings-web-deployment.yaml

  6. Deploy the "4-ratings-web-service.yaml" file.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 4-ratings-web-service.yaml -n ratingsapp" --file 4-ratings-web-service.yaml

(Optional) Deploy the Ingress without support for HTTPS

This step is optional. If you would like to go straight to using https which is the secure option, skip this section and go straight to the Update the Ingress to support HTTPS traffic section.

It is important to first configure the NSG for the Application Gateway to accept traffic on port 80 if using the HTTP option. Run the following command to allow HTTP.

   APPGWSUBNSG=<Name of NSG for AppGwy>
   az network nsg rule create -g $SPOKERG --nsg-name $APPGWSUBNSG -n AllowHTTPInbound --priority 1000 \
      --source-address-prefixes '*' --source-port-ranges '*' \
      --destination-address-prefixes '*' --destination-port-ranges 80 --access Allow \
      --protocol Tcp --description "Allow Inbound traffic through the Application Gateway on port 80"

  1. Deploy the "5-ratings-web-ingress.yaml" file.

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 5-http-ratings-web-ingress.yaml -n ratingsapp" --file 5-http-ratings-web-ingress.yaml

  2. Get the ip address of your ingress controller

    az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl get ingress -n ratingsapp"

Check your deployed workload

  1. Copy the ip address displayed, open a browser, navigate to the IP address obtained above from the ingress controller and explore your website

    deployed workload

It is important to delete the rule that allows HTTP traffic to keep the cluster safe since we have completed the test.

   az network nsg rule delete -g $SPOKERG --nsg-name $APPGWSUBNSG -n AllowHTTPInbound

the optional steps end here

Deploy the Ingress with HTTPS support

Please note: This section is still in development

A fully qualified DNS name and a certificate are needed to configure HTTPS support on the the front end of the web application. You are welcome to bring your own certificate and DNS if you have them available, however a simple way to demonstrate this is to use a self-signed certificate with an FQDN configured on the IP address used by the Application Gateway.

Objectives

  1. Configure the Public IP address of your Application Gateway to have a DNS name. It will be in the format of customPrefix.region.cloudapp.azure.com
  2. Create a certificate using the FQDN and store it in Key Vault.

Creating Public IP address for your Application Gateway

  1. Find your application gateway in your landing zone resource group and click on it. By default it should be in the spoke resource group.

  2. Click on the Frontend public IP address

    front end public ip address

  3. Click on configuration in the left blade of the resulting page.

  4. Enter a unique DNS name in the field provided and click Save.

    creating nds

Create the self-signed certificate using Lets Encrypt

We are going to use Lets Encrypt and Cert-Manager to provide easy to use certificate management for the application within AKS. Cert-Manager will also handle future certificate renewals removing any manual processes.

  1. First of all, you will need to install cert-manager into your cluster.
   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.0/cert-manager.yaml"

First of all this will create a new namespace called cert-manager which is where all of the resources for cert-manager will be kept. This will then go ahead and download some CRDs (CustomResourceDefinitions) which provides extra functionality in the cluster for the creation of certificates.

We will then proceed to test this certificate process with a staging certificate.

  1. Edit the 'certificateIssuer.yaml' file and include your email address. This will be used for certificate renewal notifications.

Deploy certificateIssuer.yaml

   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f certificateIssuer.yaml -n ratingsapp" --file certificateIssuer.yaml
  1. Edit the '5-https-ratings-web-ingress.yaml' file with the FQDN of your host that you created earlier on the public IP of the Application Gateway.

Deploy 5-https-ratings-web-ingress.yaml

   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl apply -f 5-https-ratings-web-ingress.yaml -n ratingsapp" --file 5-https-ratings-web-ingress.yaml

After updating the ingress, A request will be sent to letsEncrypt to provide a 'staging' certificate. This can take a few minutes. You can check on the progress by running the below command. When the status Ready = True. You should be able to browse to the same URL you configured on the PIP of the Application Gateway earlier.

   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl get certificate -n ratingsapp"

If you notice the status is not changing after a few minutes, there could be a problem with your certificate request. You can gather more information by running a describe on the request using the below command.

   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl get certificaterequest -n ratingsapp"
   az aks command invoke --resource-group $ClusterRGName --name $ClusterName   --command "kubectl describe certificaterequest <certificaterequestname> -n ratingsapp"

Upon navigating to your new FQDN you will see you receive a certificate warning because it is not a production certificate. deployed workload https

Note: For production clusters, it is better to use a paid for SSL certificate because they can offer better liability and protection than a free SSL certificate.

Next Step

▶️ Cleanup