Cloud Computing Project

This repository contains the project and its related files for the course Special Topics Cloud Computing Architectures, Processes and Operations (510.211) of the JKU Linz.

The original proposal for this project can be found in PROPOSAL.md

Introduction

The goal of this project is to create a dashboard for deploying routing engines (e.g. OSRM) and visualising their covered area.

Based on the dashboard, users are able to view which routing engines currently running, visualise which area they cover and deploy new routing engines based on selecting from a set of possible areas where data is existing.

All deployed routing engines are shown on a map, such that a user can easily see which areas are already covered and running. Setting up a new routing engine generally involves two steps: preprocessing (done via a kubernetes Job) and running the routing engine (done in a kubernetes Service).

The pre-processed street network data is stored on AWS S3 and Kubernetes (via Amazon EKS) is used to deploy the routing engines.

Description

In the dashboard, the user can see all currently active routing-areas, their mode and the areas they cover. Via Add New Region, the user is prompted to sub-page where a new area and the specific mode can be selected for deployment. Furthermore, we've added the possibility of deleting a specific routing-area by pressing Delete Selected Region after selecting an area.

We want to futher clarify how the exact steps that happen after a user chooses to deploy a new "area" for routing

When a user chooses to deploy a new routing-area, the following "deployment pipeline" is triggered:

• the dashboard-backend receives the requested area and a transport mode (either car, bike or foot) from the UI and starts a routing-preprocess (Kubernetes) job via the REST API.
• the routing-preprocess job...
  • downloads data about the street network in the selected area
  • pre-processes the street network data for the selected transport mode
  • uploads the pre-processed result to an S3 bucket
• the dashboard-backend is notified when the pre-processed results were uploaded and creates a kubernetes routing-app service
• the routing-app downloads the pre-processed routing data and starts serving the routing API

The "deployment" of each server is done along the following pipeline:

Deployment Architecture

Responsibilities

Sebastian Tanzer

• UI for dashboard application
• Part of backend for dashboard application

Christopher Stelzmüller

• Docker Container for routing engine (tuesd4y/osrm-backend-eks) - including options for running and preprocessing
• Kubernetes Service for running routing engines
• Kubernets Job for preprocessing routing engines
• Dashboard Backend for interfacing with kubernetes API

Research & Insights

Horizontal Pod Autoscaling

https://docs.aws.amazon.com/eks/latest/userguide/metrics-server.html

A HorizontalPodAutoscaler allows scaling up (or down) the number of running instances of a service based on its load. In order to create a horizontalAutoscaler, we generally have to specify which metrics should be observed (e.g. CPU utilisation), under what constraints Deployments can be scaled (e.g. 1-4 running instances) and what Deployment should be the target of scaling.

In our project, one AutoscalerInstance is created for each deployment of a routing service. We're scaling our services, such that an average CPU utilisation of 30% is reached. While this is (for production use) a rather low value, it helps illustrate the Autoscalers capabilities during our live demo.

In our project, the read autoscalers to deploye are created in a Kotlin Code and deployed using the Kubernetes Java client API, but an exemplary definition of an autoscaler can also be found in autoscaler.yaml.

In order for the Autoscaler to work correctly based on metrics, we need to deploy a Kubernetes metrics server, that is later used for measuring the utilisation of our Deployments. As outlined in the AWS EKS docs, a metrics-server can be easily deployed by running the following command.

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Logging into the Kubernetes API using AWS EKS

Logging into our kubernetes cluster with AWS API credentials proved more complicated than expected. Whereas the AWS root account can directly login to the AWS CLI using aws configure , create a kubeconfig file with aws eks update-kubeconfig --name routing-cluster and subsequently use the kubectl to interact with the cluster, this is not the case for other accounts.

In addition to giving the accounts permission to access the AWS EKS service, access to the kubernetes cluster itself has to be configured. The AWS docs for allowing user access to clusters outline that there is an aws-auth ConfigMap that outlines which users can access a cluster. We can add additional users by running kubectl edit -n kube-system configmap/aws-auth from the root account and adding a mapUsers block at top level that looks similar to the following (Make sure to use the correct arn of the AWS user).

mapUsers:
- userarn: arn:aws:iam::614832161943:user/uni-sebastian
  username: uni-sebastian
  groups:
    - system:masters

Configuring the Kubernetes Java API

Since our dashboard application is using the gradle build system, we can simply install the kubernetes Java Client by adding compile("io.kubernetes:client-java:14.0.0") to our gradle dependencies block. In order to access the API.

Before interacting with the kubernetes API, we first need to configure the API client. In our project, this is happening in the init block in KubernetesApiProvider.kt. With the following code part, we instruct the Kubernetes API Client to use the default loading mechanism for getting credentials.

val client = ClientBuilder.defaultClient()
setDefaultApiClient(client)

Afterwards, the kubernetes API can be accessed directly be the individual API Objects (e.g. by constructing a new CoreV1Api()). In order to be able to connect to and query the API, we also need to setup authentication and connection information to the kubernetes cluster - This is done using a kubeconfig file locally, or via a service account when the routing-dashboard is deployed.

Authentication via kubeconfig

If we're running the java routing-dashboard locally, and have a kubeconfig file in the default location (e.g. by creating it as outlined in the Logging into the Kubernetes API using AWS EKS-section), we need no further configuration

Authentication via Service accounts

In order to utilise the kubernetes API from within a cluster, we need to configure a ServiceAccount that has access to the API. Such a service account can also automatically be used by the kubernetes Java client in order to interact with the cluster.

As outlined in the kubernetes docs, we can create a new service account for our routing-dashboard application by running the following code snippet:

kubectl create serviceaccount -n routing-service routing-dashboard-admin
kubectl create rolebinding admin-sa \
  --clusterrole=cluster-admin \
  --serviceaccount=routing-service:routing-dashboard-admin \
  --namespace=routing-service

Note, that here we create a routing-dashboard-admin serviceaccount and give it admin access to our cluster. We also need to configure the deployment of our routing-backed (which is handled in routing-dashboard.yaml) such that the deployment automatically gets assigned the service account. This is done by adding the service account into the deplyments template spec.

...
  spec:
    serviceAccountName: routing-dashboard-admin
    containers:
    - name: routing-dashboard
    ...

Kubernetes Jobs

We're using a kubernetes job in order to download, preprocess and upload the files needed for our routing services. We've found, that our initial docker containers used for describing the processing inside a job did not complete with an exit code, but rather stayed running indefinitely. In order to solve this, we've reconfigured the underlying docker container to include an OSRM_EXIT_AFTER_UPLOAD environment variable - if this set to a non-empty value, the container exits after processing is done and data have been uploaded to S3.

Lessons Learned

• We've found out that using special or uppercase characters in kubernetes API name fields doesn't work.
• We've found out that configuring access to EKS clusters for user accounts is different or AWS root users and regular users. As outlined in the Research-section, this needs special handling.
• We've found out, that AWS has a very cool feature that automatically quarantines (i.e. disallow access to all operations) AWS accounts whose API keys are unintentionally uploaded to github or otherwise exposed to the internet.

Usage Example

Requirements

• namespace routing-service
• AWS credentials in routing-dashboard.yaml
• configure AWS S3 bucket url in Java code
• apply routing-dashboard.yaml
• Setup of infrastructure and user accounts

Tutorial

Setup & Prerequisites

• clone this repo from github https://github.com/tuesd4y/cloud-computing-project.git
• Make sure you have a connection to your kubernetes cluster, you have a routing-service namespace and have configured an routing-dashboard-admin as outlined in Authentication via Service accounts
• update routing-dashboard/routing-dashboard.yaml with your AWS S3 API keys
• deploy the routing dashboard using kubectl apply -f routing-dashboard/routing-dashboard.yaml
• port-forward the routing-dashboard service to your localhost by running kubectl port-forward service/routing-dashboard -n routing-service 8080:8080
• You can now access the routing dashboard at http://localhost:8080

Deploying New Service Area

By clicking on Add New Region a sub-window is opened, which allows to select a specific area the required mode of transport and submit it via Load the REST-Call for pre-processing is sent to the server.

Check Newly Added Area in UI

Depending on the size of the area, pre-preprocessing it and starting the specific routing server can take up to 24 hours. As soon as the area is ready, it will be shown in the UI under the Active Region Table and on the map.

Demonstrate autoscaling

• Run kubectl get hpa -n routing-service --watch in your terminal to see the currently running HorizontalPodAutoscalers
• In another window, run kubectl get services -n routing-service and choose one of the services for testing autoscaling, note its name
• Run kubectl run -i --tty load-generator -n routing-service --rm --image=busybox --restart=Never -- /bin/sh -c "while sleep 0.001; do wget -q -O- 'http://SERVICE:5000/route/v1/driving/13.388860,52.517037;13.385983,52.496891?steps=true'; done"
• You should now see the HorizontalPodAutoscaler scaling up your Deployment, and scale it down once the script in the other window is terminated.

Open questions and Next Steps

Hiding AWS Credentials from Storage

Currently, the AWS Credentials are added in the application code. In the future, we will be using Kubernetes Secrets to store them so that we do not need to include confidential data in our application code or yaml files.

Show Current Scaling & Configure Mem/CPU

The parameters for scaling a deployment are set in AutoscalerTemplate, while the allocation of memory and CPU resources is handled in DeploymentTemplate. For now, those are all static values. However, depending on the size of an area, the needed resources can be quite different. Therefore, we will add the possibility of setting these parameters in the UI for each area specifically.

DNS based on service names + security for outside access

In order to access our routing service from the outside world, we want to automatically set up URLs for them so that they can easily be reached and secure them.