LOVE-1113 Using composability on basic AKS/EKS/GKE cluster blueprints

aws/basic-eks-cluster/README.md

@@ -1,10 +1,10 @@
-# Amazon EKS Cluster
+# Basic AWS Elastic Kubernetes Service(EKS) Cluster
 
 ## Introduction
 
 Amazon Elastic Container Service for Kubernetes (EKS) allows you to deploy, manage, and scale containerized applications in the cloud using Kubernetes.
 
-Use this blueprint to provision a simple EKS cluster. The release template that the blueprint generates provisions a new cluster.
+Use this blueprint to provision an EKS cluster using CloudFormation.
 
 ## Before you get started
 
@@ -16,66 +16,60 @@ If you're new to XebiaLabs blueprints, check out:
 
 ## Usage
 
-To use this blueprint, run `xl blueprint` and select:
+To use this blueprint, run `xl blueprint` in an empty directory and select:
 
-    aws/basic-eks-cluster
+```plain
+aws/basic-eks-cluster
+```
 
 ## Tools and technologies
 
 This blueprint includes the following tools and technologies:
 
 * Target:
     * [Amazon Elastic Container Service for Kubernetes (EKS)](https://aws.amazon.com/eks/)
-    * [Amazon Simple Storage Service (S3)](https://aws.amazon.com/s3/)
 * Tools:
-    * [XebiaLabs Release Orchestration](https://xebialabs.com/products/xl-release/)
     * [XebiaLabs Deployment Automation](https://xebialabs.com/products/xl-deploy/)
-    * [AWS CloudFormation](https://aws.amazon.com/cloudformation/)
     * [Kubernetes](https://kubernetes.io/)
-    * [AWS Lambda](https://aws.amazon.com/lambda/)
+    * [CloudFormation](https://aws.amazon.com/cloudformation/)
 
-## Minimum Required versions
+## Minimum required versions
 
-This blueprint version requires at least the below versions of the specified tools to work properly.
+This blueprint version requires at least the following versions of the specified tools to work properly:
 
-XL Release: Version 9.0.0
-XL Deploy: Version 9.0.0
-XL CLI: Version 9.0.0
+* XL Deploy: Version 9.0.0
+* XL CLI: Version 9.0.0
 
 ## Prerequisites
 
 To run the YAML that this blueprint generates, you need:
 
-* XebiaLabs Release Orchestration and Deployment Automation up and running
-* Access to an AWS account to deploy the application to
+* XebiaLabs Deployment Automation up and running
+* Access to an AWS account that can deploy EKS clusters and create S3 storage
+* [AWS CLI](https://aws.amazon.com/cli/)
 
 ## Information required
-
-This blueprint requires:
-
-* AWS credentials
-* An AWS region
+* AWS Access Key (find it in `~/.aws/credentials`)
+* AWS Secret Access Key (find it in `~/.aws/credentials`)
 
 ## Output
 
 This blueprint will output:
 
-* Release templates
-* AWS CloudFormation templates
+* CloudFormation templates
 * Infrastructure:
-    * Amazon EKS cluster (master, workers, config map, namespace)
-    * Amazon S3 bucket for the application
-    * AWS Lambda artifacts for the application
-    * Networking infrastructure: Virtual Private Cloud (VPC), subnets, route table
-    * Security infrastructure: IAM role
-* A docker-compose setup for XL Release and XL Deploy
+    * EKS cluster (master, nodes)
+
+**Note:** You will find more instructions in `xebialabs/USAGE-aws-basic-eks-cluster.md` after you have run the blueprint.
 
-## Tips and tricks
+## Notes
 
-* The YAML that the blueprint generates includes optional steps to remove the application and deprovision the cluster.
+None
 
 ## Labels
 
 * Cloud
+* Amazon
 * AWS
 * Kubernetes
+* CloudFormation

aws/basic-eks-cluster/__test__/answers.yaml

@@ -0,0 +1,9 @@
+UseAWSCredentialsFromSystem: false
+AWSAccessKey: A
+AWSAccessSecret: B
+AWSRegion: eu-west-3
+ClusterName: cluster1
+NodeAutoScalingGroupDesiredSize: 3
+NodeInstanceType: t2.medium
+GenerateDockerComposeSetup: false
+ClusterVersion: '1.11'

aws/basic-eks-cluster/__test__/test.yaml

@@ -0,0 +1,11 @@
+answers-file: answers.yaml
+expected-files:
+- xebialabs/aws-basic-eks-cluster-applications.yaml
+- xebialabs/aws-basic-eks-cluster-environments.yaml
+- xebialabs/aws-basic-eks-cluster.yaml
+- xebialabs/USAGE-aws-basic-eks-cluster.md
+- kubernetes/aws-auth-cm.yaml
+- cloudformation/eks-master.yaml
+- cloudformation/eks-user.yaml
+- cloudformation/eks-vpc.yaml
+- cloudformation/eks-workers.yaml

aws/basic-eks-cluster/blueprint.yaml

@@ -2,33 +2,29 @@ apiVersion: xl/v2
 kind: Blueprint
 
 metadata:
-  name: AWS-EKS-Cluster
+  name: AWS-EKS-Basic
   description: |
-    The blueprint deploys a simple AWS EKS Cluster (with EFS if requested).
-    XL Deploy does the provisioning and deployment, while XL Release orchestrates everything.
+    The blueprint provisions a very basic AWS EKS cluster. XL Deploy does the provisioning.
   author: XebiaLabs
   version: 2.0
-  instructions: Please read the generated file "xebialabs/USAGE.md" for further usage instructions.
+  instructions: Please read the generated file "xebialabs/USAGE-aws-basic-eks-cluster.md" for further usage instructions.
 
 spec:
   parameters:
-  # General variables
-  - name: AppName
-    type: Input
-    prompt: What is the name of the application? (16 characters max)
-    description: Application name will be used to generate cloud resource names
-    validate: !expr "regex('^[a-zA-Z0-9-]{1,16}$', AppName)"
-
+  # ############################################################################
   # AWS specific variables
+  # ############################################################################
   - name: UseAWSCredentialsFromSystem
     type: Confirm
     prompt: Do you want to use AWS credentials from ~/.aws/credentials file?
     promptIf: !expr "awsCredentials('IsAvailable')"
+
   - name: AWSAccessKey
     type: SecretInput
     prompt: What is the AWS Access Key ID?
     promptIf: !expr "!UseAWSCredentialsFromSystem"
     default: !expr "awsCredentials('AccessKeyID')"
+
   - name: AWSAccessSecret
     type: SecretInput
     prompt: What is the AWS Secret Access Key?
@@ -39,35 +35,40 @@ spec:
     type: Select
     prompt: "Select the AWS region:"
     options:
-    # This needs to be updated in the RegionMap of cloudformation/eks-workers.yaml as well
-      - label: US West (Oregon)
-        value: us-west-2
-      # commented out because of https://github.com/boto/boto3/issues/811
-      # - us-east-1
-      - label: US East (Ohio)
-        value: us-east-2
-      - label: EU (Frankfurt)
-        value: eu-central-1
-      - label: EU (Ireland)
-        value: eu-west-1
-      - label: EU (London)
-        value: eu-west-2
-      - label: EU (Paris)
-        value: eu-west-3
-      - label: Asia Pacific (Tokyo)
-        value: ap-northeast-1
-      - label: Asia Pacific (Seoul)
-        value: ap-northeast-2
-      - label: Asia Pacific (Mumbai)
-        value: ap-south-1
-      - label: Asia Pacific (Singapore)
-        value: ap-southeast-1
-      - label: Asia Pacific (Sydney)
-        value: ap-southeast-2
+    # commented out because of https://github.com/boto/boto3/issues/811
+    # - label: US East (N. Virginia)
+    #   value: us-east-1
+    - label: US East (Ohio)
+      value: us-east-2
+    - label: US West (Oregon)
+      value: us-west-2
+    - label: Asia Pacific (Singapore)
+      value: ap-southeast-1
+    - label: Asia Pacific (Tokyo)
+      value: ap-northeast-1
+    - label: Asia Pacific (Sydney)
+      value: ap-southeast-2
+    - label: Asia Pacific (Seoul)
+      value: ap-northeast-2
+    - label: Asia Pacific (Mumbai)
+      value: ap-south-1
+    - label: EU (Ireland)
+      value: eu-west-1
+    - label: EU (Frankfurt)
+      value: eu-central-1
+    - label: EU (London)
+      value: eu-west-2
+    - label: EU (Paris)
+      value: eu-west-3
+    - label: EU (Stockholm)
+      value: eu-north-1
 
   - name: ClusterName
     type: Input
-    prompt: What is the name of the cluster?
+    prompt: What do you want to name the cluster?
+    description: Cluster name will be used to generate cloud resource names
+    validate: !expr "regex('^[a-zA-Z0-9-]{1,16}$', ClusterName)"
+
   - name: ClusterVersion
     type: Select
     default: '1.10'
@@ -144,11 +145,6 @@ spec:
     - p3.8xlarge
     - p3.16xlarge
 
-  - name: ProvisionEFS
-    type: Confirm
-    prompt: "Do you want to provision Amazon Elastic File System?"
-    default: false
-
   - name: GenerateDockerComposeSetup
     type: Confirm
     prompt: "Do you want to generate a docker-compose setup with the required tools to run this blueprint?"
@@ -176,17 +172,14 @@ spec:
       value: false
 
   files:
-  - path: xebialabs/xld-infra-env.yaml.tmpl
-  - path: xebialabs/xld-cloudformation-apps.yaml.tmpl
-  - path: xebialabs/xlr-pipeline-ci-cd.yaml.tmpl
-  - path: xebialabs/xlr-pipeline-destroy.yaml.tmpl
-  - path: xebialabs/USAGE.md.tmpl
+  - path: xebialabs/aws-basic-eks-cluster.yaml
+  - path: xebialabs/aws-basic-eks-cluster-applications.yaml.tmpl
+  - path: xebialabs/aws-basic-eks-cluster-environments.yaml.tmpl
+  - path: xebialabs/USAGE-aws-basic-eks-cluster.md.tmpl
+
   - path: kubernetes/aws-auth-cm.yaml
   - path: cloudformation/cfn-secret-provider.zip
   - path: cloudformation/eks-master.yaml
   - path: cloudformation/eks-user.yaml
   - path: cloudformation/eks-vpc.yaml
   - path: cloudformation/eks-workers.yaml.tmpl
-  - path: cloudformation/efs.yaml
-    writeIf: ProvisionEFS
-  - path: xebialabs.yaml

aws/basic-eks-cluster/cloudformation/eks-master.yaml

@@ -87,4 +87,4 @@ Outputs:
     Description: ClusterControlPlaneSecurityGroup
     Export:
       Name:
-        Fn::Sub: "${AWS::StackName}-ClusterControlPlaneSecurityGroup"
+        Fn::Sub: "${AWS::StackName}-ClusterControlPlaneSecurityGroup"

aws/basic-eks-cluster/cloudformation/eks-vpc.yaml

@@ -143,4 +143,4 @@ Outputs:
     Description: VPC CIDR Block
     Export:
       Name:
-        Fn::Sub: "${AWS::StackName}-VPCCIDR" 
+        Fn::Sub: "${AWS::StackName}-VPCCIDR"

aws/basic-eks-cluster/xebialabs/USAGE-aws-basic-eks-cluster.md.tmpl

@@ -0,0 +1,57 @@
+{{$clusterName := .ClusterName | kebabcase}}
+
+## Notes
+* This example does not create a namespace, but uses the `default` namespace
+
+## Provision simple EKS cluster on AWS
+
+To deploy this blueprint, follow the steps below:
+
+> **Note:** Deployment of basic AWS cluster from XL Deploy requires a few more steps than the Azure or GCP ones.
+
+1. Apply the generated YAML configuration using the XL CLI. Open up a terminal in the folder where you generated the configuration. Then run:
+
+    ```plain
+    xl apply -f xebialabs/aws-basic-eks-cluster.yaml
+    ```
+
+2. Go to XL Deploy and look for the `{{$clusterName}}` folder under `Applications`.
+
+3. To provision the lambda artifacts:
+  3.1. Select `{{$clusterName}}/EKS-LAMBDA/{{$clusterName}}-lambda-bucket`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+  3.2. Select `{{$clusterName}}/EKS-LAMBDA/{{$clusterName}}-lambda-artifacts`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and Click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+
+4. To provision the VPC and IAM resources:
+  4.1. Select `EKS-CLOUDFORMATION/{{$clusterName}}-cloudformation-eks-user`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+  4.2. Select `EKS-CLOUDFORMATION/{{$clusterName}}-cloudformation-eks-vpc`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+
+5. To provision the EKS cluster:
+  5.1 Select `EKS-CLOUDFORMATION/{{$clusterName}}-cloudformation-eks-master`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+  5.2 Select `EKS-CLOUDFORMATION/{{$clusterName}}-cloudformation-eks-workers`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+  5.3 Select `EKS-CLOUDFORMATION/{{$clusterName}}-k8s-config-map`, click on the ellipsis and select 'Deploy'. Select the `Environments/{{$clusterName}}/aws-eks-{{$clusterName}}-kube-system` Environment and click 'Continue'. Verify that there are no error messages, then click 'Deploy'.
+
+6. After the deployment process is complete, an EKS cluster will have been provisioned on AWS and a configured `K8s.master` environment will have been registered in XL Deploy.
+
+7. In order to deploy to the k8s cluster, you can execute any k8s blueprints to generate configurations, apply those and deploy the application to your new cluster using XL Deploy.
+
+8. To deprovision the EKS cluster, you will have to perform the above steps in reverse.
+
+9. To deprovision the EKS cluster:
+  9.1. Select `Environments/{{$clusterName}}/aws-eks-{{$clusterName}}-kube-system/{{$clusterName}}-k8s-configmap` and click 'Undeploy'. Follow the instructions on screen.
+  9.2. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-cloudformation-eks-workers` and click 'Undeploy'. Follow the instructions on screen.
+  9.3. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-cloudformation-eks-master` and click 'Undeploy'. Follow the instructions on screen.
+
+10. To remove the S3 storage:
+  10.1. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-lambda-artifacts` and click 'Undeploy'. Follow the instructions on screen.
+  10.2. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-lambda-bucket` and click 'Undeploy'. Follow the instructions on screen.
+
+11. To deprovision the VPC and IAM resources:
+  11.1. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-cloudformation-eks-user`and click 'Undeploy'. Follow the instructions on screen.
+  11.2. Select `Environments/{{$clusterName}}/aws-cloudformation-{{$clusterName}}/{{$clusterName}}-cloudformation-eks-vpc`and click 'Undeploy'. Follow the instructions on screen.
+
+## Minimum required versions
+
+This blueprint version requires at least the following versions of the specified tools to work properly:
+
+- XL Deploy: Version 8.6.1
+- XL CLI: Version 8.6.1