fix: adding examples and documentation for state management

- adding documentation around GOOGLE_APPLICATION_CREDENTIALS

fixes #39
fixes #62

README.md

@@ -3,49 +3,56 @@
 
 ![Terraform Version](https://img.shields.io/badge/tf-%3E%3D0.12.0-blue.svg)
 
-This repo contains a [Terraform](https://www.terraform.io/) Module for provisioning a Kubernetes cluster for [Jenkins X](https://jenkins-x.io/) on [Google Cloud](https://cloud.google.com/).
+This repo contains a [Terraform](https://www.terraform.io/) module for provisioning a Kubernetes cluster for [Jenkins X](https://jenkins-x.io/) on [Google Cloud](https://cloud.google.com/).
 
-<!-- TOC depthfrom:2 -->
+<!-- TOC depthfrom:2 anchormode:github.com -->
 
-- [What is a Terraform Module](#what-is-a-terraform-module)
-- [How do you use this Module](#how-do-you-use-this-module)
+- [What is a Terraform module](#what-is-a-terraform-module)
+- [How do you use this module](#how-do-you-use-this-module)
     - [Prerequisites](#prerequisites)
     - [Cluster provisioning](#cluster-provisioning)
         - [Inputs](#inputs)
         - [Outputs](#outputs)
     - [Running `jx boot`](#running-jx-boot)
     - [Using a custom domain](#using-a-custom-domain)
+    - [Production cluster considerations](#production-cluster-considerations)
+    - [Configuring a Terraform backend](#configuring-a-terraform-backend)
+- [FAQ](#faq)
+    - [How do I get the latest version of the terraform-google-jx module](#how-do-i-get-the-latest-version-of-the-terraform-google-jx-module)
+    - [Why do I need Application Default Credentials](#why-do-i-need-application-default-credentials)
 - [Development](#development)
     - [Releasing](#releasing)
 - [How do I contribute](#how-do-i-contribute)
 
 <!-- /TOC -->
 
-## What is a Terraform Module
-<a id="markdown-What%20is%20a%20Terraform%20Module" name="What%20is%20a%20Terraform%20Module"></a>
+## What is a Terraform module
+<a id="markdown-What%20is%20a%20Terraform%20module" name="What%20is%20a%20Terraform%20module"></a>
 
-A Terraform Module refers to a self-contained package of Terraform configurations that are managed as a group.
-For more information around Modules refer to the Terraform [documentation](https://www.terraform.io/docs/modules/index.html).
+A Terraform "module" refers to a self-contained package of Terraform configurations that are managed as a group.
+For more information around modules refer to the Terraform [documentation](https://www.terraform.io/docs/modules/index.html).
 
-## How do you use this Module
-<a id="markdown-How%20do%20you%20use%20this%20Module" name="How%20do%20you%20use%20this%20Module"></a>
+## How do you use this module
+<a id="markdown-How%20do%20you%20use%20this%20module" name="How%20do%20you%20use%20this%20module"></a>
 
 ### Prerequisites
 <a id="markdown-Prerequisites" name="Prerequisites"></a>
 
-To make use of this Module, you need a Google Cloud project.
-Instructions on how to do this can be found [here](https://cloud.google.com/deployment-manager/docs/step-by-step-guide/installation-and-setup).
-You need your Google Cloud project id as an input variable for using this Module.
+To make use of this module, you need a Google Cloud project.
+Instructions on how to setup such a project can be found in the  [Google Cloud Installation and Setup](https://cloud.google.com/deployment-manager/docs/step-by-step-guide/installation-and-setup) guide.
+You need your Google Cloud project id as an input variable for using this module.
 
 You also need to install the Cloud SDK, in particular `gcloud`.
-You find instructions on how to install and authenticate in the documentation mentioned above.
+You find instructions on how to install and authenticate in the [Google Cloud Installation and Setup](https://cloud.google.com/deployment-manager/docs/step-by-step-guide/installation-and-setup) guide as well.
 
 Once you have `gcloud` installed, you need to create [Application Default Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login) by running:
 
 ```bash
 gcloud auth application-default login
 ```
 
+Alternatively, you can export the environment variable _GOOGLE_APPLICATION_CREDENTIALS_ referencing the path to a Google Cloud [service account key file](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
+
 Last but not least, ensure you have the following binaries installed:
 
 - `gcloud`
@@ -75,17 +82,22 @@ terraform apply
 ```
 
 This creates a cluster within the specified Google Cloud project with all possible configuration options defaulted.
-On completion of `terraform apply` there will also be a _jx-requirements.yml_ in the working directory which can be used as input to `jx boot`.
+
+:warning: **Note**: This example is for getting up and running quickly.
+It is not intended for a production cluster.
+Refer to [Production cluster considerations](#production-cluster-considerations) for things to consider when creating a production cluster.
+
+On completion of `terraform apply` there will be a _jx-requirements.yml_ in the working directory which can be used as input to `jx boot`.
 Refer to [Running `jx boot`](#running-jx-boot) for more information.
 
 No custom domain is used.
 Instead DNS resolution occurs via [nip.io](https://nip.io/).
 For more information on how to configure and use a custom domain, refer to [Using a custom domain](#using-a-custom-domain).
 
 If you just want to experiment with Jenkins X, you can set `force_destroy` to `true`.
-This allows you to remove all generated resources when running `terraform destroy`, including any generated buckets with their content.
+This allows you to remove all generated resources when running `terraform destroy`, including any generated buckets including their content.
 
-The following two paragraphs provide the full list of configuration and output variables of this Terraform Module.
+The following two paragraphs provide the full list of configuration and output variables of this Terraform module.
 
 #### Inputs
 <a id="markdown-Inputs" name="Inputs"></a>
@@ -131,25 +143,25 @@ The following two paragraphs provide the full list of configuration and output v
 ### Running `jx boot`
 <a id="markdown-Running%20%60jx%20boot%60" name="Running%20%60jx%20boot%60"></a>
 
-As an output of applying this Terraform Module a _jx-requirements.yml_ file is generated in the current directory.
-This file can be used as input to [Jenkins X Boot](https://jenkins-x.io/docs/getting-started/setup/boot/) which is responsible for installing all the required Jenkins X components into the cluster created by this Module.
+An output of applying this Terraform module is a _jx-requirements.yml_ file in the current directory.
+This file can be used as input to [Jenkins X Boot](https://jenkins-x.io/docs/getting-started/setup/boot/) which is responsible for installing all the required Jenkins X components into the cluster created by this module.
 
-Copy the generated _jx-requirements.yml_ into an empty directory, change into this directory and execute:
+Change into  an empty directory and execute:
 
 ```bash
-jx boot --requirements jx-requirements.yml
+jx boot --requirements <path-to-jx-requirements.yml>
 ```
 
-You have to provide some more required configuration via interactive prompts.
+You are prompted for any further required configuration.
 The number of prompts depends on how much you have [pre-configured](#inputs) via your Terraform variables.
 
 ### Using a custom domain
 <a id="markdown-Using%20a%20custom%20domain" name="Using%20a%20custom%20domain"></a>
 
 If you want to use a custom domain with your Jenkins X installation, you need to provide values for the [variables](#inputs) _parent_domain_ and _tls_email_.
-_parent_domain_ is the fully qualified domain name you want to use and _tls_email_ is the email you want to use for issuing Let's Encrypt TLS certificates.
+_parent_domain_ is the fully qualified domain name you want to use and _tls_email_ is the email address you want to use for issuing Let's Encrypt TLS certificates.
 
-Before you run the Terraform configuration, you also need to create a [Cloud DNS managed zone](https://cloud.google.com/dns/zones), with the DNS name in the managed zone matching your custom domain name, for example in the case of _example.jenkins-x.rocks_ as domain:
+Before you apply the Terraform configuration, you also need to create a [Cloud DNS managed zone](https://cloud.google.com/dns/zones), with the DNS name in the managed zone matching your custom domain name, for example in the case of _example.jenkins-x.rocks_ as domain:
 
 ![Creating a Managed Zone](./images/create_managed_zone.png)
 
@@ -164,6 +176,75 @@ When a custom domain is provided, Jenkins X uses [ExternalDNS](https://github.co
 
 If _parent_domain_ id not set, your cluster will use [nip.io](https://nip.io/) in order to create publicly resolvable URLs of the form ht<span>tp://\<app-name\>-\<environment-name\>.\<cluster-ip\>.nip.io.
 
+### Production cluster considerations
+<a id="markdown-Production%20cluster%20considerations" name="Production%20cluster%20considerations"></a>
+
+The configuration as seen in [Cluster provisioning](#cluster-provisioning) is not suited for creating and maintaining a production Jenkins X cluster.
+The following is a list of considerations for a production usecase.
+
+- Specify the version attribute of the module, for example:
+
+    ```terraform
+    module "jx" {
+      source  = "jenkins-x/jx/google"
+      version = "1.2.4"
+      # insert your configuration
+    }
+    ```
+
+  Specifying the version ensures that you are using a fixed version and that version upgrades cannot occur unintented.
+
+- Keep the Terraform configuration under version control,  by creating a dedicated repository for your cluster configuration or by adding it to an already existing infrastructure repository.
+
+- Setup a Terraform backend to securely store and share the state of your cluster. For more information refer to [Configuring a Terraform backend](##configuring-a-terraform-backend).
+
+### Configuring a Terraform backend
+<a id="markdown-Configuring%20a%20Terraform%20backend" name="Configuring%20a%20Terraform%20backend"></a>
+
+A "[backend](https://www.terraform.io/docs/backends/index.html)" in Terraform determines how state is loaded and how an operation such as _apply_ is executed.
+By default, Terraform uses the _local_ backend which keeps the state of the created resources on the local file system.
+This is problematic since sensitive information will be stored on disk and it is not possible to share state across a team.
+When working with Google Cloud a good choice for your Terraform backend is the [_gcs_ backend](https://www.terraform.io/docs/backends/types/gcs.html)  which stores the Terraform state in a Google Cloud Storage bucket.
+The [examples](./examples) directory of this repository contains configuration examples for using the gcs backed with and without optionally configured customer supplied encryption key.
+
+To use the gcs backend you will need to create the bucket upfront.
+You can use `gsutil` to create the bucket:
+
+```sh
+gsutil mb gs://<my-bucket-name>/
+```
+
+It is also recommended to enable versioning on the bucket as an additional safety net in case of state corruption.
+
+```sh
+gsutil versioning set on gs://<my-bucket-name>
+```
+
+You can verify whether a bucket has versioning enabled via:
+
+```sh
+gsutil versioning get gs://<my-bucket-name>
+```
+
+## FAQ
+<a id="markdown-FAQ" name="FAQ"></a>
+
+### How do I get the latest version of the terraform-google-jx module
+<a id="markdown-How%20do%20I%20get%20the%20latest%20version%20of%20the%20terraform-google-jx%20module" name="How%20do%20I%20get%20the%20latest%20version%20of%20the%20terraform-google-jx%20module"></a>
+
+```sh
+terraform init -upgrade
+```
+
+### Why do I need Application Default Credentials
+<a id="markdown-Why%20do%20I%20need%20Application%20Default%20Credentials" name="Why%20do%20I%20need%20Application%20Default%20Credentials"></a>
+
+The recommended way to authenticate to the Google Cloud API is by using a [service account](https://cloud.google.com/docs/authentication/getting-started).
+This allows for authentication regardless of where your code runs.
+This Terraform module expects authentication via a service account key.
+You can either specify the path to this key directly using the _GOOGLE_APPLICATION_CREDENTIALS_ environment variable or you can run `gcloud auth application-default login`.
+In the latter case `gcloud` obtains user access credentials via a web flow and puts them in the well-known location for Application Default Credentials (ADC), usually _~/.config/gcloud/application_default_credentials.json_.
+
 ## Development
 <a id="markdown-Development" name="Development"></a>
 

examples/backend-encrypted/main.tf

@@ -0,0 +1,15 @@
+terraform {
+  backend "gcs" {
+    bucket = "<my-terraform-state-bucket>"
+    # You can generate a key with 'openssl rand -base64 32'
+    # You can set the key explicitly or via exporting GOOGLE_ENCRYPTION_KEY
+    # Make sure to safely store the key, since your Terraform state cannot be recovered if the key is lost
+    encryption_key = "<my-encryptionkey>"
+  }
+}
+
+module "jx" {
+  source = "jenkins-x/jx/google"
+
+  gcp_project = "<my-gcp-project-id>"
+}

examples/backend/main.tf

@@ -0,0 +1,13 @@
+terraform {
+  backend "gcs" {
+    bucket = "<my-terraform-state-bucket>"
+    # arbitrary prefix/directory within the bucket
+    prefix = "jx"
+  }
+}
+
+module "jx" {
+  source = "jenkins-x/jx/google"
+
+  gcp_project = "<my-gcp-project-id>"
+}

examples/basic/main.tf

@@ -0,0 +1,5 @@
+module "jx" {
+  source  = "jenkins-x/jx/google"
+
+  gcp_project = "<my-gcp-project-id>"
+}