diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..aa63dde --- /dev/null +++ b/.gitignore @@ -0,0 +1,39 @@ +# Terraform files +.terraform +terraform.tfstate +terraform.tfvars +*.tfstate* + +# OS X files +.history +.DS_Store + +# lambda zip files +lambda.zip + +# IntelliJ files +.idea_modules +*.iml +*.iws +*.ipr +.idea/ +build/ +*/build/ +out/ + +# Go best practices dictate that libraries should not include the vendor directory +vendor + +# Python stuff +dist +aws_auth_configmap_generator.* +.python-version +.tox +__pycache__ +*.pyc + +# Folder used to store temporary test data by Terratest +.test-data + +# Generic temporary files +/tmp diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..b128490 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,99 @@ +# Contribution Guidelines + +Contributions to this Package are very welcome! We follow a fairly standard [pull request process]( +https://help.github.com/articles/about-pull-requests/) for contributions, subject to the following guidelines: + +1. [File a GitHub issue](#file-a-github-issue) +1. [Update the documentation](#update-the-documentation) +1. [Update the tests](#update-the-tests) +1. [Update the code](#update-the-code) +1. [Create a pull request](#create-a-pull-request) +1. [Merge and release](#merge-and-release) + +## File a GitHub issue + +Before starting any work, we recommend filing a GitHub issue in this repo. This is your chance to ask questions and +get feedback from the maintainers and the community before you sink a lot of time into writing (possibly the wrong) +code. If there is anything you're unsure about, just ask! + +## Update the documentation + +We recommend updating the documentation *before* updating any code (see [Readme Driven +Development](http://tom.preston-werner.com/2010/08/23/readme-driven-development.html)). This ensures the documentation +stays up to date and allows you to think through the problem at a high level before you get lost in the weeds of +coding. + +## Update the tests + +We also recommend updating the automated tests *before* updating any code (see [Test Driven +Development](https://en.wikipedia.org/wiki/Test-driven_development)). That means you add or update a test case, +verify that it's failing with a clear error message, and *then* make the code changes to get that test to pass. This +ensures the tests stay up to date and verify all the functionality in this Module, including whatever new +functionality you're adding in your contribution. Check out the [tests](https://github.com/gruntwork-io/terraform-kubernetes-helm/tree/master/test) folder for instructions on running the +automated tests. + +## Update the code + +At this point, make your code changes and use your new test case to verify that everything is working. As you work, +keep in mind two things: + +1. Backwards compatibility +1. Downtime + +### Backwards compatibility + +Please make every effort to avoid unnecessary backwards incompatible changes. With Terraform code, this means: + +1. Do not delete, rename, or change the type of input variables. +1. If you add an input variable, it should have a `default`. +1. Do not delete, rename, or change the type of output variables. +1. Do not delete or rename a module in the `modules` folder. + +If a backwards incompatible change cannot be avoided, please make sure to call that out when you submit a pull request, +explaining why the change is absolutely necessary. + +### Downtime + +Bear in mind that the Terraform code in this Module is used by real companies to run real infrastructure in +production, and certain types of changes could cause downtime. For example, consider the following: + +1. If you rename a resource (e.g. `aws_instance "foo"` -> `aws_instance "bar"`), Terraform will see that as deleting + the old resource and creating a new one. +1. If you change certain attributes of a resource (e.g. the `name` of an `aws_elb`), the cloud provider (e.g. AWS) may + treat that as an instruction to delete the old resource and a create a new one. + +Deleting certain types of resources (e.g. virtual servers, load balancers) can cause downtime, so when making code +changes, think carefully about how to avoid that. For example, can you avoid downtime by using +[create_before_destroy](https://www.terraform.io/docs/configuration/resources.html#create_before_destroy)? Or via +the `terraform state` command? If so, make sure to note this in our pull request. If downtime cannot be avoided, +please make sure to call that out when you submit a pull request. + + +### Formatting and pre-commit hooks + +You must run `terraform fmt` on the code before committing. You can configure your computer to do this automatically +using pre-commit hooks managed using [pre-commit](http://pre-commit.com/): + +1. [Install pre-commit](http://pre-commit.com/#install). E.g.: `brew install pre-commit`. +1. Install the hooks: `pre-commit install`. + +That's it! Now just write your code, and every time you commit, `terraform fmt` will be run on the files you're +committing. + + +## Create a pull request + +[Create a pull request](https://help.github.com/articles/creating-a-pull-request/) with your changes. Please make sure +to include the following: + +1. A description of the change, including a link to your GitHub issue. +1. The output of your automated test run, preferably in a [GitHub Gist](https://gist.github.com/). We cannot run + automated tests for pull requests automatically due to [security + concerns](https://circleci.com/docs/fork-pr-builds/#security-implications), so we need you to manually provide this + test output so we can verify that everything is working. +1. Any notes on backwards incompatibility or downtime. + +## Merge and release + +The maintainers for this repo will review your code and provide feedback. If everything looks good, they will merge the +code and release a new version, which you'll be able to find in the [releases page](../../releases). diff --git a/GRUNTWORK_PHILOSOPHY.md b/GRUNTWORK_PHILOSOPHY.md new file mode 100644 index 0000000..02d9873 --- /dev/null +++ b/GRUNTWORK_PHILOSOPHY.md @@ -0,0 +1,64 @@ +# Gruntwork Philosophy + +At Gruntwork, we strive to accelerate the deployment of production grade infrastructure by prodiving a library of +stable, reusable, and battle tested infrastructure as code organized into a series of [modules](#what-is-a-module) with +[submodules](#what-is-a-submodule). Each module represents a particular set of infrastructure that is componentized into +smaller pieces represented by the submodules within the module. By doing so, we have built a composable library that can +be combined into building out everything from simple single service deployments to complicated microservice setups so +that your infrastructure can grow with your business needs. Every module we provide is built with the [production grade +infrastruture checklist](#production-grade-infrastructure-checklist) in mind, ensuring that the services you deploy are +resilient, fault tolerant, and scalable. + + +## What is a Module? + +A Module is a reusable, tested, documented, configurable, best-practices definition of a single piece of Infrastructure +(e.g., Docker cluster, VPC, Jenkins, Consul), written using a combination of [Terraform](https://www.terraform.io/), Go, +and Bash. A module contains a set of automated tests, documentation, and examples that have been proven in production, +providing the underlying infrastructure for [Gruntwork's customers](https://www.gruntwork.io/customers). + +Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse existing code +that has been proven in production. And instead of maintaining all that infrastructure code yourself, you can leverage +the work of the community to pick up infrastructure improvements through a version number bump. + + +## What is a Submodule? + +Each Infrastructure Module consists of one or more orthogonal Submodules that handle some specific aspect of that +Infrastructure Module's functionality. Breaking the code up into multiple submodules makes it easier to reuse and +compose to handle many different use cases. Although Modules are designed to provide an end to end solution to manage +the relevant infrastructure by combining the Submodules defined in the Module, Submodules can be used independently for +specific functionality that you need in your infrastructure code. + + +## Production Grade Infrastructure Checklist + +At Gruntwork, we have learned over the years that it is not enough to just get the services up and running in a publicly +accessible space to call your application "production-ready." There are many more things to consider, and oftentimes +many of these considerations are missing in the deployment plan of applications. These topics come up as afterthoughts, +and are learned the hard way after the fact. That is why we codified all of them into a checklist that can be used as a +reference to help ensure that they are considered before your application goes to production, and conscious decisions +are made to neglect particular components if needed, as opposed to accidentally omitting them from consideration. + + + +| Task | Description | Example tools | +|--------------------|-------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------| +| Install | Install the software binaries and all dependencies. | Bash, Chef, Ansible, Puppet | +| Configure | Configure the software at runtime. Includes port settings, TLS certs, service discovery, leaders, followers, replication, etc. | Bash, Chef, Ansible, Puppet | +| Provision | Provision the infrastructure. Includes EC2 instances, load balancers, network topology, security gr oups, IAM permissions, etc. | Terraform, CloudFormation | +| Deploy | Deploy the service on top of the infrastructure. Roll out updates with no downtime. Includes blue-green, rolling, and canary deployments. | Scripts, Orchestration tools (ECS, k8s, Nomad) | +| High availability | Withstand outages of individual processes, EC2 instances, services, Availability Zones, and regions. | Multi AZ, multi-region, replication, ASGs, ELBs | +| Scalability | Scale up and down in response to load. Scale horizontally (more servers) and/or vertically (bigger servers). | ASGs, replication, sharding, caching, divide and conquer | +| Performance | Optimize CPU, memory, disk, network, GPU, and usage. Includes query tuning, benchmarking, load testing, and profiling. | Dynatrace, valgrind, VisualVM, ab, Jmeter | +| Networking | Configure static and dynamic IPs, ports, service discovery, firewalls, DNS, SSH access, and VPN access. | EIPs, ENIs, VPCs, NACLs, SGs, Route 53, OpenVPN | +| Security | Encryption in transit (TLS) and on disk, authentication, authorization, secrets management, server hardening. | ACM, EBS Volumes, Cognito, Vault, CIS | +| Metrics | Availability metrics, business metrics, app metrics, server metrics, events, observability, tracing, and alerting. | CloudWatch, DataDog, New Relic, Honeycomb | +| Logs | Rotate logs on disk. Aggregate log data to a central location. | CloudWatch logs, ELK, Sumo Logic, Papertrail | +| Backup and Restore | Make backups of DBs, caches, and other data on a scheduled basis. Replicate to separate region/account. | RDS, ElastiCache, ec2-snapper, Lambda | +| Cost optimization | Pick proper instance types, use spot and reserved instances, use auto scaling, and nuke unused resources. | ASGs, spot instances, reserved instances | +| Documentation | Document your code, architecture, and practices. Create playbooks to respond to incidents. | READMEs, wikis, Slack | +| Tests | Write automated tests for your infrastructure code. Run tests after every commit and nightly. | Terratest | diff --git a/README.md b/README.md new file mode 100644 index 0000000..90c5780 --- /dev/null +++ b/README.md @@ -0,0 +1,100 @@ +[![Maintained by Gruntwork.io](https://img.shields.io/badge/maintained%20by-gruntwork.io-%235849a6.svg)](https://gruntwork.io/?ref=repo_aws_eks) + +# Helm Server Modules + +This repo contains a Module for deploying Helm Server on Kubernetes clusters with [Terraform](https://www.terraform.io). +This repo is a part of [the Gruntwork Infrastructure as Code +Library](https://gruntwork.io/infrastructure-as-code-library/), a collection of reusable, battle-tested, production +ready infrastructure code. Read the [Gruntwork Philosophy](GRUNTWORK_PHILOSOPHY.md) document to learn more about how +Gruntwork builds production grade infrastructure code. + + +## What is in this repo + +This repo provides a Gruntwork IaC Package and has the following folder structure: + +* [modules](/modules): This folder contains the main implementation code for this Module, broken down into multiple + standalone Submodules. +* [examples](/examples): This folder contains examples of how to use the Submodules. The [example root + README](/examples/README.md) provides a quickstart guide on how to use the Submodules in this Module. +* [test](/test): Automated tests for the Modules and examples. + +The following submodules are available in this module: + +- [k8s-namespace](/modules/k8s-namespace): Provision a Kubernetes `Namespace` with a default set of RBAC roles. +- [k8s-service-account](/modules/k8s-service-account): Provision a Kubernetes `ServiceAccount`. +- [k8s-helm-server](/modules/k8s-helm-server): Provision a namespaced secure Helm Server on an existing Kubernetes + cluster. + +## What is Kubernetes? + +[Kubernetes](https://kubernetes.io) is an open source container management system for deploying, scaling, and managing +containerized applications. Kubernetes is built by Google based on their internal proprietary container management +systems (Borg and Omega). Kubernetes provides a cloud agnostic platform to deploy your containerized applications with +built in support for common operational tasks such as replication, autoscaling, self-healing, and rolling deployments. + +You can learn more about Kubernetes from [the official documentation](https://kubernetes.io/docs/tutorials/kubernetes-basics/). + + +## What is Helm? + +[Helm](https://helm.sh/) is a package and module manager for Kubernetes that allows you to define, install, and manage +Kubernetes applications as reusable packages called Charts. Helm provides support for official charts in their +repository that contains various applications such as Jenkins, MySQL, and Consul to name a few. Gruntwork uses Helm +under the hood for the Kubernetes modules in this package. + +Helm consists of two components: the Helm Client, and the Helm Server (Tiller) + +### What is the Helm Client? + +The Helm client is a command line utility that provides a way to interact with Tiller. It is the primary interface to +installing and managing Charts as releases in the Helm ecosystem. In addition to providing operational interfaces (e.g +install, upgrade, list, etc), the client also provides utilities to support local development of Charts in the form of a +scaffolding command and repository management (e.g uploading a Chart). + +### What is the Helm Server? + +The Helm Server (Tiller) is a component of Helm that runs inside the Kubernetes cluster. Tiller is what +provides the functionality to apply the Kubernetes resource descriptions to the Kubernetes cluster. When you install a +release, the helm client essentially packages up the values and charts as a release, which is submitted to Tiller. +Tiller will then generate Kubernetes YAML files from the packaged release, and then apply the generated Kubernetes YAML +file from the charts on the cluster. + + + + +## Who maintains this Module? + +This Module and its Submodules are maintained by [Gruntwork](http://www.gruntwork.io/). If you are looking for help or +commercial support, send an email to +[support@gruntwork.io](mailto:support@gruntwork.io?Subject=GKE%20Module). + +Gruntwork can help with: + +* Setup, customization, and support for this Module. +* Modules and submodules for other types of infrastructure, such as VPCs, Docker clusters, databases, and continuous + integration. +* Modules and Submodules that meet compliance requirements, such as HIPAA. +* Consulting & Training on AWS, Terraform, and DevOps. + + +## How do I contribute to this Module? + +Contributions are very welcome! Check out the [Contribution Guidelines](/CONTRIBUTING.md) for instructions. + + +## How is this Module versioned? + +This Module follows the principles of [Semantic Versioning](http://semver.org/). You can find each new release, along +with the changelog, in the [Releases Page](../../releases). + +During initial development, the major version will be 0 (e.g., `0.x.y`), which indicates the code does not yet have a +stable API. Once we hit `1.0.0`, we will make every effort to maintain a backwards compatible API and use the MAJOR, +MINOR, and PATCH versions on each release to indicate any incompatibilities. + + +## License + +Please see [LICENSE](/LICENSE) for how the code in this repo is licensed. + +Copyright © 2018 Gruntwork, Inc.