-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Features: Core IAM module, X-ray submodule (#22)
### Overview This (massive) PR is intended to be the initial release of: - The core IAM module to support a `serverless` framework application - An optional X-Ray submodule to add IAM permissions needed for `serverless` X-ray support. The companion reference application PR to review is: FormidableLabs/aws-lambda-serverless-reference#7 Things I'd suggest to review (if all the files are too beastly): - Core README: https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/README.md - X-ray README: https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/modules/xray/README.md - The variables (and all the ARNs to limit gathered in one place): https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/variables.tf ### Sample Integration Basic, simplest integration is like this: ```hcl # Base `serverless` IAM support. module "serverless" { source = "FormidableLabs/serverless/aws" region = "${var.region}" service_name = "${var.service_name}" stage = "${var.stage}" } # OPTION(Xray): Add X-ray support to lambda execution roles. # **NOTE**: No explicit dependency on core, but it **is** needed. module "serverless_xray" { source = "FormidableLabs/serverless/aws//modules/xray" region = "${var.region}" service_name = "${var.service_name}" stage = "${var.stage}" } ``` ### Work Completed: - Add CONTRIBUTING.md document. Fixes #20 - Per-module/submodule READMEs. Fixes #17 - Root README.md. Fixes #16 - Rename project. Fixes #14 - CI: Set up. Fixes #7 - Feature: Stack / resource naming. Fixes #4 - Feature: Roles / Groups for admin, developer, CI enhancement. Fixes #2 - Feature/Option: Xray enhancement. Fixes #9 In progress: - Review/remove all source TODOs. #15 - Publish: Terraform Registry. #5
- Loading branch information
1 parent
601053e
commit a25e951
Showing
18 changed files
with
1,429 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
language: node_js | ||
|
||
node_js: | ||
- "10" | ||
|
||
branches: | ||
only: | ||
- master | ||
|
||
env: | ||
global: | ||
- TF_VERSION=0.11.10 | ||
|
||
before_install: | ||
- wget https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip -O /tmp/terraform.zip | ||
- sudo unzip -d /usr/local/bin/ /tmp/terraform.zip | ||
|
||
script: | ||
- terraform --version | ||
- yarn check:ci |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Changes | ||
======= | ||
|
||
## UNRELEASED | ||
|
||
* Module: Core IAM support for `serverless` framework. | ||
* Submodule: AWS X-ray support for `serverless` apps. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
Contributing | ||
============ | ||
|
||
Thanks for contributing! | ||
|
||
## Development | ||
|
||
We develop this project using `terraform` and `yarn` / Node.js for convenience. Make sure you have both installed. | ||
|
||
Because of how Terraform works, we have to format / generate code that goes back into git source. So, make sure to periodically run: | ||
|
||
```sh | ||
$ yarn build | ||
``` | ||
|
||
to update the built files. | ||
|
||
### `variables.tf` | ||
|
||
The root project `variables.tf` are copied programmatically to all submodules (`modules/*/variables.tf`) and will overwrite any modifications. If you need to change / add to this file, be careful, and do it in the root `variables.tf`. | ||
|
||
## Testing | ||
|
||
We test out this project using a simple reference app that consumes it: [aws-lambda-serverless-reference](https://github.com/FormidableLabs/aws-lambda-serverless-reference). When making changes, make sure to check out that project, make changes to the `source` of the `serverless*` modules like: | ||
|
||
```diff | ||
--- a/terraform/main.tf | ||
+++ b/terraform/main.tf | ||
@@ -12,7 +12,7 @@ terraform { | ||
# Base `serverless` IAM support. | ||
module "serverless" { | ||
- source = "FormidableLabs/serverless/aws" # NORMAL from registry | ||
+ source = "../../terraform-aws-serverless" # CHANGE to relative path | ||
``` | ||
|
||
in every place that uses the `FormidableLabs/serverless/aws` module. | ||
|
||
## Before submitting a PR... | ||
|
||
Before you go ahead and submit a PR, make sure that you have done the following: | ||
|
||
```sh | ||
$ yarn run build | ||
``` | ||
|
||
## Releasing a new version to Terraform Registry | ||
|
||
_Only for project administrators_. | ||
|
||
1. Update `CHANGELOG.md`, following format for previous versions | ||
2. Commit as "Changes for version NUMBER" | ||
3. Run `npm version patch` (or `minor|major|VERSION`) to run tests and lint, | ||
build published directories, then update `package.json` + add a git tag. | ||
4. Run `git push && git push --tags` | ||
|
||
TODO(REGISTRY): Add registry publishing step!!! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
The MIT License (MIT) | ||
|
||
Copyright (c) 2019 Formidable Labs | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,24 +1,158 @@ | ||
Serverless IAM via Terraform | ||
============================ | ||
AWS Serverless Module | ||
===================== | ||
[![Terraform][tf_img]][tf_site] | ||
[![Travis Status][trav_img]][trav_site] | ||
|
||
Tuned, locked-down IAM roles for the minimum set of privileges to run the `serverless` framework. | ||
Get your [serverless][] framework application to AWS, the **right way**. | ||
|
||
## Contents | ||
|
||
## Installation | ||
<!-- START doctoc generated TOC please keep comment here to allow auto update --> | ||
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> | ||
|
||
First, [download Terraform](https://www.terraform.io/downloads.html) and [install the binary](https://www.terraform.io/intro/getting-started/install.html). You should be able to run: | ||
|
||
```sh | ||
$ terraform --version | ||
- [Overview](#overview) | ||
- [Concepts](#concepts) | ||
- [Modules](#modules) | ||
- [Integration](#integration) | ||
- [Reference project](#reference-project) | ||
- [Module integration](#module-integration) | ||
- [AWS IAM group integration](#aws-iam-group-integration) | ||
|
||
<!-- END doctoc generated TOC please keep comment here to allow auto update --> | ||
|
||
## Overview | ||
|
||
Getting a [serverless][] application all the way to production in AWS **correctly** and **securely** can be quite challenging. In particular, things like: | ||
|
||
- Locking down IAM permissions to the minimum needed for different conceptual "roles" (e.g., `admin`, `developer`, `ci`). | ||
- Providing a scheme for different environments/stages (e.g., `development`, `staging`, `production`). | ||
|
||
... lack reasonable guidance to practically achieve in real world applications. | ||
|
||
This [Terraform][] module provides a production-ready base of AWS permissions / resources to support a `serverless` framework application and help manage development / deployment workflows and maintenance. Specifically, it provides: | ||
|
||
- **IAM Groups**: Role-specific groups to attach to AWS users to give humans and CI the minimum level of permissions locked to both a specific `serverless` service and stage/environment. | ||
|
||
## Concepts | ||
|
||
This module allows practical isolation / compartmentalization of privileges within a single AWS account along the folowing axes: | ||
|
||
* **Stage/Environment**: An arbitrary environment to isolate -- this module doesn't restrict selection in any way other than there has to be at least one. In practice, a good set of choices may be something like `sandbox`, `development`, `staging`, `production`. | ||
* **IAM Groups**: This module creates/enforces a scheme wherein: | ||
* **Admin**: AWS users assigned to the `admin` group can create/update/delete a `serverless` application and do pretty much anything that the `serverless` framework permits out of the box. | ||
* **Developer, CI**: AWS users assigned to the `developer|ci` groups can update a `serverless` application and do other things like view logs, perform rollbacks, etc. | ||
|
||
In this manner, once an AWS superuser deploys a Terraform stack with this module and assigns IAM groups, the rest of the development / devops teams and CI can build and deploy Serverless applications to appropriate cloud targets with the minimum necessary privileges and isolation across services + environments + IAM roles. | ||
|
||
## Modules | ||
|
||
This project provides a core base module that is the minimum that must be used. Once the core is in place, then other optional submodules can be added. | ||
|
||
- **Core (`/*`)**: Provides supporting IAM policies, roles, and groups so that an engineering team / CI can effectively create and maintain `serverless` Framework applications locked down to specific applications + environments with the minimum permissions needed. | ||
- **X-Ray (`modules/xray`)**: Optional submodule to add needed IAM support to enable AWS X-Ray performance tracing in a Serverless framework application. See the [submodule documentation](./modules/xray/README.md). | ||
|
||
## Integration | ||
|
||
### Reference project | ||
|
||
Perhaps the easiest place to start is our [sample reference project][ref_project] that creates a Serverless framework service named `simple-reference` that integrates the core module and submodules of this project. The relevant files to review include: | ||
|
||
- Terraform infrastructure | ||
- [aws/bootstrap.yml](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/aws/bootstrap.yml): Terraform remote state storage / bootstrap. | ||
- [terraform/variables.tf](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/terraform/variables.tf): Terraform variables. | ||
- [terraform/main.tf](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/terraform/main.tf): Terraform resources / integration. | ||
- Serverless framework | ||
- [serverless.yml](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/serverless.yml): Serverless framework configuration. | ||
- Example Node.js handlers/servers | ||
- [src/server/base.js](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/src/server/base.js): Example "hello world" server using only the core `serverless` module. | ||
|
||
### Module integration | ||
|
||
Here's a basic integration of the core `serverless` module: | ||
|
||
```hcl | ||
# variables.tf | ||
variable "stage" { | ||
description = "The stage/environment to deploy to. Suggest: `sandbox`, `development`, `staging`, `production`." | ||
default = "development" | ||
} | ||
# main.tf | ||
provider "aws" { | ||
region = "us-east-1" | ||
version = "~> 1.19" | ||
} | ||
# Core `serverless` IAM support. | ||
module "serverless" { | ||
source = "FormidableLabs/serverless/aws" | ||
region = "us-east-1" | ||
service_name = "sparklepants" | ||
stage = "${var.stage}" | ||
# (Default values) | ||
# iam_region = `*` | ||
# iam_partition = `*` | ||
# iam_account_id = `AWS_CALLER account` | ||
# tf_service_name = `tf-SERVICE_NAME` | ||
# sls_service_name = `sls-SERVICE_NAME` | ||
} | ||
``` | ||
|
||
- [ ] TODO: Add note about minimum TF version. | ||
That pairs with a `serverless.yml` configuration: | ||
|
||
```yml | ||
# This value needs to either be `sls-` + `service_name` module input *or* | ||
# be specified directly as the module input `sls_service_name`. | ||
service: sls-sparklepants | ||
|
||
provider: | ||
name: aws | ||
runtime: nodejs8.10 | ||
region: "us-east-1" | ||
stage: ${opt:stage, "development"} | ||
|
||
functions: | ||
server: | ||
# ... | ||
``` | ||
|
||
Let's unpack the parameters a bit more (located in [variables.tf](variables.tf)): | ||
|
||
- `service_name`: A service name is something that defines the unique application that will match up with the serverless application. E.g., something boring like `simple-reference` or `graphql-server` or exciting like `unicorn` or `sparklepants`. | ||
- `stage`: The current stage that will match up with the `serverless` framework deployment. These are arbitrary, but can be something like `development`/`staging`/`production`. | ||
- `region`: The deployed region of the service. Defaults to the current caller's AWS region. E.g., `us-east-1`. | ||
- `iam_region`: The [AWS region][] to limit IAM privileges to. Defaults to `*`. The difference with `region` is that `region` has to be one specific region like `us-east-1` to match up with Serverless framework resources, whereas `iam_region` can be a single region or `*` wildcard as it's just an IAM restriction. | ||
- `iam_partition`: The [AWS partition][] to limit IAM privileges to. Defaults to `*`. | ||
- `iam_account_id`: The [AWS account ID][] to limit IAM privileges to. Defaults to the current caller's account ID. | ||
- `tf_service_name`: The service name for Terraform-created resources. It is very useful to distinguish between those created by Terraform / this module and those created by the Serverless framework. By default, `tf-${service_name}` for "Terraform". E.g., `tf-simple-reference` or `tf-sparklepants`. | ||
- `sls_service_name`: The service name for Serverless as defined in `serverless.yml` in the `service` field. Highly recommended to match our default of `sls-${service_name}` for "Serverless". | ||
|
||
Most likely, an AWS superuser will be needed to run the Terraform application for these IAM / other resources. | ||
|
||
### AWS IAM group integration | ||
|
||
Once the core module is applied, three IAM groups will be created in the form of `${tf_service_name}-${stage}-(admin|developer|ci)`. This typically looks something like: | ||
|
||
- `tf-${service_name}-${stage}-admin`: Can create/delete/update the Severless app. | ||
- `tf-${service_name}-${stage}-developer`: Can deploy the Severless app. | ||
- `tf-${service_name}-${stage}-ci`: Can deploy the Severless app. | ||
|
||
## Usage | ||
Once these groups exist, an AWS superuser can then attach these groups to AWS individual users as appropriate for the combination of service + stage + role (admin, developer, CI). Or, the IAM group attachments could be controlled via Terraform as well! | ||
|
||
- [ ] TODO: Usage | ||
- [ ] TODO: AWS_PROFILE or AWS_*_KEYS docs | ||
The main upshot of this is after attachment, a given AWS user has the minimum necessary privileges for exactly the level of Serverless framework commands they need. Our example Serverless application [reference project][ref_project] documentation has many examples of various `serverless` commands and which IAM group can properly run them. | ||
|
||
## IAM roles and groups | ||
[serverless]: https://serverless.com/ | ||
[Terraform]: https://www.terraform.io | ||
[AWS region]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-region | ||
[AWS partition]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-partition | ||
[AWS account ID]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-accountid | ||
[AWS X-Ray]: https://aws.amazon.com/xray/ | ||
|
||
- [ ] TODO: discuss admin, developer, ci | ||
[tf_img]: https://img.shields.io/badge/terraform-published-blue.svg | ||
[tf_site]: https://registry.terraform.io/modules/FormidableLabs/serverless/aws | ||
[trav_img]: https://api.travis-ci.org/FormidableLabs/inspectpack.svg | ||
[trav_site]: https://travis-ci.org/FormidableLabs/inspectpack | ||
[ref_project]: https://github.com/FormidableLabs/aws-lambda-serverless-reference |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
############################################################################### | ||
# IAM Groups | ||
# | ||
# - `admin`: An administrator that can create, delete, develop the services. | ||
# - `developer`: A developer that deploy/update an existing service. | ||
# - `ci`: The CI service can deploy/update an existing service. | ||
# | ||
# General reference | ||
# - https://iam.cloudonaut.io/ | ||
# - https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html | ||
# - http://awspolicygen.s3.amazonaws.com/policygen.html | ||
############################################################################### | ||
|
||
# admin | ||
resource "aws_iam_group" "admin" { | ||
name = "${local.tf_service_name}-${local.stage}-admin" | ||
} | ||
|
||
resource "aws_iam_group_policy_attachment" "admin_admin" { | ||
group = "${aws_iam_group.admin.name}" | ||
policy_arn = "${aws_iam_policy.admin.arn}" | ||
} | ||
|
||
resource "aws_iam_group_policy_attachment" "admin_developer" { | ||
group = "${aws_iam_group.admin.name}" | ||
policy_arn = "${aws_iam_policy.developer.arn}" | ||
} | ||
|
||
# ci | ||
resource "aws_iam_group" "ci" { | ||
name = "${local.tf_service_name}-${local.stage}-ci" | ||
} | ||
|
||
resource "aws_iam_group_policy_attachment" "ci_developer" { | ||
group = "${aws_iam_group.ci.name}" | ||
policy_arn = "${aws_iam_policy.developer.arn}" | ||
} | ||
|
||
# developer | ||
resource "aws_iam_group" "developer" { | ||
name = "${local.tf_service_name}-${local.stage}-developer" | ||
} | ||
|
||
resource "aws_iam_group_policy_attachment" "developer_developer" { | ||
group = "${aws_iam_group.developer.name}" | ||
policy_arn = "${aws_iam_policy.developer.arn}" | ||
} |
Oops, something went wrong.