Skip to content

Commit

Permalink
Features: Core IAM module, X-ray submodule (#22)
Browse files Browse the repository at this point in the history
### Overview

This (massive) PR is intended to be the initial release of:

- The core IAM module to support a `serverless` framework application
- An optional X-Ray submodule to add IAM permissions needed for `serverless` X-ray support.

The companion reference application PR to review is: FormidableLabs/aws-lambda-serverless-reference#7

Things I'd suggest to review (if all the files are too beastly):

- Core README: https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/README.md
- X-ray README: https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/modules/xray/README.md
- The variables (and all the ARNs to limit gathered in one place): https://github.com/FormidableLabs/terraform-aws-serverless/blob/feature/xray/variables.tf

### Sample Integration

Basic, simplest integration is like this:

```hcl
# Base `serverless` IAM support.
module "serverless" {
  source = "FormidableLabs/serverless/aws"

  region       = "${var.region}"
  service_name = "${var.service_name}"
  stage        = "${var.stage}"
}

# OPTION(Xray): Add X-ray support to lambda execution roles.
# **NOTE**: No explicit dependency on core, but it **is** needed.
module "serverless_xray" {
  source = "FormidableLabs/serverless/aws//modules/xray"

  region       = "${var.region}"
  service_name = "${var.service_name}"
  stage        = "${var.stage}"
}
```

### Work

Completed:

- Add CONTRIBUTING.md document. Fixes #20
- Per-module/submodule READMEs. Fixes #17
- Root README.md. Fixes #16
- Rename project. Fixes #14
- CI: Set up. Fixes #7
- Feature: Stack / resource naming. Fixes #4
- Feature: Roles / Groups for admin, developer, CI enhancement. Fixes #2
- Feature/Option: Xray enhancement. Fixes #9

In progress:

- Review/remove all source TODOs. #15
- Publish: Terraform Registry. #5
  • Loading branch information
ryan-roemer committed Feb 22, 2019
1 parent 601053e commit a25e951
Show file tree
Hide file tree
Showing 18 changed files with 1,429 additions and 13 deletions.
20 changes: 20 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
language: node_js

node_js:
- "10"

branches:
only:
- master

env:
global:
- TF_VERSION=0.11.10

before_install:
- wget https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip -O /tmp/terraform.zip
- sudo unzip -d /usr/local/bin/ /tmp/terraform.zip

script:
- terraform --version
- yarn check:ci
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Changes
=======

## UNRELEASED

* Module: Core IAM support for `serverless` framework.
* Submodule: AWS X-ray support for `serverless` apps.
56 changes: 56 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
Contributing
============

Thanks for contributing!

## Development

We develop this project using `terraform` and `yarn` / Node.js for convenience. Make sure you have both installed.

Because of how Terraform works, we have to format / generate code that goes back into git source. So, make sure to periodically run:

```sh
$ yarn build
```

to update the built files.

### `variables.tf`

The root project `variables.tf` are copied programmatically to all submodules (`modules/*/variables.tf`) and will overwrite any modifications. If you need to change / add to this file, be careful, and do it in the root `variables.tf`.

## Testing

We test out this project using a simple reference app that consumes it: [aws-lambda-serverless-reference](https://github.com/FormidableLabs/aws-lambda-serverless-reference). When making changes, make sure to check out that project, make changes to the `source` of the `serverless*` modules like:

```diff
--- a/terraform/main.tf
+++ b/terraform/main.tf
@@ -12,7 +12,7 @@ terraform {
# Base `serverless` IAM support.
module "serverless" {
- source = "FormidableLabs/serverless/aws" # NORMAL from registry
+ source = "../../terraform-aws-serverless" # CHANGE to relative path
```

in every place that uses the `FormidableLabs/serverless/aws` module.

## Before submitting a PR...

Before you go ahead and submit a PR, make sure that you have done the following:

```sh
$ yarn run build
```

## Releasing a new version to Terraform Registry

_Only for project administrators_.

1. Update `CHANGELOG.md`, following format for previous versions
2. Commit as "Changes for version NUMBER"
3. Run `npm version patch` (or `minor|major|VERSION`) to run tests and lint,
build published directories, then update `package.json` + add a git tag.
4. Run `git push && git push --tags`

TODO(REGISTRY): Add registry publishing step!!!
21 changes: 21 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2019 Formidable Labs

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
160 changes: 147 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,158 @@
Serverless IAM via Terraform
============================
AWS Serverless Module
=====================
[![Terraform][tf_img]][tf_site]
[![Travis Status][trav_img]][trav_site]

Tuned, locked-down IAM roles for the minimum set of privileges to run the `serverless` framework.
Get your [serverless][] framework application to AWS, the **right way**.

## Contents

## Installation
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

First, [download Terraform](https://www.terraform.io/downloads.html) and [install the binary](https://www.terraform.io/intro/getting-started/install.html). You should be able to run:

```sh
$ terraform --version
- [Overview](#overview)
- [Concepts](#concepts)
- [Modules](#modules)
- [Integration](#integration)
- [Reference project](#reference-project)
- [Module integration](#module-integration)
- [AWS IAM group integration](#aws-iam-group-integration)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

## Overview

Getting a [serverless][] application all the way to production in AWS **correctly** and **securely** can be quite challenging. In particular, things like:

- Locking down IAM permissions to the minimum needed for different conceptual "roles" (e.g., `admin`, `developer`, `ci`).
- Providing a scheme for different environments/stages (e.g., `development`, `staging`, `production`).

... lack reasonable guidance to practically achieve in real world applications.

This [Terraform][] module provides a production-ready base of AWS permissions / resources to support a `serverless` framework application and help manage development / deployment workflows and maintenance. Specifically, it provides:

- **IAM Groups**: Role-specific groups to attach to AWS users to give humans and CI the minimum level of permissions locked to both a specific `serverless` service and stage/environment.

## Concepts

This module allows practical isolation / compartmentalization of privileges within a single AWS account along the folowing axes:

* **Stage/Environment**: An arbitrary environment to isolate -- this module doesn't restrict selection in any way other than there has to be at least one. In practice, a good set of choices may be something like `sandbox`, `development`, `staging`, `production`.
* **IAM Groups**: This module creates/enforces a scheme wherein:
* **Admin**: AWS users assigned to the `admin` group can create/update/delete a `serverless` application and do pretty much anything that the `serverless` framework permits out of the box.
* **Developer, CI**: AWS users assigned to the `developer|ci` groups can update a `serverless` application and do other things like view logs, perform rollbacks, etc.

In this manner, once an AWS superuser deploys a Terraform stack with this module and assigns IAM groups, the rest of the development / devops teams and CI can build and deploy Serverless applications to appropriate cloud targets with the minimum necessary privileges and isolation across services + environments + IAM roles.

## Modules

This project provides a core base module that is the minimum that must be used. Once the core is in place, then other optional submodules can be added.

- **Core (`/*`)**: Provides supporting IAM policies, roles, and groups so that an engineering team / CI can effectively create and maintain `serverless` Framework applications locked down to specific applications + environments with the minimum permissions needed.
- **X-Ray (`modules/xray`)**: Optional submodule to add needed IAM support to enable AWS X-Ray performance tracing in a Serverless framework application. See the [submodule documentation](./modules/xray/README.md).

## Integration

### Reference project

Perhaps the easiest place to start is our [sample reference project][ref_project] that creates a Serverless framework service named `simple-reference` that integrates the core module and submodules of this project. The relevant files to review include:

- Terraform infrastructure
- [aws/bootstrap.yml](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/aws/bootstrap.yml): Terraform remote state storage / bootstrap.
- [terraform/variables.tf](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/terraform/variables.tf): Terraform variables.
- [terraform/main.tf](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/terraform/main.tf): Terraform resources / integration.
- Serverless framework
- [serverless.yml](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/serverless.yml): Serverless framework configuration.
- Example Node.js handlers/servers
- [src/server/base.js](https://github.com/FormidableLabs/aws-lambda-serverless-reference/blob/master/src/server/base.js): Example "hello world" server using only the core `serverless` module.

### Module integration

Here's a basic integration of the core `serverless` module:

```hcl
# variables.tf
variable "stage" {
description = "The stage/environment to deploy to. Suggest: `sandbox`, `development`, `staging`, `production`."
default = "development"
}
# main.tf
provider "aws" {
region = "us-east-1"
version = "~> 1.19"
}
# Core `serverless` IAM support.
module "serverless" {
source = "FormidableLabs/serverless/aws"
region = "us-east-1"
service_name = "sparklepants"
stage = "${var.stage}"
# (Default values)
# iam_region = `*`
# iam_partition = `*`
# iam_account_id = `AWS_CALLER account`
# tf_service_name = `tf-SERVICE_NAME`
# sls_service_name = `sls-SERVICE_NAME`
}
```

- [ ] TODO: Add note about minimum TF version.
That pairs with a `serverless.yml` configuration:

```yml
# This value needs to either be `sls-` + `service_name` module input *or*
# be specified directly as the module input `sls_service_name`.
service: sls-sparklepants

provider:
name: aws
runtime: nodejs8.10
region: "us-east-1"
stage: ${opt:stage, "development"}

functions:
server:
# ...
```

Let's unpack the parameters a bit more (located in [variables.tf](variables.tf)):

- `service_name`: A service name is something that defines the unique application that will match up with the serverless application. E.g., something boring like `simple-reference` or `graphql-server` or exciting like `unicorn` or `sparklepants`.
- `stage`: The current stage that will match up with the `serverless` framework deployment. These are arbitrary, but can be something like `development`/`staging`/`production`.
- `region`: The deployed region of the service. Defaults to the current caller's AWS region. E.g., `us-east-1`.
- `iam_region`: The [AWS region][] to limit IAM privileges to. Defaults to `*`. The difference with `region` is that `region` has to be one specific region like `us-east-1` to match up with Serverless framework resources, whereas `iam_region` can be a single region or `*` wildcard as it's just an IAM restriction.
- `iam_partition`: The [AWS partition][] to limit IAM privileges to. Defaults to `*`.
- `iam_account_id`: The [AWS account ID][] to limit IAM privileges to. Defaults to the current caller's account ID.
- `tf_service_name`: The service name for Terraform-created resources. It is very useful to distinguish between those created by Terraform / this module and those created by the Serverless framework. By default, `tf-${service_name}` for "Terraform". E.g., `tf-simple-reference` or `tf-sparklepants`.
- `sls_service_name`: The service name for Serverless as defined in `serverless.yml` in the `service` field. Highly recommended to match our default of `sls-${service_name}` for "Serverless".

Most likely, an AWS superuser will be needed to run the Terraform application for these IAM / other resources.

### AWS IAM group integration

Once the core module is applied, three IAM groups will be created in the form of `${tf_service_name}-${stage}-(admin|developer|ci)`. This typically looks something like:

- `tf-${service_name}-${stage}-admin`: Can create/delete/update the Severless app.
- `tf-${service_name}-${stage}-developer`: Can deploy the Severless app.
- `tf-${service_name}-${stage}-ci`: Can deploy the Severless app.

## Usage
Once these groups exist, an AWS superuser can then attach these groups to AWS individual users as appropriate for the combination of service + stage + role (admin, developer, CI). Or, the IAM group attachments could be controlled via Terraform as well!

- [ ] TODO: Usage
- [ ] TODO: AWS_PROFILE or AWS_*_KEYS docs
The main upshot of this is after attachment, a given AWS user has the minimum necessary privileges for exactly the level of Serverless framework commands they need. Our example Serverless application [reference project][ref_project] documentation has many examples of various `serverless` commands and which IAM group can properly run them.

## IAM roles and groups
[serverless]: https://serverless.com/
[Terraform]: https://www.terraform.io
[AWS region]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-region
[AWS partition]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-partition
[AWS account ID]: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html#cfn-pseudo-param-accountid
[AWS X-Ray]: https://aws.amazon.com/xray/

- [ ] TODO: discuss admin, developer, ci
[tf_img]: https://img.shields.io/badge/terraform-published-blue.svg
[tf_site]: https://registry.terraform.io/modules/FormidableLabs/serverless/aws
[trav_img]: https://api.travis-ci.org/FormidableLabs/inspectpack.svg
[trav_site]: https://travis-ci.org/FormidableLabs/inspectpack
[ref_project]: https://github.com/FormidableLabs/aws-lambda-serverless-reference
47 changes: 47 additions & 0 deletions main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
###############################################################################
# IAM Groups
#
# - `admin`: An administrator that can create, delete, develop the services.
# - `developer`: A developer that deploy/update an existing service.
# - `ci`: The CI service can deploy/update an existing service.
#
# General reference
# - https://iam.cloudonaut.io/
# - https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
# - http://awspolicygen.s3.amazonaws.com/policygen.html
###############################################################################

# admin
resource "aws_iam_group" "admin" {
name = "${local.tf_service_name}-${local.stage}-admin"
}

resource "aws_iam_group_policy_attachment" "admin_admin" {
group = "${aws_iam_group.admin.name}"
policy_arn = "${aws_iam_policy.admin.arn}"
}

resource "aws_iam_group_policy_attachment" "admin_developer" {
group = "${aws_iam_group.admin.name}"
policy_arn = "${aws_iam_policy.developer.arn}"
}

# ci
resource "aws_iam_group" "ci" {
name = "${local.tf_service_name}-${local.stage}-ci"
}

resource "aws_iam_group_policy_attachment" "ci_developer" {
group = "${aws_iam_group.ci.name}"
policy_arn = "${aws_iam_policy.developer.arn}"
}

# developer
resource "aws_iam_group" "developer" {
name = "${local.tf_service_name}-${local.stage}-developer"
}

resource "aws_iam_group_policy_attachment" "developer_developer" {
group = "${aws_iam_group.developer.name}"
policy_arn = "${aws_iam_policy.developer.arn}"
}
Loading

0 comments on commit a25e951

Please sign in to comment.