API Gateway -> Lambda (Go)

Example using multiple Go functions using the same common library - with integrated CI/CD

Table of Contents

• Installation
• Deployment & Hosting
• Documentation
• Examples & Tests
• Code Standards
• Maintainers
• Contributing
• License

Installation

Prerequisites

• An AWS account
  • Running functions locally requires permission to: CodePipeline and KMS
  • Deploying requires permission to: KMS, SSM, Secrets Manager and Cloud Formation
• AWS CLI (brew install awscli)
• SAM CLI (brew tap aws/tap && brew install aws-sam-cli)
  • Running functions locally requires: Docker
• Golang (brew install go)
• Docker Used for running the local version of the API and functions
• DockerHub Used for custom Go images in CodeBuild

Clone or go get the files locally

go get github.com/mrz1836/go-api-gateway
cd $GOPATH/src/github.com/mrz1836/go-api-gateway

Setup to run locally

1) Start docker on your computer

2) Build and start running! (runs on 127.0.0.1:3000)

make build;
make start;

Deployment & Hosting

This repository has CI integration using AWS CodePipeline.

Deploying to the master branch will automatically start the process of shipping the code to AWS Lambda.

Any changes to the environment via the AWS CloudFormation template will be applied. The actual build process can be found in the buildspec.yml file.

The application relies on AWS Secrets Manager and AWS SSM to store environment variables. Sensitive environment variables are encrypted using AWS KMS and then decrypted at runtime.

Deploy different environments by changing the <stage> to production or staging as an example. The default stage is production if not specified.

Create Environment Encryption Key(s) (AWS)

Create a KMS Key per <stage> for your application(s) to encrypt environment variables

make create-env-key stage="<stage>";

This will also store the kms_key_id in SSM located at: /<application>/<stage>/kms_key_id

Manage DockerHub Credentials (AWS)

• username is your DockerHub username
• password is either your DockerHub password or auth_token (use auth_token if your account has 2FA)
• kms_key_id is from the previous step (Create Environment Encryption Keys)

Add or update your application DockerHub credentials

make save-dockerhub-credentials \
      username="YOUR_DOCKERHUB_USERNAME" \
      password="YOUR_DOCKERHUB_PASSWORD" \
      kms_key_id="YOUR_KMS_KEY_ID" \
      stage="<stage>";

Next run:

make save-dockerhub-arn \
      arn="YOUR_SECRETS_ARN" \
      stage="<stage>";

Manage Environment Secrets (AWS)

• github_token is a GitHub Personal Token for webhooks
• example_secret is an example secret value that will be encrypted
• kms_key_id is from the previous step (Create Environment Encryption Keys)

Add or update your GitHub personal access token

make save-secrets \
      github_token="YOUR_GITHUB_TOKEN" \
      example_secret="YOUR_EXAMPLE_SECRET_VALUE" \
      kms_key_id="YOUR_KMS_KEY_ID" \
      stage="<stage>";

Create New CI & Hosting Environment (AWS)

This will create a new AWS CloudFormation stack with:

• (1) API Gateway RESTful API interface
• (1) Lambda Function(s) (Golang Runtime)
• (1) CloudWatch LogGroup for the Lambda function output
• (1) CodePipeline with multiple stages to deploy the application from GitHub
• (1) CodePipeline Webhook to receive GitHub notifications from a specific branch:name
• (1) CodeBuild Project to test, build and deploy the app
• (2) Service Roles for working with CodeBuild and CodePipeline

NOTE: Requires an existing S3 bucket for artifacts and sam-cli deployments (located in the Makefile)

One command will build, test, package and deploy the application to AWS using the default production stage and using default tags. After initial deployment, updating the function is as simple as committing to GitHub.

make deploy;

(Example) Customized deployment for another stage

make deploy stage="development" branch="development";

(Example) Customized deployment for a feature branch

make deploy stage="development" branch="some-feature" feature="some-feature";

(Example) Customized S3 bucket location

make deploy bucket="some-S3-bucket-location";

(Example) Customized tags for the deployment

make deploy tags="MyTag=some-value AnotherTag=some-value";

Tear Down CI & Hosting Environment (AWS)

Remove the stack (using default stage: production)

make teardown;

(Example) Teardown another stack via stage

make teardown stage="development";

(Example) Teardown a feature/branch stack

make teardown stage="development" feature="some-feature";

Lambda Logging

View all the logs in AWS CloudWatch via Log Groups

/aws/lambda/<app_name>-<stage_name>

Documentation

Release Deployment

goreleaser for easy binary or library deployment to GitHub and can be installed via: brew install goreleaser.

The .goreleaser.yml file is used to configure goreleaser.

Use make release-snap to create a snapshot version of the release, and finally make release to ship to production.

Makefile Commands

View all makefile commands

make help

List of all current commands:

aws-param-certificate           Returns the ssm location for the domain ssl certificate id
aws-param-dockerhub             Returns the ssm location for the DockerHub ARN
aws-param-zone                  Returns the ssm location for the host zone id
build                           Build the SAM application
clean                           Remove previous builds, test cache, and packaged releases
clean-mods                      Remove all the Go mod cache
coverage                        Shows the test coverage
create-env-key                  Creates a new key in KMS for a new stage
create-secret                   Creates an secret into AWS SecretsManager
decrypt                         Decrypts data using a KMY Key ID (awscli v2)
decrypt-deprecated              Decrypts data using a KMY Key ID (awscli v1)
deploy                          Build, prepare and deploy
diff                            Show the git diff
encrypt                         Encrypts data using a KMY Key ID (awscli v2)
env-key-location                Returns the environment encryption key location
generate                        Runs the go generate command in the base of the repo
godocs                          Sync the latest tag with GoDocs
help                            Show this help message
install                         Install the application
install-go                      Install the application (Using Native Go)
install-releaser                Install the GoReleaser application
invalidate-cache                Invalidates a cloudfront cache based on path
lint                            Run the golangci-lint application (install if not found)
package                         Process the CF template and prepare for deployment
release                         Full production release (creates release in Github)
release-snap                    Test the full release (build binaries)
release-test                    Full production test release (everything except deploy)
replace-version                 Replaces the version in HTML/JS (pre-deploy)
save-dockerhub-arn              Updates the ARN for the DockerHub secret
save-dockerhub-credentials      Helper for saving DockerHub credentials to Secrets Manager
save-domain-info                Saves the zone id and the ssl id for use by CloudFormation
save-param                      Saves a plain-text string parameter in SSM
save-param-encrypted            Saves an encrypted string value as a parameter in SSM
save-secrets                    Helper for saving application secrets to Secrets Manager (extendable for more secrets)
start                           Start the application
tag                             Generate a new tag and push (tag version=0.0.0)
tag-remove                      Remove a tag if found (tag-remove version=0.0.0)
tag-update                      Update an existing tag to current commit (tag-update version=0.0.0)
teardown                        Deletes the entire stack
test                            Runs lint and ALL tests
test-ci                         Runs all tests via CI (exports coverage)
test-ci-no-race                 Runs all tests via CI (no race) (exports coverage)
test-ci-short                   Runs unit tests via CI (exports coverage)
test-no-lint                    Runs just tests
test-short                      Runs vet, lint and tests (excludes integration tests)
test-unit                       Runs tests and outputs coverage
uninstall                       Uninstall the application (and remove files)
update-linter                   Update the golangci-lint package (macOS only)
update-secret                   Updates an existing secret in AWS SecretsManager
upload-files                    Upload/puts files into S3 bucket
vet                             Run the Go vet application

Examples & Tests

All unit tests run via GitHub Actions and uses Go version 1.20.x. View the configuration file.

Run all tests (including integration tests)

make test

Run unit tests (excluding integration tests)

make test-short

Code Standards

Read more about this Go project's code standards.

Maintainers

MrZ

Contributing

View the contributing guidelines and please follow the code of conduct.

How can I help?

All kinds of contributions are welcome 🙌! The most basic way to show your support is to star 🌟 the project, or to raise issues 💬. You can also support this project by becoming a sponsor on GitHub 👏 or by making a bitcoin donation to ensure this journey continues indefinitely! 🚀

License