Skip to content

Home

Jump to bottom
joshuayhwu edited this page Sep 2, 2022 · 33 revisions

Welcome to the ops wiki!

We are in the process of doing knowledge transfer and building out this wiki to be useful to all team members. If you think there is something that could be added or revised, please bring it up at the Community of Practice meeting or add it to the following issue

Joining the Ops team

We use the following tools

Documentation

Users

Incubator Hosting

  • Documentation can be found in the incubator repository installation-instructions/documentation directory
    • Topics
      • Adding-Project.md
      • Deploying-With-Incubator.md
      • Getting-Started.md
      • Installation-Instructions.md
      • Known_Common_Errors.md
      • Setting-Up-Incubator-Environment.md

The following documentation is a comprehensive list of steps to deploy incubator. The documentation includes:

  1. Software Installation Instructions
  2. Instructions to set up Incubator environment
  3. Project HCL template
  4. Using Terragrunt to deploy to Incubator
  5. Starting your own incubator

Installation Instructions

Deploying with the Incubator requires:

1. Installing AWS CLI version 2

Find your corresponding platform here https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html and make sure to install CLI Version 2.

Confirm aws installation with which aws.

which aws
/usr/local/bin

2. Setting up the AWS CLI with Hack for LA credentials

Access keys should be given to you by administrators in order to use with aws configure. This command will setup a default profile, but best practices would to configure a named profile.

Named profiles will setup different profiles for AWS to distinguish credentials. One's AWS account may already have other aws credentials from previous AWS projects and works, so a default profile won't suffice. With --profile, make sure to switch to the right named profile by adding --profile <profile name> at the end of aws commands.

aws configure --profile hfla-example

//response
AWS Access Key ID [None]: <insert access key id>
AWS Secret Access Key [None]: <insert secret accesskey>
Default Region Name [None]: us-west-2
Default output format [None]: <press Enter for a default output>

Afterwards, verify with aws sts --profile hfla-example get-caller-identity

aws sts --profile hfla-example get-caller-identity
{
    "UserId": "ABCDEFGH12345",
    "Account": "1234567891011",
    "Arn": "arn:aws:iam::1234568791011:user/user@email.com"
}

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

3. Installing Terraform

The latest Terraform won't apply for incubator applications, instead Terraform version 0.15.1 needs to be installed. This assumes a package manager like Homebrew is installed.

Homebrew (OSX/Linux)

brew update
brew install terraform@0.15.1

Apt-get (Linux)

sudo apt-get update
sudo apt-get install terraform=0.15.1

Confirm installation with terraform --version.

terraform --version
Terraform v0.15.1
on linux_amd64

4. Installing Terragrunt

Homebrew (OSX/Linux)

https://terragrunt.gruntwork.io/docs/getting-started/install/

brew update
brew install terragrunt

Confirm installation with terragrunt --version

terragrunt --version
terragrunt version v0.32.1

Setting up Incubator environment

For the incubator environment, one should have their aws credentials, terraform, and terragrunt setup.

1.) Clone the repo

Clone the incubator repo from here https://github.com/hackforla/incubator using preferred method.

git clone https://github.com/hackforla/incubator

2.) Obtain the rds.hcl file from administrators

This is a file with sensitive database information. Do NOT commit this file to git for the public to view.

echo 'incubator/rds.hcl` >> .gitignore

One will obtain a rds.hcl file from administrators, and it will be placed in the /incubator directory.

3.) Set AWS_PROFILE variable

export AWS_PROFILE=<named-profile-used-for-hfla>

After this one should be able to deploy to the incubator with Terragrunt.

Project HCL

Every project will own their own project.hcl file, in their devops 1password vault.

When they want to do deployment with it, the team's ops delegate will share it with the vault "Ops-project hcl files"

It should have a name similar to this: ctj_project.hcl - last updated 2022-07-20

After it's used by the ops team, it should be deleted from the ops team members local machine.

Project.hcl template

locals {
  // General Variables
  project_name = "project-name-here"
  environment  = "stage"

  // ALB
  aws_managed_dns = false
  host_names      = [""]
  path_patterns   = ["/*"] 

  // ECS
  application_type = "fullstack"
  launch_type      = "FARGATE"

  // RDS (Database)
  postgres_database = {
    // no RDS credentials here
    // this is only for creating a new db 
  }

  // Container variables
  container_image   = "035866691871.dkr.ecr.us-west-2.amazonaws.com/civictechjobs-fullstack-stage:latest" 
  desired_count     = 1
  container_port    = 8000 // should match server used by application 
  container_memory  = 512
  container_cpu     = 256
  health_check_path = "/"

  container_env_vars = {
    // key : value 
    // any env variables necessary for the docker container_image
    // RDS info here
  }
}

Using Terragrunt to deploy to Incubator

Setup Incubator Project Directories

Under /incubator, select the corresponding environment to deploy to. For example, to deploy a Django application into Hack For LA's AWS account within a dev environment, one should create a incubator/project-dev/django-application-example directory for the codebase.

Configure project.hcl

project.hcl helps provide environmental variables and configuration for the incubator deployed application. This file will have aspects such as the host_names, container settings, and health checks url routes.

There should also be a terragrunt.hcl within the directory as well, but it doesn't need to be configured. This file will will copy the Terraform configurations specified by the source parameter, along with any files in the working directory, into a temporary folder, and execute Terraform commands in that folder.

Deploy with Terragrunt

terragrunt init
terragrunt plan
terragrunt apply

After apply one can examine AWS ECS console for activity.

To remove application from AWS ECS:

terragrunt destroy

Requirements for setting your own Incubator

  1. AWS access/credentials

    • It's recommended to have Administrator access to ensure proper permissions
    • Documentation

  2. Binaries

  3. Hosted Zone in AWS Route53 via 2 methods

  4. Terraform State and Lock files requires pre-created resources Documentation

    • S3 Bucket for storing state file
    • DynamoDB Table for storing lock files (recommended default: terraform-locks)
    • see examples folder for how to properly set

  5. Database Credentials for RDS Database, create file in REPO_ROOT/incubator/rds.hcl

locals {
  db_username="postgres"
  db_password="super-secret-password"
}

Dedicated Hosting

Assets to be reviewed

Welcome!

Community
CoP Meetings
Ops Open Roles
Dev Resources

Clone this wiki locally