Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First Pass: Deployment to LZ0 using GitHub actions #7

Merged
merged 25 commits into from
Sep 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d723e2b
First pass: setup supporting infrastructure
unoah May 19, 2022
d644e9c
First pass: added Glue related resources based on discovery on LZ2 ma…
unoah May 19, 2022
011d411
Created billing-cpf-1068.py - initial setup for using EventBridge
unoah May 21, 2022
86d3c4e
First pass: some cleanup on functions used to perform Athena Queries
unoah May 24, 2022
ae4f6e2
Begin cleanup of billing.py - created BillingManager.py and helpers.p…
unoah May 25, 2022
104721c
Minor clenup around STS related function
unoah May 27, 2022
7d79337
Added CMK for use with Athena Query output bucket. Some cleanup.
unoah May 30, 2022
cb6a791
Moved AthenaQuery output bucket to master account. Updated AthenaQuer…
unoah Jun 1, 2022
4c42f57
Clean up on start/end dates for quarterly and monthly functions. Init…
unoah Jun 2, 2022
faf26b3
Moved send_email function to helpers.py. Removed billing.py and email…
unoah Jun 2, 2022
2d67acc
Fiscal week begins Wed at 00:00:00 and ends Tue at 11:59:59. Updated …
unoah Jun 3, 2022
f2840b2
Updates to date range for weekly and manual reports. Cleaned up requi…
unoah Jun 6, 2022
2bb6c09
Cleanup of Dockerfile and Terraform config for LZ Operations and Mana…
unoah Jun 7, 2022
9b73a0c
Minor cleanup on Terraform config files for LZ operator and managemen…
unoah Jun 7, 2022
af032de
Terraform resource cleanup
unoah Jun 7, 2022
3656ba3
Initial commit of revised readme file.
unoah Jun 8, 2022
4cfad28
Additional updates to readme file. Added DrawIO figures.
unoah Jun 8, 2022
ceeb3d5
Basic formatting of Python files. Added container insightes to ECS cl…
unoah Jun 9, 2022
71e32c2
Merge pull request #6 from BCDevOps/cpf-1068
unoah Jun 14, 2022
d41e936
Cleaned up deployment using GitHub actions.
unoah Jun 24, 2022
2c32e08
Removed ARM64 arch from ECS task definition. GitHub actions set to us…
unoah Jun 27, 2022
052e49e
Added FX converion and alerting system to billing utility
prabhukiran9999 Aug 18, 2022
45cac0b
Modify permissions to access parameters
prabhukiran9999 Aug 19, 2022
50f6d1c
modify resource names
wrnu Sep 21, 2022
035757a
remove s3 backend config
wrnu Sep 21, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 144 additions & 0 deletions .github/workflows/deploy-dev-resources.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
---
name: "Deploy Dev (LZ0) Resources"

on:
push:
branches: [dev]

# permission can be added at job level or workflow level
permissions:
id-token: write
contents: read # This is required for actions/checkout@v2

jobs:
DeployMgmtAccountResources:
runs-on: ubuntu-latest
environment: dev

env:
AWS_REGION: "ca-central-1"
TF_VAR_ops_account_id: ${{ secrets.TF_VAR_OPS_ACCOUNT_ID }}
TF_VAR_mgmt_account_phase1_bucket_suffix: ${{ secrets.TF_VAR_MGMT_ACCOUNT_PHASE1_BUCKET_SUFFIX }}

steps:
- name: Git clone the repository
uses: actions/checkout@v2

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.MGMT_TFRB_ROLE_TO_ASSUME }}
role-session-name: GitHubOidcBillingUtilitySession
aws-region: ${{ env.AWS_REGION }}

# Ref for Teffaform Versions: https://releases.hashicorp.com/terraform/
- uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.0.11

- name: Terraform Init
id: init
run: |
cd ./management-account-terraform-resources
terraform init -input=false -force-copy \
-backend-config="bucket=${{ secrets.MGMT_TFRB_S3_BUCKET_NAME }}" \
-backend-config="key=terraform-octk-aws-sea-billing-reports/management-account-terraform-resources.tfstate" \
-backend-config="dynamodb_table=${{ secrets.MGMT_TFRB_DYNAMODB_TABLE_NAME }}" \
-backend-config="region=${{ env.AWS_REGION }}"

- name: Terraform Fmt
id: fmt
run: |
cd ./management-account-terraform-resources
terraform fmt
continue-on-error: true

- name: Terraform Validate
id: validate
run: |
cd ./management-account-terraform-resources
terraform validate
#
- name: Terraform Plan
id: plan
run: |
cd ./management-account-terraform-resources
terraform plan

- name: Terraform Apply
id: apply
run: |
cd ./management-account-terraform-resources
terraform apply -auto-approve
#
# - name: Terraform Destroy
# id: destroy
# run: |
# cd ./management-account-terraform-resources
# terraform destroy -auto-approve

DeployOpsAccountResources:
needs: [DeployMgmtAccountResources]
runs-on: ubuntu-latest
environment: dev

env:
AWS_REGION: "ca-central-1"
TF_VAR_lz_mgmt_account_id: ${{ secrets.TF_VAR_MGMT_ACCOUNT_ID }}

steps:
- name: Git clone the repository
uses: actions/checkout@v2

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.OPS_TFRB_ROLE_TO_ASSUME }}
role-session-name: GitHubOidcBillingUtilitySession
aws-region: ${{ env.AWS_REGION }}

# Ref for Teffaform Versions: https://releases.hashicorp.com/terraform/
- uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.11

- name: Terraform Init
id: init
run: |
cd ./operations-account-terraform-resources
terraform init -input=false -force-copy \
-backend-config="bucket=${{ secrets.OPS_TFRB_S3_BUCKET_NAME }}" \
-backend-config="key=terraform-octk-aws-sea-billing-reports/operations-account-terraform-resources.tfstate" \
-backend-config="dynamodb_table=${{ secrets.OPS_TFRB_DYNAMODB_TABLE_NAME }}" \
-backend-config="region=${{ env.AWS_REGION }}"

- name: Terraform Fmt
id: fmt
run: |
cd ./operations-account-terraform-resources
terraform fmt
continue-on-error: true

- name: Terraform Validate
id: validate
run: |
cd ./operations-account-terraform-resources
terraform validate

- name: Terraform Plan
id: plan
run: |
cd ./operations-account-terraform-resources
terraform plan

- name: Terraform Apply
id: apply
run: |
cd ./operations-account-terraform-resources
terraform apply -auto-approve
#
# - name: Terraform Destroy
# id: destroy
# run: |
# cd ./operations-account-terraform-resources
# terraform destroy -auto-approve
131 changes: 131 additions & 0 deletions .github/workflows/destroy-dev-resources.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
name: "Destroy Dev (LZ0) Resources"

on:
workflow_dispatch:

# permission can be added at job level or workflow level
permissions:
id-token: write
contents: read # This is required for actions/checkout@v2

jobs:
DestroyOpsAccountResources:
runs-on: ubuntu-latest
environment: dev

env:
AWS_REGION: "ca-central-1"
TF_VAR_lz_mgmt_account_id: ${{ secrets.TF_VAR_MGMT_ACCOUNT_ID }}

steps:
- name: Git clone the repository
uses: actions/checkout@v2

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.OPS_TFRB_ROLE_TO_ASSUME }}
role-session-name: GitHubOidcBillingUtilitySession
aws-region: ${{ env.AWS_REGION }}

# Ref for Teffaform Versions: https://releases.hashicorp.com/terraform/
- uses: hashicorp/setup-terraform@v1
with:
terraform_version: 1.0.11

- name: Terraform Init
id: init
run: |
cd ./operations-account-terraform-resources
terraform init -input=false -force-copy \
-backend-config="bucket=${{ secrets.OPS_TFRB_S3_BUCKET_NAME }}" \
-backend-config="key=terraform-octk-aws-sea-billing-reports/operations-account-terraform-resources.tfstate" \
-backend-config="dynamodb_table=${{ secrets.OPS_TFRB_DYNAMODB_TABLE_NAME }}" \
-backend-config="region=${{ env.AWS_REGION }}"

- name: Terraform Fmt
id: fmt
run: |
cd ./operations-account-terraform-resources
terraform fmt
continue-on-error: true

- name: Terraform Validate
id: validate
run: |
cd ./operations-account-terraform-resources
terraform validate

- name: Terraform Plan
id: plan
run: |
cd ./operations-account-terraform-resources
terraform plan
#
# - name: Terraform Destroy
# id: destroy
# run: |
# cd ./operations-account-terraform-resources
# terraform destroy -auto-approve

DestroyMgmtAccountResources:
needs: [DestroyOpsAccountResources]
runs-on: ubuntu-latest
environment: dev

env:
AWS_REGION: "ca-central-1"
TF_VAR_ops_account_id: ${{ secrets.TF_VAR_OPS_ACCOUNT_ID }}
TF_VAR_mgmt_account_phase1_bucket_suffix: ${{ secrets.TF_VAR_MGMT_ACCOUNT_PHASE1_BUCKET_SUFFIX }}

steps:
- name: Git clone the repository
uses: actions/checkout@v2

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
role-to-assume: ${{ secrets.MGMT_TFRB_ROLE_TO_ASSUME }}
role-session-name: GitHubOidcBillingUtilitySession
aws-region: ${{ env.AWS_REGION }}

# Ref for Teffaform Versions: https://releases.hashicorp.com/terraform/
- uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.0.11

- name: Terraform Init
id: init
run: |
cd ./management-account-terraform-resources
terraform init -input=false -force-copy \
-backend-config="bucket=${{ secrets.MGMT_TFRB_S3_BUCKET_NAME }}" \
-backend-config="key=terraform-octk-aws-sea-billing-reports/management-account-terraform-resources.tfstate" \
-backend-config="dynamodb_table=${{ secrets.MGMT_TFRB_DYNAMODB_TABLE_NAME }}" \
-backend-config="region=${{ env.AWS_REGION }}"

- name: Terraform Fmt
id: fmt
run: |
cd ./management-account-terraform-resources
terraform fmt
continue-on-error: true

- name: Terraform Validate
id: validate
run: |
cd ./management-account-terraform-resources
terraform validate
#
- name: Terraform Plan
id: plan
run: |
cd ./management-account-terraform-resources
terraform plan

# - name: Terraform Destroy
# id: destroy
# run: |
# cd ./management-account-terraform-resources
# terraform destroy -auto-approve
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ Thumbs.db

##### Terraform-specific ignores.

# backend configuration
**/backend.hcl

# Local .terraform directories
**/.terraform/*

Expand Down Expand Up @@ -90,3 +93,5 @@ terraform.rc
/lib
/pyenv.cfg
pyvenv.cfg

.vscode/
20 changes: 0 additions & 20 deletions .terraform.lock.hcl

This file was deleted.

11 changes: 6 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3 python3-pip
COPY ./billing-report-utility .
RUN pip3 install -r requirements.txt
ENTRYPOINT python3 billing.py $ARGS
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=America/Vancouver
RUN apt-get update && apt-get install -y python3 python3-pip awscli
WORKDIR /app
COPY ./billing-report-utility/ ./
RUN pip3 install -r requirements.txt
Loading