Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scrum 37 email sender sqs #17

Merged
merged 81 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
a9c6124
refactor: add Terraform configuration for dev, poc, and prod environm…
sh1un Jun 10, 2024
cd8e4a4
refactor: update IAM role and Lambda function names
sh1un Jun 10, 2024
a035403
feat(terraform): update Terraform configuration for all of APIs
sh1un Jun 10, 2024
cc4dce0
build: add GitHub Actions workflow for Terraform infrastructure chang…
sh1un Jun 10, 2024
578f77d
Merge pull request #14 from aws-educate-tw/SCRUM-22-Build-a-CI-CD-pip…
sh1un Jun 10, 2024
484aebe
feat: add dev env Terraform
sh1un Jun 10, 2024
3ffaf6f
fix: fixed wrong Terraform configuration for variable
sh1un Jun 10, 2024
51a4cc5
fix: remove extra quotation marks
sh1un Jun 10, 2024
fc3dad1
fix: remove extra quotation marks
sh1un Jun 10, 2024
61999c9
chore: add GITHUB_TOKEN to environment variables in Terraform workflow
sh1un Jun 10, 2024
5048a2f
chore: update GITHUB_TOKEN in Terraform workflow
sh1un Jun 10, 2024
130ab46
chore: update permissions in Terraform workflow
sh1un Jun 10, 2024
8e3e737
chore: add line break in Terraform workflow comment
sh1un Jun 10, 2024
aa57e56
chore: add Slack Deployment Notification workflow
sh1un Jun 10, 2024
0cc053d
Merge pull request #15 from aws-educate-tw/SCRUM-22-Build-a-CI-CD-pip…
sh1un Jun 10, 2024
7913f0a
chore: ignore SAM CLI build directory in .gitignore SCRUM-37
Rich627 Jun 12, 2024
e65b33b
refactor: remove unnecessary files SCRUM-37
Rich627 Jun 15, 2024
16b48b1
chore: update dependencies boto3, botocore, requests, and openpyxl
Rich627 Jun 15, 2024
e2f8745
chore: remove unused .aws-sam/build.toml file
Rich627 Jun 15, 2024
89f228f
feat: implement SQS triggered lambda function to send email
Rich627 Jun 15, 2024
e6e3021
feat: add lambda function to validate user;s input and send requests …
Rich627 Jun 15, 2024
8fadd26
fix: delete sqs message successfully SCRUM-37
Rich627 Jun 24, 2024
b3a63b2
refactor: reorganize import statements
Rich627 Jun 24, 2024
a274a9b
feat: add local-dev public s3 bucket
sh1un Jun 28, 2024
548d068
feat(email_service): add email service infra (SCRUM-52)
sh1un Jun 28, 2024
061918d
feat(email_service/send_email): add necessary lambda environment vari…
sh1un Jun 28, 2024
a614015
feat(email_service): add terraform lock file
sh1un Jun 28, 2024
62d41a7
fix(email_service/terraform): fixed wrong Dockerfile path for send em…
sh1un Jun 28, 2024
acb4297
chore(file_service): move stale files to backup folder (SCRUM-52)
sh1un Jun 28, 2024
2a2dec5
feat(email_service): add send email lambda function Dockerfile
sh1un Jun 28, 2024
61f0d4f
fix(email_service/terraform): resolve error where terraform docker mo…
sh1un Jun 28, 2024
83b5d97
chore(email_service/terraform): change backend s3 object key (SCRUM-52)
sh1un Jun 28, 2024
684ca15
feat(file_service): init file service terraform (SCRUM-52)
sh1un Jun 28, 2024
cc0d166
refactor(file_service): change lambda function file name and close cr…
sh1un Jun 28, 2024
18535b4
refactor(file_service/get_file): change file name and environment var…
sh1un Jun 28, 2024
a66a14f
feat(file_service/terraform): add new lambda function `get_file` (SCR…
sh1un Jun 28, 2024
396b665
feat(file_service/terraform): add new lambda function and API endpoin…
sh1un Jun 28, 2024
c3b8707
fix(email_service/send_email): access denied caused by insufficient p…
sh1un Jun 28, 2024
efae95a
build(email_service/send_email): remove unnecessary dependencies
sh1un Jun 28, 2024
d985ee9
feat: add examples for ACM and Terraform configuration (SCRUM-52)
sh1un Jul 6, 2024
095246f
feat: creating api gateway custom domain and DNS record (SCRUM-52)
sh1un Jul 6, 2024
f5a5387
feat(terraform/acm_shared_cert): request `*.awseducate.systems` cert …
sh1un Jul 6, 2024
59573f4
refactor: use Terraform AWS provider to create custom domain and reco…
sh1un Jul 7, 2024
7f2589c
refactor: rename resources and update descriptions to improve readabi…
sh1un Jul 7, 2024
e20a02d
feat(scr/file_service): add new api `GET /files` (SCRUM-52)
sh1un Jul 7, 2024
f5ee367
fix(file_service): fix insufficient access to use GSI in list files e…
sh1un Jul 7, 2024
09cbdd0
chore: remove .tfvars in .gitignore (SCRUM-52)
sh1un Jul 7, 2024
65ad4cc
feat(actions): add file service and email service CI/CD configuration…
sh1un Jul 7, 2024
1f232b8
fix(actions): fixed wrong Terraform path (SCRUM-52)
sh1un Jul 7, 2024
d2d962d
fix(file_service): fixed wrong terraform backend region (SCRUM-52)
sh1un Jul 7, 2024
30b5c77
feat(storage/s3): deploy S3 bucket for each environment (SCRUM-52)
sh1un Jul 7, 2024
bddbfcf
refactor(actions): update CI/CD triggers (SCRUM-52)
sh1un Jul 7, 2024
2349062
test: CI/CD test
sh1un Jul 7, 2024
94e7d8f
test: CI/CD test
sh1un Jul 7, 2024
8708e5d
test: test CI/CD (SCRUM-52)
sh1un Jul 7, 2024
906bca4
test: test CI/CD (SCRUM-52)
sh1un Jul 7, 2024
b3c208f
test: test CI/CD (SCRUM-52)
sh1un Jul 7, 2024
fe37e4c
refactor(actions): upgrade actions/checkout version (SCRUM-52)
sh1un Jul 7, 2024
cc945b9
test: add additional steps for debug (SCRUM-52)
sh1un Jul 7, 2024
ffc5eb0
test: add additional steps for debug (SCRUM-52)
sh1un Jul 7, 2024
8c3ac36
test: test CI/CD (SCRUM-52)
sh1un Jul 7, 2024
40fd77e
test: test CI/CD (SCRUM-52)
sh1un Jul 7, 2024
fc4ab02
chore(actions): remove unnecessary code (SCRUM-52)
sh1un Jul 7, 2024
73700bb
Merge pull request #22 from aws-educate-tw/SCRUM-52-Shiun-Terraform-m…
sh1un Jul 7, 2024
6909806
test: add additional steps for debugging (SCRUM-52)
sh1un Jul 7, 2024
9588a73
Merge pull request #23 from aws-educate-tw/SCRUM-52-Shiun-Terraform-m…
sh1un Jul 7, 2024
2e8c602
test: add additional steps for debugging (SCRUM-52)
sh1un Jul 7, 2024
a18aae2
test: update trigger condition (SCRUM-52)
sh1un Jul 7, 2024
9101b42
Merge pull request #24 from aws-educate-tw/SCRUM-52-Shiun-Terraform-m…
sh1un Jul 7, 2024
f8a3aef
fix(actions): fixed if...else syntax error (SCRUM-52)
sh1un Jul 7, 2024
51fc2ca
Merge pull request #25 from aws-educate-tw/SCRUM-52-Shiun-Terraform-m…
sh1un Jul 7, 2024
cccd184
Merge branch 'dev' into SCRUM-37-Email-sender-SQS
sh1un Jul 9, 2024
66e6f1c
feat(email_service): add sqs for sending email (SCRUM-37)
sh1un Jul 10, 2024
1d0f439
fix(email_service): rename the directory to use an underscore instead…
sh1un Jul 10, 2024
6c89a9d
refactor(email_service/validate_input/Dockerfile): remove unused code…
sh1un Jul 10, 2024
f1c7e07
refactor(email_service): improve readability and move the constants t…
sh1un Jul 10, 2024
47279c6
feat(email_service): add send_email API infrastructure (SCRUM-37)
sh1un Jul 10, 2024
bc2b154
refactor(email_service): use GSI instead of LSI (SCRUM-37)
sh1un Jul 11, 2024
b01730a
chore(file_service): rename module resource (SCRUM-37)
sh1un Jul 11, 2024
953282f
refactor(email_service/send_email): add DLQ and change logic to only …
sh1un Jul 11, 2024
f824ad8
refactor(email_service/send_email): split the lambda_function into se…
sh1un Jul 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions .github/workflows/email_service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
name: "Email Service Infrastructure Change Management Pipeline with GitHub Actions"

on:
pull_request:
paths:
- src/email_service/**
push:
branches:
- main
- dev
paths:
- src/email_service/**

env:
TF_LOG: INFO
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_DIR: ./src/email_service/terraform

permissions:
contents: write
issues: write
pull-requests: write
actions: read
checks: write

jobs:
preview:
name: "Deploy Preview Environment"
if: github.event_name == 'pull_request' && github.event.action != 'closed'
runs-on: ubuntu-latest
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Terraform Init
id: init
run: terraform init -backend-config="preview.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy (Cleanup)
id: destroy
run: terraform destroy -auto-approve -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Plan
id: plan
run: terraform plan -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Apply
id: apply
run: terraform apply -var-file='preview.tfvars' -auto-approve
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- uses: actions/github-script@v6
if: github.event_name == 'pull_request'
env:
APPLY: "terraform\n${{ steps.apply.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});

const botComment = comments.find(comment => {
return comment.user.type === 'Bot' && comment.body.includes('Terraform Apply Results');
});

const output = `#### Terraform Apply Results 🌟
Preview environment has been deployed. You can now test your changes.

<details><summary>Show Actual Apply</summary>

\`\`\`\n
${{ steps.apply.outputs.stdout }}
\`\`\`

</details>

*Pushed by: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: output,
});
} else {
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output,
});
}

cleanup_preview:
name: "Cleanup Preview Environment"
if: github.event_name == 'pull_request' && github.event.action == 'closed' || github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Terraform Init
run: terraform init -backend-config="preview.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy
run: terraform destroy -auto-approve -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}

deploy:
name: "Deploy to Target Environment"
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev')
runs-on: ubuntu-latest
needs: [cleanup_preview]
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Set Environment Variables
run: |
if [[ "${{ github.ref }}" == "refs/heads/main" ]]; then
echo "ENVIRONMENT=prod" >> $GITHUB_ENV
echo "TF_VARS_FILE=prod.tfvars" >> $GITHUB_ENV
elif [[ "${{ github.ref }}" == "refs/heads/dev" ]]; then
echo "ENVIRONMENT=dev" >> $GITHUB_ENV
echo "TF_VARS_FILE=dev.tfvars" >> $GITHUB_ENV
else
echo "Unsupported branch: ${{ github.ref }}"
exit 1
fi

- name: Terraform Init
id: init
run: terraform init -backend-config="${{ env.ENVIRONMENT }}.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy (Cleanup)
id: destroy
run: terraform destroy -auto-approve -var-file='${{ env.TF_VARS_FILE }}'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Plan
id: plan
run: terraform plan -var-file='${{ env.TF_VARS_FILE }}' -out=tfplan
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Apply
id: apply
run: terraform apply -var-file='${{ env.TF_VARS_FILE }}' -auto-approve
working-directory: ${{ env.TF_DIR }}
continue-on-error: true
185 changes: 185 additions & 0 deletions .github/workflows/file_service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
name: "File Service Infrastructure Change Management Pipeline with GitHub Actions"

on:
pull_request:
paths:
- src/file_service/**
push:
branches:
- main
- dev
paths:
- src/file_service/**

env:
TF_LOG: INFO
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TF_DIR: ./src/file_service/terraform

permissions:
contents: write
issues: write
pull-requests: write
actions: read
checks: write

jobs:
preview:
name: "Deploy Preview Environment"
if: github.event_name == 'pull_request' && github.event.action != 'closed'
runs-on: ubuntu-latest
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v4
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Terraform Init
id: init
run: terraform init -backend-config="preview.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy (Cleanup)
id: destroy
run: terraform destroy -auto-approve -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Plan
id: plan
run: terraform plan -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Apply
id: apply
run: terraform apply -var-file='preview.tfvars' -auto-approve
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- uses: actions/github-script@v6
if: github.event_name == 'pull_request'
env:
APPLY: "terraform\n${{ steps.apply.outputs.stdout }}"
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});

const botComment = comments.find(comment => {
return comment.user.type === 'Bot' && comment.body.includes('Terraform Apply Results');
});

const output = `#### Terraform Apply Results 🌟
Preview environment has been deployed. You can now test your changes.

<details><summary>Show Actual Apply</summary>

\`\`\`\n
${{ steps.apply.outputs.stdout }}
\`\`\`

</details>

*Pushed by: @${{ github.actor }}, Action: \`${{ github.event_name }}\`*`;

if (botComment) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: botComment.id,
body: output,
});
} else {
await github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output,
});
}

cleanup_preview:
name: "Cleanup Preview Environment"
if: github.event_name == 'pull_request' && github.event.action == 'closed' || github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Terraform Init
run: terraform init -backend-config="preview.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy
run: terraform destroy -auto-approve -var-file='preview.tfvars'
working-directory: ${{ env.TF_DIR }}

deploy:
name: "Deploy to Target Environment"
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev')
runs-on: ubuntu-latest
needs: [cleanup_preview]
steps:
- name: "Print Debug Information"
run: |
echo "Event Name: ${{ github.event_name }}"
echo "Event Action: ${{ github.event.action }}"
echo "Merged: ${{ github.event.pull_request.merged }}"
- uses: actions/checkout@v3
- uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.8.3

- name: Set Environment Variables
run: |
if [[ "${{ github.ref }}" == "refs/heads/main" ]]; then
echo "ENVIRONMENT=prod" >> $GITHUB_ENV
echo "TF_VARS_FILE=prod.tfvars" >> $GITHUB_ENV
elif [[ "${{ github.ref }}" == "refs/heads/dev" ]]; then
echo "ENVIRONMENT=dev" >> $GITHUB_ENV
echo "TF_VARS_FILE=dev.tfvars" >> $GITHUB_ENV
else
echo "Unsupported branch: ${{ github.ref }}"
exit 1
fi

- name: Terraform Init
id: init
run: terraform init -backend-config="${{ env.ENVIRONMENT }}.tfbackend" -reconfigure
working-directory: ${{ env.TF_DIR }}

- name: Terraform Destroy (Cleanup)
id: destroy
run: terraform destroy -auto-approve -var-file='${{ env.TF_VARS_FILE }}'
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Plan
id: plan
run: terraform plan -var-file='${{ env.TF_VARS_FILE }}' -out=tfplan
working-directory: ${{ env.TF_DIR }}
continue-on-error: true

- name: Terraform Apply
id: apply
run: terraform apply -var-file='${{ env.TF_VARS_FILE }}' -auto-approve
working-directory: ${{ env.TF_DIR }}
continue-on-error: true
26 changes: 26 additions & 0 deletions .github/workflows/slack_notification.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Slack Deployment Notification

on:
workflow_run:
workflows:
- "Terraform Infrastructure Change Management Pipeline with GitHub Actions"
types:
- completed

jobs:
notify:
runs-on: ubuntu-latest
if: >
github.event.workflow_run.conclusion == 'success' ||
github.event.workflow_run.conclusion == 'failure'
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Slack Notification
uses: 8398a7/action-slack@v3
with:
status: ${{ github.event.workflow_run.conclusion }}
fields: repo,message,commit,author
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
Loading
Loading