Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: add GHA for publishing CF templates to S3 #2800

Merged
merged 23 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
275984c
feat(cli): add cloudformation templates for fargate and lambda
InesNi May 23, 2024
1c8b643
ci: add gha for publishing cf templates
InesNi May 29, 2024
064dc01
refactor: fix path
InesNi May 29, 2024
a6b3f82
refactor: try without install
InesNi May 29, 2024
2599f0c
refactor: remove aws cli install step
InesNi May 29, 2024
72bf786
refactor: adjust trigger
InesNi May 29, 2024
30abe0f
refactor: fix templates
InesNi May 30, 2024
86b9b21
refactor: allow any principal to assume the role
InesNi May 30, 2024
5637315
refactor: add workflow_dispatch to GHA to test changes
InesNi May 30, 2024
ca415d7
refactor: fix lambda template
InesNi May 30, 2024
98233e7
refactor: change name to match
InesNi May 31, 2024
99149b1
refactor: set the Principal of the IAM roles to the account
InesNi Jun 3, 2024
19cdc44
refactor: fix role names in output
InesNi Jun 6, 2024
791a9f5
feat: updating templates with parameters for easier setup
InesNi Jun 11, 2024
14da7de
refactor: fix param name
InesNi Jun 11, 2024
5709164
refactor: use new role and bucket
InesNi Jun 12, 2024
c5c2f89
refactor: set public read explicitly
InesNi Jun 12, 2024
d9d2b1a
refactor: try updating individual files
InesNi Jun 12, 2024
a06e4ad
refactor: make parameters more descriptive
InesNi Jun 17, 2024
f014286
refactor(cf): order parameters correctly
InesNi Jun 17, 2024
dff5ce7
refactor: remove GitHub OIDC configuration
InesNi Jun 18, 2024
74e7827
refactor(ci): publish templates on releases
InesNi Jun 21, 2024
d540e95
refactor: trigger after tests
InesNi Jun 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion .github/workflows/npm-publish-all-packages-canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,15 @@ jobs:
ARTILLERY_CLOUD_API_KEY_TEST: ${{ secrets.ARTILLERY_CLOUD_API_KEY_TEST }}
DD_TESTS_API_KEY: ${{ secrets.DD_TESTS_API_KEY }}
DD_TESTS_APP_KEY: ${{ secrets.DD_TESTS_APP_KEY }}
AWS_TEST_EXECUTION_ROLE_ARN_TEST5: ${{ secrets.AWS_TEST_EXECUTION_ROLE_ARN_TEST5 }}
AWS_TEST_EXECUTION_ROLE_ARN_TEST5: ${{ secrets.AWS_TEST_EXECUTION_ROLE_ARN_TEST5 }}

publish-cloudformation-templates-canary-to-s3:
uses: ./.github/workflows/s3-publish-cf-templates.yml
needs: run-distributed-tests
with:
canary: true
permissions:
contents: read
id-token: write
secrets:
AWS_ASSET_UPLOAD_ROLE_ARN: ${{ secrets.AWS_ASSET_UPLOAD_ROLE_ARN }}
13 changes: 12 additions & 1 deletion .github/workflows/npm-publish-all-packages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,4 +104,15 @@ jobs:
ARTILLERY_CLOUD_API_KEY_TEST: ${{ secrets.ARTILLERY_CLOUD_API_KEY_TEST }}
DD_TESTS_API_KEY: ${{ secrets.DD_TESTS_API_KEY }}
DD_TESTS_APP_KEY: ${{ secrets.DD_TESTS_APP_KEY }}
AWS_TEST_EXECUTION_ROLE_ARN_TEST5: ${{ secrets.AWS_TEST_EXECUTION_ROLE_ARN_TEST5 }}
AWS_TEST_EXECUTION_ROLE_ARN_TEST5: ${{ secrets.AWS_TEST_EXECUTION_ROLE_ARN_TEST5 }}

publish-cloudformation-templates-to-s3:
uses: ./.github/workflows/s3-publish-cf-templates.yml
needs: run-distributed-tests
with:
canary: true
permissions:
contents: read
id-token: write
secrets:
AWS_ASSET_UPLOAD_ROLE_ARN: ${{ secrets.AWS_ASSET_UPLOAD_ROLE_ARN }}
42 changes: 39 additions & 3 deletions .github/workflows/s3-publish-cf-templates.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,51 @@
name: Publish CloudFormation templates to AWS S3

on:
workflow_call:
inputs:
canary:
type: boolean
default: false
description: 'Whether to deploy the canary versions of the templates'
secrets:
AWS_ASSET_UPLOAD_ROLE_ARN:
description: 'ARN of the IAM role to assume to upload assets to S3'
required: true

workflow_dispatch:
inputs:
canary:
type: boolean
default: false
description: 'Whether to deploy the canary versions of the templates'

env:
CF_LAMBDA_TEMPLATE: ${{ inputs.canary && 'aws-iam-lambda-cf-template-canary.yml' || 'aws-iam-lambda-cf-template.yml' }}
CF_FARGATE_TEMPLATE: ${{ inputs.canary && 'aws-iam-fargate-cf-template-canary.yml' || 'aws-iam-fargate-cf-template.yml' }}

jobs:
put-cloudformation-templates:
runs-on: ubuntu-latest

permissions:
id-token: write
contents: read

steps:
- name: Checkout code
uses: actions/checkout@v2

- name: "Hello world"
run: echo "Hello world"

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v2
env:
SHOW_STACK_TRACE: true
with:
aws-region: us-east-1
role-to-assume: ${{ secrets.AWS_ASSET_UPLOAD_ROLE_ARN }}
role-session-name: OIDCSession
mask-aws-account-id: true

- name: Update IAM CloudFormation templates
run: |
aws s3 cp --acl public-read ./packages/artillery/lib/platform/aws/iam-cf-templates/aws-iam-fargate-cf-template.yml s3://artilleryio-cf-templates/${{ env.CF_FARGATE_TEMPLATE }}
aws s3 cp --acl public-read ./packages/artillery/lib/platform/aws/iam-cf-templates/aws-iam-lambda-cf-template.yml s3://artilleryio-cf-templates/${{ env.CF_LAMBDA_TEMPLATE }}
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
AWSTemplateFormatVersion: "2010-09-09"
Description: "Template to create an IAM Role with an attached policy that provides all necessary permissions for Artillery.io to run distributed tests on AWS Fargate.
By default the IAM role is configured to trust your AWS account, meaning it will allow any IAM User, Role or service from your account to assume it. You can restrict the role to allow only by a specific IAM user or role to assume it by filling out the appropriate parameter value below."

Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
- Label:
default: "Restrict to specific IAM User (optional)"
Parameters:
- User
- Label:
default: "Restrict to specific IAM Role (optional)"
Parameters:
- Role
ParameterLabels:
User:
default: "IAM user name or ARN"
Role:
default: "IAM role name or ARN"

Parameters:

User:
Type: String
Default: ""
Description: Use when you want to allow the created role to be assumed only by a specific IAM user (by default any user, role or service from your account will be allowed to assume it). Provide the user name or ARN.

Role:
Type: String
Default: ""
Description: Use when you want to allow the created role to be assumed only by a specific IAM role (by default any user, role or service from your account will be allowed to assume it). Provide the role name or ARN.

Conditions:
ShouldTrustAccount:
!And
- !Equals [!Ref User, ""]
- !Equals [!Ref Role, ""]
ShouldTrustUser:
!Not [!Equals [!Ref User, ""]]
IsUserArn:
!Equals [!Select [0, !Split [":", !Ref User]], "arn"]
ShouldTrustRole:
!Not [!Equals [!Ref Role, ""]]
IsRoleArn:
!Equals [!Select [0, !Split [":", !Ref Role]], "arn"]


Resources:
ArtilleryDistributedTestingFargateRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: "ArtilleryDistributedTestingFargateRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
AWS: [
!If [ShouldTrustAccount, !Ref "AWS::AccountId", !Ref "AWS::NoValue"],
!If [ShouldTrustUser, !If [IsUserArn, !Ref User, !Sub "arn:aws:iam::${AWS::AccountId}:user/${User}"], !Ref "AWS::NoValue"],
!If [ShouldTrustRole, !If [IsRoleArn, !Ref Role, !Sub "arn:aws:iam::${AWS::AccountId}:role/${Role}"], !Ref "AWS::NoValue"]
]
Action: [
"sts:AssumeRole"
]

Path: "/"
Policies:
- PolicyName: "ArtilleryDistributedTestingFargatePolicy"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: "CreateOrGetECSRole"
Effect: "Allow"
Action:
- "iam:CreateRole"
- "iam:GetRole"
- "iam:AttachRolePolicy"
- "iam:PassRole"
Resource:
Fn::Sub: "arn:aws:iam::${AWS::AccountId}:role/artilleryio-ecs-worker-role"
- Sid: "CreateECSPolicy"
Effect: "Allow"
Action:
- "iam:CreatePolicy"
Resource:
Fn::Sub: "arn:aws:iam::${AWS::AccountId}:policy/artilleryio-ecs-worker-policy"
- Effect: "Allow"
Action:
- "iam:CreateServiceLinkedRole"
Resource:
- "arn:aws:iam::*:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS*"
Condition:
StringLike:
iam:AWSServiceName: "ecs.amazonaws.com"
- Effect: "Allow"
Action:
- "iam:PassRole"
Resource:
- Fn::Sub: "arn:aws:iam::${AWS::AccountId}:role/artilleryio-ecs-worker-role"
- Sid: "SQSPermissions"
Effect: "Allow"
Action:
- "sqs:*"
Resource:
Fn::Sub: "arn:aws:sqs:*:${AWS::AccountId}:artilleryio*"
- Sid: "SQSListQueues"
Effect: "Allow"
Action:
- "sqs:ListQueues"
Resource: "*"
- Sid: "ECSPermissionsGeneral"
Effect: "Allow"
Action:
- "ecs:ListClusters"
- "ecs:CreateCluster"
- "ecs:RegisterTaskDefinition"
- "ecs:DeregisterTaskDefinition"
Resource: "*"
- Sid: "ECSPermissionsScopedToCluster"
Effect: "Allow"
Action:
- "ecs:DescribeClusters"
- "ecs:ListContainerInstances"
Resource:
Fn::Sub: "arn:aws:ecs:*:${AWS::AccountId}:cluster/*"
- Sid: "ECSPermissionsScopedWithCondition"
Effect: "Allow"
Action:
- "ecs:SubmitTaskStateChange"
- "ecs:DescribeTasks"
- "ecs:ListTasks"
- "ecs:ListTaskDefinitions"
- "ecs:DescribeTaskDefinition"
- "ecs:StartTask"
- "ecs:StopTask"
- "ecs:RunTask"
Condition:
ArnEquals:
ecs:cluster:
Fn::Sub: "arn:aws:ecs:*:${AWS::AccountId}:cluster/*"
Resource: "*"
- Sid: "S3Permissions"
Effect: "Allow"
Action:
- "s3:CreateBucket"
- "s3:DeleteObject"
- "s3:GetObject"
- "s3:GetObjectAcl"
- "s3:GetObjectTagging"
- "s3:GetObjectVersion"
- "s3:PutObject"
- "s3:PutObjectAcl"
- "s3:ListBucket"
- "s3:GetBucketLocation"
- "s3:GetBucketLogging"
- "s3:GetBucketPolicy"
- "s3:GetBucketTagging"
- "s3:PutBucketPolicy"
- "s3:PutBucketTagging"
- "s3:PutMetricsConfiguration"
- "s3:GetLifecycleConfiguration"
- "s3:PutLifecycleConfiguration"
Resource:
- "arn:aws:s3:::artilleryio-test-data-*"
- "arn:aws:s3:::artilleryio-test-data-*/*"
- Sid: "LogsPermissions"
Effect: "Allow"
Action:
- "logs:PutRetentionPolicy"
Resource:
- Fn::Sub: "arn:aws:logs:*:${AWS::AccountId}:log-group:artilleryio-log-group/*"
- Effect: "Allow"
Action:
- "secretsmanager:GetSecretValue"
Resource:
- Fn::Sub: "arn:aws:secretsmanager:*:${AWS::AccountId}:secret:artilleryio/*"
- Effect: "Allow"
Action:
- "ssm:PutParameter"
- "ssm:GetParameter"
- "ssm:GetParameters"
- "ssm:DeleteParameter"
- "ssm:DescribeParameters"
- "ssm:GetParametersByPath"
Resource:
- Fn::Sub: "arn:aws:ssm:us-east-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:us-east-2:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:us-west-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:us-west-2:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ca-central-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:eu-west-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:eu-west-2:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:eu-west-3:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:eu-central-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:eu-north-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-south-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-east-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-northeast-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-northeast-2:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-southeast-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:ap-southeast-2:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:me-south-1:${AWS::AccountId}:parameter/artilleryio/*"
- Fn::Sub: "arn:aws:ssm:sa-east-1:${AWS::AccountId}:parameter/artilleryio/*"
- Effect: "Allow"
Action:
- "ec2:DescribeRouteTables"
- "ec2:DescribeVpcs"
- "ec2:DescribeSubnets"
Resource: "*"

Outputs:
RoleArn:
Description: "ARN of the created IAM Role"
Value:
Fn::GetAtt:
- "ArtilleryDistributedTestingFargateRole"
- "Arn"
Loading
Loading