-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* initial implementations of dynamodb table * initial implementation of rekognition and bucket setup with CR is tested to work * documentation updates * further documentation optimisations * further updates to bucket creation. PutBucketInventoryConfiguration is still having issues though * got inventory creation setup. implemented initial setup for lambda layers. basic helper classes added * used better deep copy of dictionary * implemented lambdalayers and bucket>sns>sqs for event queue * Lots of updates. Cleaned configurations. Added SNS. Fixed Layers. Simplified CRs. Integrated Layers usage in Features * logging bucket was having events listened to on it * small typos in update and delete for bucket linking * converted project to use nested stacks under 1 root stack to simplify deployment * updated readme with changes * Create build and test pipeline * Create dependency review * Create codeql * added name prefixes * moved feature list over to github issues * major updates. refactored out CRs for event linking bucket to sns to sqs * mass refactor. feature methods now use state machine. repo structures and names have changed. more standardised * bux gixes. got everything working with the state machine and helper libraries, permissions, logging, etc * Upgraded to latest TS and CDK. Added github actions for code scanning * upgraded CI workflow to use node v18 now * simplifying where codacy runs * limit CI to only run on any push to master
- Loading branch information
Showing
54 changed files
with
6,587 additions
and
5,680 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # This workflow uses actions that are not certified by GitHub. | ||
| # They are provided by a third-party and are governed by | ||
| # separate terms of service, privacy policy, and support | ||
| # documentation. | ||
|
|
||
| # This workflow checks out code, performs a Codacy security scan | ||
| # and integrates the results with the | ||
| # GitHub Advanced Security code scanning feature. For more information on | ||
| # the Codacy security scan action usage and parameters, see | ||
| # https://github.com/codacy/codacy-analysis-cli-action. | ||
| # For more information on Codacy Analysis CLI in general, see | ||
| # https://github.com/codacy/codacy-analysis-cli. | ||
|
|
||
| name: Codacy Security Scan | ||
|
|
||
| on: | ||
| push: | ||
| branches: [ "master" ] | ||
| pull_request: | ||
| # The branches below must be a subset of the branches above | ||
| branches: [ "master" ] | ||
| schedule: | ||
| - cron: '37 8 * * 4' | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| codacy-security-scan: | ||
| permissions: | ||
| contents: read # for actions/checkout to fetch code | ||
| security-events: write # for github/codeql-action/upload-sarif to upload SARIF results | ||
| actions: read # only required for a private repository by github/codeql-action/upload-sarif to get the Action run status | ||
| name: Codacy Security Scan | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| # Checkout the repository to the GitHub Actions runner | ||
| - name: Checkout code | ||
| uses: actions/checkout@v3 | ||
|
|
||
| # Execute Codacy Analysis CLI and generate a SARIF output with the security issues identified during the analysis | ||
| - name: Run Codacy Analysis CLI | ||
| uses: codacy/codacy-analysis-cli-action@d840f886c4bd4edc059706d09c6a1586111c540b | ||
| with: | ||
| # Check https://github.com/codacy/codacy-analysis-cli#project-token to get your project token from your Codacy repository | ||
| # You can also omit the token and run the tools that support default configurations | ||
| # project-token: ${{ secrets.CODACY_PROJECT_TOKEN }} | ||
| verbose: true | ||
| output: results.sarif | ||
| format: sarif | ||
| # Adjust severity of non-security issues | ||
| gh-code-scanning-compat: true | ||
| # Force 0 exit code to allow SARIF file generation | ||
| # This will handover control about PR rejection to the GitHub side | ||
| max-allowed-issues: 2147483647 | ||
|
|
||
| # Upload the SARIF file generated in the previous step | ||
| - name: Upload SARIF results file | ||
| uses: github/codeql-action/upload-sarif@v2 | ||
| with: | ||
| sarif_file: results.sarif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Dependency Review Action | ||
| # | ||
| # This Action will scan dependency manifest files that change as part of a Pull Request, surfacing known-vulnerable versions of the packages declared or updated in the PR. Once installed, if the workflow run is marked as required, PRs introducing known-vulnerable packages will be blocked from merging. | ||
| # | ||
| # Source repository: https://github.com/actions/dependency-review-action | ||
| # Public documentation: https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement | ||
| name: 'Dependency Review' | ||
| on: [pull_request] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| dependency-review: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: 'Checkout Repository' | ||
| uses: actions/checkout@v3 | ||
| - name: 'Dependency Review' | ||
| uses: actions/dependency-review-action@v1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # This is a basic workflow to help you get started with Actions | ||
|
|
||
| name: CI | ||
|
|
||
| # Controls when the workflow will run | ||
| on: | ||
| # Triggers the workflow on push or pull request events but only for the "master" branch | ||
| push: | ||
| branches: [ "master" ] | ||
|
|
||
| # Allows you to run this workflow manually from the Actions tab | ||
| workflow_dispatch: | ||
|
|
||
| # A workflow run is made up of one or more jobs that can run sequentially or in parallel | ||
| jobs: | ||
| # This workflow contains a single job called "build" | ||
| build: | ||
| # The type of runner that the job will run on | ||
| runs-on: ubuntu-latest | ||
|
|
||
| # Steps represent a sequence of tasks that will be executed as part of the job | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
|
|
||
| - name: Build and Compile | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: '18.x' | ||
| - run: npm ci | ||
| - run: npm run build --if-present | ||
| - run: npm test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| import { Construct } from "constructs"; | ||
| import { ServicePrincipals } from "cdk-constants"; | ||
| import { | ||
| aws_sqs as sqs, | ||
| aws_iam as iam, | ||
| aws_s3 as s3, | ||
| aws_sns as sns, | ||
| } from 'aws-cdk-lib' | ||
| import { FormatUtils } from "../../lib/utils/formatutils"; | ||
|
|
||
| export interface BucketEventHandlerProps { | ||
| buckets: Array<s3.IBucket> | ||
| eventTopicName: string | ||
|
|
||
| } | ||
|
|
||
| export class BucketEventHandler extends Construct{ | ||
|
|
||
| public readonly eventTopic: sns.Topic | ||
|
|
||
| constructor(scope:Construct, id:string, props: BucketEventHandlerProps){ | ||
| super(scope, id) | ||
|
|
||
| this.eventTopic = new sns.Topic(this, `BucketEventHandlerTopic`, { | ||
| displayName: `BucketEventHandlerTopic`, | ||
| topicName: props.eventTopicName | ||
| }) | ||
|
|
||
| // Add EventTopic Policy to Allow our buckets to send notifications to it | ||
| this.eventTopic.addToResourcePolicy( | ||
| new iam.PolicyStatement({ | ||
| principals:[ | ||
| new iam.ServicePrincipal(ServicePrincipals.S3) | ||
| ], | ||
| actions:[ | ||
| "sns:Publish", | ||
| ], | ||
| resources:[ | ||
| this.eventTopic.topicArn | ||
| ], | ||
| conditions:{ | ||
| "ArnLike": { | ||
| "aws:SourceArn": FormatUtils.convertBucketsToPolicyArns(props.buckets) | ||
| } | ||
| } | ||
| }) | ||
| ) | ||
|
|
||
| } | ||
| } |
96 changes: 96 additions & 0 deletions
96
archive/bucket-topic-event-linker/bucket-topic-event-linker.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| import { Construct } from "constructs"; | ||
|
|
||
| import { | ||
| aws_iam as iam, | ||
| aws_lambda as lambda, | ||
| aws_logs as logs, | ||
| custom_resources as cr, | ||
| aws_s3 as s3, | ||
| aws_sqs as sqs, | ||
| aws_sns as sns, | ||
| hashMapper | ||
| } from "aws-cdk-lib" | ||
| import { | ||
| Duration, | ||
| CustomResource | ||
| } from "aws-cdk-lib" | ||
| import * as path from 'path' | ||
| import * as crypto from 'crypto' | ||
| import { ServicePrincipals } from "cdk-constants"; | ||
| import { HashUtil } from "../../lib/utils/hashutil"; | ||
| import { Sns } from "aws-cdk-lib/aws-ses-actions"; | ||
| import { FormatUtils } from "../../lib/utils/formatutils"; | ||
| import { LayerTypes } from "../../lib/constructs/lambda-layers/lambda-layers"; | ||
|
|
||
|
|
||
|
|
||
| export interface BucketTopicEventLinkerProps { | ||
| buckets: Array<s3.IBucket> | ||
| topic: sns.Topic, | ||
| onLayerRequestListener: (layerTypes: Array<LayerTypes>) => Array<lambda.LayerVersion> | ||
| } | ||
|
|
||
| export class BucketTopicEventLinker extends Construct{ | ||
|
|
||
| constructor(scope: Construct, id: string, props: BucketTopicEventLinkerProps){ | ||
| super(scope, id) | ||
|
|
||
| //const hashCode = HashUtil.generateIDSafeHash(props.bucket.bucketArn + props.bucket.bucketName + props.topic.topicArn, 15) | ||
|
|
||
| const eventLinkingLambdaRole = new iam.Role(this, `CustomResourceRole`, { | ||
| roleName: `btel-lambda-service-role`, | ||
| description: "Assumed Role By btel-event-linking-function", | ||
| assumedBy: new iam.ServicePrincipal(ServicePrincipals.LAMBDA) | ||
| }) | ||
|
|
||
| const eventLinkingLambdaS3Policy = new iam.Policy(this, `CustomResourceRoleS3Policy`, { | ||
| policyName: `btel-lambda-s3-policy`, | ||
| roles: [ | ||
| eventLinkingLambdaRole | ||
| ], | ||
| statements: [ | ||
| new iam.PolicyStatement({ | ||
| actions:[ | ||
| 's3:PutBucketNotification', | ||
| 's3:GetBucketNotification' | ||
| ], | ||
| resources: FormatUtils.convertBucketsToArns(props.buckets) | ||
| }) | ||
| ] | ||
| }) | ||
|
|
||
| const eventLinkingLambda = new lambda.Function(this, `CustomResourceLambda`,{ | ||
| functionName: `btel-function`, | ||
| description: 'Event Linking For S3 Bucket Events To SNS', | ||
| runtime: lambda.Runtime.PYTHON_3_8, | ||
| layers: props.onLayerRequestListener([LayerTypes.COMMONLIBLAYER]), | ||
| handler: 'lambda_function.on_event', | ||
| code: lambda.Code.fromAsset(path.join(__dirname, './res')), | ||
| role: eventLinkingLambdaRole, | ||
| timeout: Duration.minutes(15) | ||
| }) | ||
|
|
||
|
|
||
| const eventLinkingCustomResourceProvider = new cr.Provider(this, `CustomResourceProvider`, { | ||
| onEventHandler: eventLinkingLambda, | ||
| logRetention: logs.RetentionDays.ONE_DAY, | ||
| }) | ||
|
|
||
|
|
||
| const eventLinkingCustomResource = new CustomResource(this, `CustomResource`, { | ||
| resourceType: `Custom::BucketTopic-EventLinker`, | ||
| serviceToken: eventLinkingCustomResourceProvider.serviceToken, | ||
| properties: { | ||
| "bucketArns": FormatUtils.convertBucketsToArns(props.buckets), | ||
| "bucketNames": FormatUtils.convertBucketsToNames(props.buckets), | ||
| "snsTopicArn": props.topic.topicArn, | ||
| } | ||
| }) | ||
|
|
||
| for(const bucket of props.buckets){ | ||
| eventLinkingCustomResource.node.addDependency(bucket) | ||
| } | ||
| eventLinkingCustomResource.node.addDependency(props.topic) | ||
|
|
||
| } | ||
| } |
Oops, something went wrong.