diff --git a/README.md b/README.md index 8e82ed8..3f735c9 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,15 @@ ## Git webhooks with AWS services ### Linking your Git repository to Amazon S3 and AWS services for continuous code integration, testing, and deployment -This Quick Start deploys HTTPS endpoints and AWS Lambda functions for implementing webhooks, to enable event-driven integration between Git services and Amazon Web Services (AWS) on the AWS Cloud. +This Quick Start deploys HTTPS endpoints. AWS Lambda functions and AWS CodeBuild project for implementing webhooks, to enable event-driven integration between Git services and Amazon Web Services (AWS) on the AWS Cloud. After you deploy the Quick Start, you can set up a webhook that uses the endpoints to create a bridge between your Git repository and AWS services like AWS CodePipeline and AWS CodeBuild. With this setup, builds and pipeline executions occur automatically when you commit your code to a Git repository, and your code can be continuously integrated, tested, built, and deployed on AWS with each change. The Quick Start includes an AWS CloudFormation template that automates the deployment. You can also use the AWS CloudFormation template as a starting point for your own implementation. -![Quick Start architecture for implementing webhooks on AWS](https://d0.awsstatic.com/partner-network/QuickStart/datasheets/git-to-s3-webhooks-architecture-on-aws.png) + + +![Quick Start architecture for implementing webhooks on AWS](./docs/images/image3.png) For implementation details, deployment instructions, and customization options, see the [deployment guide](https://fwd.aws/QQBRr). diff --git a/docs/boilerplate b/docs/boilerplate index 7feaed1..6dad7ef 160000 --- a/docs/boilerplate +++ b/docs/boilerplate @@ -1 +1 @@ -Subproject commit 7feaed118458dcda1137226edb8139a142e8ecc2 +Subproject commit 6dad7efceb9734dadee99b88ea55b95aeb8afc18 diff --git a/docs/images/architecture_diagram.png b/docs/images/architecture_diagram.png index 769bf7f..ad91da3 100644 Binary files a/docs/images/architecture_diagram.png and b/docs/images/architecture_diagram.png differ diff --git a/docs/images/image3.png b/docs/images/image3.png index edc79c6..ad91da3 100644 Binary files a/docs/images/image3.png and b/docs/images/image3.png differ diff --git a/docs/images/image5.png b/docs/images/image5.png index df3d98e..4c2f9d5 100644 Binary files a/docs/images/image5.png and b/docs/images/image5.png differ diff --git a/docs/partner_editable/additional_info.adoc b/docs/partner_editable/additional_info.adoc index d724af6..33f0a5d 100644 --- a/docs/partner_editable/additional_info.adoc +++ b/docs/partner_editable/additional_info.adoc @@ -32,7 +32,7 @@ image:../images/image7.png[Figure 4,width=648,height=198] image:../images/image8.png[Figure 5,width=646,height=348] -*Figure 5: Checking for S3 bucket updates after a commit* +*Figure 4: Checking for S3 bucket updates after a commit* === Best Practices diff --git a/docs/partner_editable/architecture.adoc b/docs/partner_editable/architecture.adoc index ae226f8..3e55d5b 100644 --- a/docs/partner_editable/architecture.adoc +++ b/docs/partner_editable/architecture.adoc @@ -3,17 +3,21 @@ Deploying this Quick Start builds the following environment in the AWS Cloud. // Replace this example diagram with your own. Send us your source PowerPoint file. Be sure to follow our guidelines here : http://(we should include these points on our contributors giude) :xrefstyle: short [#architecture1] -.Quick Start architecture for {partner-product-short-name} on AWS +// .Quick Start architecture for {partner-product-short-name} on AWS [link=images/architecture_diagram.png] image::../images/image3.png[Architecture,width=100%,height=100%] -Figure 2: Webhook endpoint architecture on AWS + + +*Figure 2: Webhook endpoint architecture on AWS* + The Quick Start deployment sets up a serverless AWS Cloud environment that includes the following components. * An API Gateway endpoint to accept the webhook requests from Git. -* Lambda functions to connect to the Git service, either over SSH or through the Git service’s endpoint. These functions zip the code and upload it to Amazon S3. +* Lambda function that processes the input git webhook payload received by the API Gateway and submits a build on AWS CodeBuild Project. +* An AWS CodeBuild project to connect to the Git service, either over SSH or through the Git service’s endpoint. This AWS CodeBuild project will zip the code and upload it to Amazon S3. -*Important* The Lambda functions that are deployed by this Quick Start must be able to communicate with your Git repository. For example, you can use a SaaS-based Git service that the Lambda service can reach through the internet. +*Important* The AWS CodeBuild project that is deployed by this Quick Start must be able to communicate with your Git repository. For example, you can use a SaaS-based Git service that the AWS CodeBuild can reach through the internet. * An AWS KMS key to encrypt the SSH private key used to connect to the repository over SSH. * Two S3 buckets: One bucket stores the zipped contents of your Git repository, and the second bucket stores the AWS KMS-encrypted SSH private keys that are generated during stack creation. Note that the first bucket has versioning enabled, and all previous versions are retained indefinitely. If you’d like to manage the retention period for old versions, follow the instructions in the http://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html[Amazon S3 documentation]. diff --git a/docs/partner_editable/deploy_steps.adoc b/docs/partner_editable/deploy_steps.adoc index 42c5a48..72efed0 100644 --- a/docs/partner_editable/deploy_steps.adoc +++ b/docs/partner_editable/deploy_steps.adoc @@ -30,10 +30,11 @@ After you have successfully deployed the Quick Start, you can configure the serv image:../images/image5.png[Figure 4,width=701,height=222] -Figure 4: Outputs tab after deployment + +*Figure 3: Outputs tab after deployment* + * *GitPullWebHookApi* is the webhook endpoint to use if you opt for the Git pull method described in the link:#webhook-endpoints[Webhook Endpoints] section of this guide. -* *ZipDownloadWebHookApi* is the webhook endpoint to use if you opt for the zip download method described in the link:#webhook-endpoints[Webhook Endpoints] section of this guide. * *PublicSSHKey* is the public SSH key that you use to connect to your repository if you’re using the Git pull endpoint. This key can be configured as a read-only machine user or as a deployment key in your Git service. The exact process to set up webhooks differs from service to service. For step-by-step instructions, consult your Git service’s documentation. diff --git a/docs/partner_editable/faq_troubleshooting.adoc b/docs/partner_editable/faq_troubleshooting.adoc index e1fd843..dacb8ae 100644 --- a/docs/partner_editable/faq_troubleshooting.adoc +++ b/docs/partner_editable/faq_troubleshooting.adoc @@ -17,6 +17,12 @@ If your commits are not being pushed through to Amazon S3, check the following: * In your Git service’s webhooks configuration, check that your configured security parameters and the endpoint are correct. Consult the Git service documentation for detailed guidance on configuration. * Check the Lambda logs for errors. These are stored in Amazon CloudWatch Logs. To access the logs, open the endpoint’s Lambda function in the AWS console, navigate to the *Monitoring* tab, and then choose *View logs in CloudWatch*. +For additional information, see https://docs.aws.amazon.com/lambda/latest/dg/lambda-troubleshooting.html[Troubleshooting issues in AWS Lambda^] on the AWS website. + +* Check the AWS CodeBuild project logs for errors. These are stored in Amazon CloudWatch Logs. To access the logs, open the endpoint’s CodeBuild Build Project in the AWS console, select a *Build run* and click on the hyperlink and navigate to the *Build logs* tab to see the build execution logs. + +For additional information, see https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html[Troubleshooting AWS CodeBuild^] on the AWS website. + == Additional Resources === AWS services diff --git a/docs/partner_editable/licenses.adoc b/docs/partner_editable/licenses.adoc index 5549497..f88ecc0 100644 --- a/docs/partner_editable/licenses.adoc +++ b/docs/partner_editable/licenses.adoc @@ -1 +1 @@ -The Quick Start provides an Amazon API Gateway endpoint and several Lambda functions to handle the download, zipping, and deployment of code to Amazon S3. AWS CodePipeline carries a cost for each active pipeline; see https://aws.amazon.com/codepipeline/pricing/[AWS CodePipeline pricing]. Depending on your configuration, the Quick Start may deploy an AWS Key Management Service (AWS KMS) key; for pricing, see https://aws.amazon.com/kms/pricing/[AWS Key Management Service pricing]. API Gateway, Amazon S3, and Lambda costs vary depending on how often you commit code to your repository; each commit triggers a request to the Lambda execution in API Gateway; for details, see the pricing pages for https://aws.amazon.com/api-gateway/pricing/[API Gateway], https://aws.amazon.com/s3/pricing/[Amazon S3], and https://aws.amazon.com/lambda/pricing/[Lambda]. +The Quick Start provides an Amazon API Gateway endpoint, several Lambda functions and an AWS CodeBuild project to handle the download, zipping, and deployment of code to Amazon S3. AWS CodePipeline carries a cost for each active pipeline; see https://aws.amazon.com/codepipeline/pricing/[AWS CodePipeline pricing]. Depending on your configuration, the Quick Start may deploy an AWS Key Management Service (AWS KMS) key; for pricing, see https://aws.amazon.com/kms/pricing/[AWS Key Management Service pricing]. API Gateway, Amazon S3, Lambda, and AWS CodeBuild costs vary depending on how often you commit code to your repository; each commit triggers a request to the Lambda execution in API Gateway; for details, see the pricing pages for https://aws.amazon.com/api-gateway/pricing/[API Gateway], https://aws.amazon.com/s3/pricing/[Amazon S3], https://aws.amazon.com/lambda/pricing/[Lambda] and https://aws.amazon.com/codebuild/pricing/[codebuild] . diff --git a/docs/partner_editable/product_description.adoc b/docs/partner_editable/product_description.adoc index 04d7ce2..c88dfc5 100644 --- a/docs/partner_editable/product_description.adoc +++ b/docs/partner_editable/product_description.adoc @@ -2,8 +2,9 @@ As Figure 1 illustrates, when code is pushed to your repository, the Git service sends an HTTPS POST to the endpoints configured by the Quick Start. The POST contains JSON data about the push operation, including the repository details that the Quick Start uses to fetch the latest version of your code. image:../images/image2.png[image,width=647,height=266] -Figure 1: Using webhooks for code commits -This Quick Start implements the required code to trigger a Lambda function that zips up the code in your repository and places the .zip file in Amazon S3. When this function is triggered by a Git webhook, it provides a convenient way to bridge Git services with AWS services like AWS CodePipeline and AWS CodeBuild, so that builds and pipeline executions occur automatically when you commit your code to a Git repository. Linking your existing code repositories to the AWS Cloud in this way enables your code to be continuously integrated, tested, built, and deployed on the AWS Cloud with each change. +*Figure 1: Using webhooks for code commits* -NOTE: The Lambda functions that are deployed by this Quick Start must be able to communicate with your Git repository. For example, you can use a SaaS-based Git service that Lambda can reach through the internet. +This Quick Start implements the required code to trigger a Lambda function and an AWS CodeBuild project working together that zips up the code in your repository and places the .zip file in Amazon S3. When this function is triggered by a Git webhook, it provides a convenient way to bridge Git services with AWS services like AWS CodePipeline and AWS CodeBuild, so that builds and pipeline executions occur automatically when you commit your code to a Git repository. Linking your existing code repositories to the AWS Cloud in this way enables your code to be continuously integrated, tested, built, and deployed on the AWS Cloud with each change. + +NOTE: The AWS CodeBuild Project deployed by this Quick Start must be able to communicate with your Git repository. For example, you can use a SaaS-based Git service that AWS CodeBuild can reach through the internet. diff --git a/templates/git2s3.template.yaml b/templates/git2s3.template.yaml index 07e7096..baeb0f6 100644 --- a/templates/git2s3.template.yaml +++ b/templates/git2s3.template.yaml @@ -38,6 +38,8 @@ Metadata: default: Quick Start S3 bucket region QSS3KeyPrefix: default: Quick Start S3 Key Prefix + ExcludeGit: + default: Exclude Git Parameters: AllowedIps: Description: gitpull method only. Comma seperated list of IP CIDR blocks for source