Skip to content

Commit

Permalink
Feat: Improve samples (#718)
Browse files Browse the repository at this point in the history
* Feat: Improve samples

**Why?**

These changes aim to improve the overall quality, maintainability, and
usability of the ADF sample templates, while also providing better
documentation and aligning with the latest AWS best practices.

- To adhere to best practices and improve maintainability of the code base.
- To enhance security and observability by enabling VPC Flow Logs.
- To address code style issues and enforce consistent formatting.

**What?**

- Specified explicit files to include in the CodeBuild output artifacts.
- Improvements to the VPC example:
      - Added support for enabling VPC Flow Logs to S3 or CloudWatch Logs.
      - Refactored resource ordering and added conditions for better readability.
      - Exported additional VPC CIDR range output for convenience.
- Fixed minor documentation issues in sample guides.
- Update README files with additional details, prerequisites, and deployment
   instructions for various samples.
- Upgrade the `sample-fargate-node-app` to use the AWS public container registry
   (public.ecr.aws) instead.
- Refactor the `sample-ec2-with-codedeploy` sample to use AWS Launch Templates,
   a newer and recommended approach instead of Launch Configurations.
- Update `sample-ec2-with-codedeploy` scripts to install newer versions of
   Amazon Linux 2023, Java, and other dependencies. Also fixed the scripts to be
   compatible to the recommended IMDSv2 authenticated APIs.
- Miscellaneous improvements and bug fixes across various sample templates.

---

By submitting this pull request, I confirm that you can use, modify, copy, and
redistribute this contribution, under the terms of your choice.

* Fix markdown table in changelog
  • Loading branch information
sbkok committed May 13, 2024
1 parent 39179a7 commit cd6ece5
Show file tree
Hide file tree
Showing 47 changed files with 1,034 additions and 457 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ samconfig.toml
samconfig.yml
samconfig.yaml
pipeline.json
template-sam.yml
deploy.sh
Makefile.new

Expand Down Expand Up @@ -52,6 +51,7 @@ wheels/
.installed.cfg
*.egg
MANIFEST
node_modules

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
78 changes: 39 additions & 39 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,33 +69,33 @@ The parameters that are managed by ADF that got their path changed are:
For the __management account__, in the __AWS Organizations region__
(`us-east-1`, or `us-gov-west-1`):

| Old Parameter Path | New Parameter Path |
|------------------------------|-------------------------------------|
| `/adf_log_level` | `/adf/adf_log_level` |
| `/adf_version` | `/adf/adf_version` |
| `/bucket_name` | `/adf/bucket_name` |
| `/confit` | `/adf/config` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/deployment_account_region` | `/adf/deployment_account_region` |
| `/kms_arn` | `/adf/kms_arn` |
| `/notification_channel` | `/adf/notification_channel` |
| `/organization_id` | `/adf/organization_id` |
| `/protected` | `/adf/protected` |
| `/scp` | `/adf/scp` |
| `/shared_modules_bucket` | `/adf/shared_modules_bucket` |
| `/tagging-policy` | `/adf/tagging_policy` |
| `/target_regions` | `/adf/target_regions` |
| Old Parameter Path | New Parameter Path |
|------------------------------|----------------------------------|
| `/adf_log_level` | `/adf/adf_log_level` |
| `/adf_version` | `/adf/adf_version` |
| `/bucket_name` | `/adf/bucket_name` |
| `/confit` | `/adf/config` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/deployment_account_region` | `/adf/deployment_account_region` |
| `/kms_arn` | `/adf/kms_arn` |
| `/notification_channel` | `/adf/notification_channel` |
| `/organization_id` | `/adf/organization_id` |
| `/protected` | `/adf/protected` |
| `/scp` | `/adf/scp` |
| `/shared_modules_bucket` | `/adf/shared_modules_bucket` |
| `/tagging-policy` | `/adf/tagging_policy` |
| `/target_regions` | `/adf/target_regions` |

For the __management account__, in __other ADF regions__:

| Old Parameter Path | New Parameter Path |
|------------------------------|-------------------------------------|
| `/adf_version` | `/adf/adf_version` |
| `/bucket_name` | `/adf/bucket_name` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/kms_arn` | `/adf/kms_arn` |
| Old Parameter Path | New Parameter Path |
|------------------------------|----------------------------------|
| `/adf_version` | `/adf/adf_version` |
| `/bucket_name` | `/adf/bucket_name` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/kms_arn` | `/adf/kms_arn` |

For the __deployment account__, in __the deployment region__:

Expand All @@ -114,24 +114,24 @@ For the __deployment account__, in __the deployment region__:

For the __deployment account__, in __other ADF regions__:

| Old Parameter Path | New Parameter Path |
|------------------------------|-------------------------------------|
| `/adf_log_level` | `/adf/adf_log_level` |
| `/adf_version` | `/adf/adf_version` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_bucket` | `/adf/deployment_account_bucket` |
| `/master_account_id` | `/adf/management_account_id` |
| `/notification_endpoint` | `/adf/notification_endpoint` |
| `/notification_type` | `/adf/notification_type` |
| `/organization_id` | `/adf/organization_id` |
| Old Parameter Path | New Parameter Path |
|------------------------------|----------------------------------|
| `/adf_log_level` | `/adf/adf_log_level` |
| `/adf_version` | `/adf/adf_version` |
| `/cross_account_access_role` | `/adf/cross_account_access_role` |
| `/deployment_account_bucket` | `/adf/deployment_account_bucket` |
| `/master_account_id` | `/adf/management_account_id` |
| `/notification_endpoint` | `/adf/notification_endpoint` |
| `/notification_type` | `/adf/notification_type` |
| `/organization_id` | `/adf/organization_id` |

For a __target account__, in __each ADF region__:

| Old Parameter Path | New Parameter Path |
|------------------------------|-------------------------------------|
| `/bucket_name` | `/adf/bucket_name` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/kms_arn` | `/adf/kms_arn` |
| Old Parameter Path | New Parameter Path |
|--------------------------|------------------------------|
| `/bucket_name` | `/adf/bucket_name` |
| `/deployment_account_id` | `/adf/deployment_account_id` |
| `/kms_arn` | `/adf/kms_arn` |

#### AWS CodeStar Connections OAuth Token support dropped

Expand Down
4 changes: 2 additions & 2 deletions docs/samples-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,8 +307,8 @@ URL on the *ECS Cluster* AWS CloudFormation stack within the target accounts.

![cfn-output](./images/cfn-output.png)

Accessing the *ExternalUrl* output in your web browser, you should be greeted
with the application running inside AWS Fargate.
Accessing the *LoadBalancerExternalUrl* output in your web browser, you should
be greeted with the application running inside AWS Fargate.

For more samples, please see the other pipeline/resource definitions in the
`samples` folder, or check out the numerous CloudFormation resource available
Expand Down
2 changes: 1 addition & 1 deletion samples/sample-cdk-app/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Sample CDK Application to showcase ADF Pipelines

This pipeline is expecting *(in the example case)* a AWS CodeCommit repository
This pipeline is expecting *(in the example case)* an AWS CodeCommit repository
on the account `111111111111` in your main deployment region named
*sample-cdk-application*.

Expand Down
5 changes: 4 additions & 1 deletion samples/sample-cdk-app/buildspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,7 @@ phases:
- cdk synth > template.yml

artifacts:
files: '**/*'
files:
- 'template.yml'
- 'params/*.json'
- 'params/*.yml'
23 changes: 23 additions & 0 deletions samples/sample-cdk-bootstrap/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Sample CDK Bootstrap pipeline

This pipeline is expecting *(in the example case)* an AWS CodeCommit repository
on the account `111111111111` in your main deployment region named
*sample-cdk-bootstrap*.

## Deployment Map example

```yaml
- name: sample-cdk-bootstrap
default_providers:
source:
provider: codecommit
properties:
account_id: 111111111111
build:
provider: codebuild
properties:
image: "STANDARD_7_0"
targets:
- /banking/testing
- /banking/production
```
22 changes: 22 additions & 0 deletions samples/sample-cdk-bootstrap/buildspec.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Copyright Amazon.com Inc. or its affiliates.
# SPDX-License-Identifier: Apache-2.0

version: 0.2

phases:
install:
runtime-versions:
python: 3.12
nodejs: 20
commands:
- aws s3 cp s3://$S3_BUCKET_NAME/adf-build/ adf-build/ --recursive --quiet
- pip install -r adf-build/requirements.txt -q
- python adf-build/generate_params.py

build:
commands:
- npm install aws-cdk -g
- cdk bootstrap --show-template > template.yml

artifacts:
files: '**/*'
62 changes: 62 additions & 0 deletions samples/sample-cdk-bootstrap/params/global.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Copyright Amazon.com Inc. or its affiliates.
# SPDX-License-Identifier: Apache-2.0
#
# Parameters of the CDK Bootstrap stack:
# TrustedAccounts:
# Description: List of AWS accounts that are trusted to publish assets and deploy stacks to this environment
# Default: ""
# Type: CommaDelimitedList
# TrustedAccountsForLookup:
# Description: List of AWS accounts that are trusted to look up values in this environment
# Default: ""
# Type: CommaDelimitedList
# CloudFormationExecutionPolicies:
# Description: List of the ManagedPolicy ARN(s) to attach to the CloudFormation deployment role
# Default: ""
# Type: CommaDelimitedList
# FileAssetsBucketName:
# Description: The name of the S3 bucket used for file assets
# Default: ""
# Type: String
# FileAssetsBucketKmsKeyId:
# Description: Empty to create a new key (default), 'AWS_MANAGED_KEY' to use a managed S3 key, or the ID/ARN of an existing key.
# Default: ""
# Type: String
# ContainerAssetsRepositoryName:
# Description: A user-provided custom name to use for the container assets ECR repository
# Default: ""
# Type: String
# Qualifier:
# Description: An identifier to distinguish multiple bootstrap stacks in the same environment
# Default: hnb659fds
# Type: String
# AllowedPattern: "[A-Za-z0-9_-]{1,10}"
# ConstraintDescription: Qualifier must be an alphanumeric identifier of at most 10 characters
# PublicAccessBlockConfiguration:
# Description: Whether or not to enable S3 Staging Bucket Public Access Block Configuration
# Default: "true"
# Type: String
# AllowedValues:
# - "true"
# - "false"
# InputPermissionsBoundary:
# Description: Whether or not to use either the CDK supplied or custom permissions boundary
# Default: ""
# Type: String
# UseExamplePermissionsBoundary:
# Default: "false"
# AllowedValues:
# - "true"
# - "false"
# Type: String
# BootstrapVariant:
# Type: String
# Default: "AWS CDK: Default Resources"

Parameters:
TrustedAccounts: 'resolve:/adf/deployment_account_id'
TrustedAccountsForLookup: 'resolve:/adf/deployment_account_id'

Tags:
Repository: sample-codebuild-vpc-repo
App: Sample CodeBuild VPC application
6 changes: 3 additions & 3 deletions samples/sample-ec2-java-app-codedeploy/pom.xml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<!-- Copyright Amazon.com Inc. or its affiliates. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<!-- Copyright Amazon.com Inc. or its affiliates. -->
<!-- SPDX-License-Identifier: Apache-2.0 -->

<groupId>org.springframework</groupId>
<artifactId>gs-spring-boot</artifactId>
<version>0.1.0</version>
Expand Down
10 changes: 4 additions & 6 deletions samples/sample-ec2-with-codedeploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,18 @@ is aimed at showcasing how to deploy a basic Spring Boot application with
[AWS CodeDeploy](https://docs.aws.amazon.com/codedeploy/latest/userguide/welcome.html)
via ADF.

This stack assumes an Amazon EC2
[Key Pair](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)
has been created in the target accounts.

This stack is a generic stack for applications that run on Amazon EC2.
This stack could be extended and used as a base for all line of business type
applications that run Amazon EC2.

This stack also requires `sample-vpc` and `sample-iam` to be in deployed as it
imports resources directly from both of them.

## Deployment Map example
## Prerequisites

### This sample stack depends on resources in sample-iam and sample-vpc
This sample stack depends on resources in `sample-iam` and `sample-vpc`.

## Deployment Map example

```yaml
- name: sample-ec2-app-codedeploy
Expand Down
5 changes: 4 additions & 1 deletion samples/sample-ec2-with-codedeploy/buildspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,7 @@ phases:
- python adf-build/generate_params.py

artifacts:
files: '**/*'
files:
- 'template.yml'
- 'params/*.json'
- 'params/*.yml'
17 changes: 8 additions & 9 deletions samples/sample-ec2-with-codedeploy/params/global.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,11 @@
# SPDX-License-Identifier: Apache-2.0

Parameters:
Environment: testing
ApplicationName: sample
InstanceMaxSize: '3'
InstanceMinSize: '1'
ImageId: 'resolve:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
InstanceType: t3.micro
CodeDeployAgentInstallScript: 'upload:path:scripts/install-codedeploy.sh'
JavaInstallScript: 'upload:path:scripts/install-deps.sh'
KeyPair: some_key_pair
Environment: "testing"
ApplicationName: "sample"
InstanceMaxSize: "3"
InstanceMinSize: "1"
ImageId: "resolve:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64"
InstanceType: "t3.micro"
CodeDeployAgentInstallScript: "upload:path:scripts/install-codedeploy.sh"
JavaInstallScript: "upload:path:scripts/install-deps.sh"
37 changes: 8 additions & 29 deletions samples/sample-ec2-with-codedeploy/scripts/install-codedeploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,11 @@ set -xe

## Code Deploy Agent Bootstrap Script ##

exec > >(tee /var/log/user-data.log | logger -t user-data -s 2> /dev/console) 2>&1
exec > >(sudo tee /var/log/user-data.log | logger -t user-data -s 2> /dev/console) 2>&1
AUTOUPDATE=false

function installdep() {
echo "Installing dependencies..."
if [ ${PLAT} = "ubuntu" ]; then
apt-get -y update
# Satisfying even Ubuntu older versions.
Expand All @@ -19,6 +20,7 @@ function installdep() {
yum -y update
yum install -y aws-cli ruby jq
fi
echo "Done installing dependencies."
}

function platformize() {
Expand All @@ -36,39 +38,15 @@ function platformize() {
}

function execute() {
if [ ${PLAT} = "ubuntu" ]; then
cd /tmp/
wget https://aws-codedeploy-${REGION}.s3.${REGION}.amazonaws.com/latest/install
chmod +x ./install

if ./install auto; then
echo "Installation completed"
if ! ${AUTOUPDATE}; then
echo "Disabling Auto Update"
sed -i '/@reboot/d' /etc/cron.d/codedeploy-agent-update
chattr +i /etc/cron.d/codedeploy-agent-update
rm -f /tmp/install
fi
exit 0
else
echo "Installation script failed, please investigate"
rm -f /tmp/install
exit 1
fi

elif [ ${PLAT} = "amz" ]; then
if [[ "${PLAT}" = "ubuntu" ]] || [[ "${PLAT}" = "amz" ]]; then
echo "Downloading CodeDeploy Agent..."
cd /tmp/
wget https://aws-codedeploy-${REGION}.s3.${REGION}.amazonaws.com/latest/install
chmod +x ./install

echo "Installing CodeDeploy Agent..."
if ./install auto; then
echo "Installation completed"
if ! ${AUTOUPDATE}; then
echo "Disabling auto update"
sed -i '/@reboot/d' /etc/cron.d/codedeploy-agent-update
chattr +i /etc/cron.d/codedeploy-agent-update
rm -f /tmp/install
fi
exit 0
else
echo "Installation script failed, please investigate"
Expand All @@ -83,5 +61,6 @@ function execute() {

platformize
installdep
REGION=$(curl -s 169.254.169.254/latest/dynamic/instance-identity/document | jq -r ".region")
export TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
export REGION=$(curl -H "X-aws-ec2-metadata-token: ${TOKEN}" -s http://169.254.169.254/latest/dynamic/instance-identity/document | jq -r ".region")
execute
Loading

0 comments on commit cd6ece5

Please sign in to comment.