diff --git a/tuts/018-ecs-ec2/README.md b/tuts/018-ecs-ec2/README.md new file mode 100644 index 00000000..676d1e2e --- /dev/null +++ b/tuts/018-ecs-ec2/README.md @@ -0,0 +1,5 @@ +# Amazon Elastic Container Service EC2 getting started + +This tutorial demonstrates how to get started with Amazon Elastic Container Service (Amazon ECS) using EC2 instances as the compute platform. You'll learn how to create an ECS cluster, define a task definition, and run containerized applications on EC2 instances within your ECS cluster. + +You can either run the automated shell script (`ecs-ec2-getting-started.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`ecs-ec2-getting-started.md`) to understand each component and manually configure your ECS environment. diff --git a/tuts/018-ecs-ec2/ecs-ec2-getting-started.md b/tuts/018-ecs-ec2/ecs-ec2-getting-started.md new file mode 100644 index 00000000..f6ab43b1 --- /dev/null +++ b/tuts/018-ecs-ec2/ecs-ec2-getting-started.md @@ -0,0 +1,693 @@ +# Creating an Amazon ECS service for the EC2 launch type with the AWS CLI + +This tutorial guides you through setting up an Amazon Elastic Container Service (ECS) cluster using the EC2 launch type with the AWS CLI. You'll learn how to create a cluster, launch a container instance, register a task definition, and create a service to run and maintain your containerized application. + +## Topics + +* [Prerequisites](#prerequisites) +* [Create an ECS cluster](#create-an-ecs-cluster) +* [Launch a container instance](#launch-a-container-instance) +* [Register a task definition](#register-a-task-definition) +* [Create and monitor a service](#create-and-monitor-a-service) +* [Clean up resources](#clean-up-resources) +* [Next steps](#next-steps) + +## Prerequisites + +Before you begin this tutorial, make sure you have the following. + +1. The AWS CLI. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). You can also [use AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html), which includes the AWS CLI. +2. Configured your AWS CLI with appropriate credentials. Run `aws configure` if you haven't set up your credentials yet. +3. Basic familiarity with command line interfaces and containerization concepts. +4. An AWS account with permissions to create and manage ECS, EC2, and IAM resources. Your IAM user should have the [AmazonECS_FullAccess](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security-iam-awsmanpol.html#security-iam-awsmanpol-AmazonECS_FullAccess) policy attached. +5. A default VPC in your AWS account. If you don't have one, you can [create a VPC](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html#create-a-vpc) using the Amazon VPC console. + +Before you start, verify your AWS CLI configuration. + +``` +$ aws sts get-caller-identity +``` + +Output: + +``` +{ + "UserId": "AIDACKCEVSQ6C2EXAMPLE", + "Account": "123456789012", + "Arn": "arn:aws:iam::123456789012:user/username" +} +``` +After verifying your CLI configuration, check that you have a default VPC available. + +``` +$ aws ec2 describe-vpcs --filters "Name=is-default,Values=true" --query 'Vpcs[0].VpcId' --output text +``` + +Output: + +``` +vpc-abcd1234 +``` + +Let's get started with creating and managing Amazon ECS resources using the CLI. + +## Create an ECS cluster + +An ECS cluster is a logical grouping of tasks or services that run on container instances. In this section, you'll create a new cluster to host your containerized applications. + +**Create a new cluster** + +The following command creates a new ECS cluster named "MyCluster". + +``` +$ aws ecs create-cluster --cluster-name MyCluster +``` + +Output: + +``` +{ + "cluster": { + "clusterName": "MyCluster", + "status": "ACTIVE", + "clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster" + } +} +``` + +The response shows that your cluster has been created successfully and is in the "ACTIVE" status. You can now launch container instances and run tasks in this cluster. + +**Verify cluster creation** + +You can list all clusters in your account to verify that your cluster was created. + +``` +$ aws ecs list-clusters +``` + +Output: + +``` +{ + "clusterArns": [ + "arn:aws:ecs:us-east-1:123456789012:cluster/default", + "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster" + ] +} +``` + +The output shows both the default cluster (created automatically) and your new MyCluster. + +## Launch a container instance + +Container instances are EC2 instances that run the Amazon ECS container agent and have been registered into a cluster. In this section, you'll launch an EC2 instance using the ECS-optimized AMI. + +**Get the ECS-optimized AMI ID** + +First, retrieve the latest ECS-optimized Amazon Linux 2 AMI ID for your region. + +``` +$ aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended --query 'Parameters[0].Value' --output text | jq -r '.image_id' +``` + +Output: + +``` +ami-abcd1234 +``` + +This command uses AWS Systems Manager Parameter Store to get the latest ECS-optimized AMI ID. The AMI includes the ECS container agent and Docker runtime pre-installed. + +**Create a security group** + +Create a security group that allows SSH access for managing your container instance and HTTP access for the web server. + +``` +$ aws ec2 create-security-group --group-name ecs-tutorial-sg --description "ECS tutorial security group" +``` + +Output: + +``` +{ + "GroupId": "sg-abcd1234" +} +``` + +``` +$ aws ec2 authorize-security-group-ingress --group-id sg-abcd1234 --protocol tcp --port 80 --cidr 0.0.0.0/0 +``` + +Output: + +``` +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-efgh5678", + "GroupId": "sg-abcd1234", + "GroupOwnerId": "123456789012", + "IsEgress": false, + "IpProtocol": "tcp", + "FromPort": 80, + "ToPort": 80, + "CidrIpv4": "0.0.0.0/0" + } + ] +} +``` + +The security group now allows SSH access from the specified IP range and HTTP access from anywhere. In a production environment, you should restrict SSH access to your specific IP address and consider limiting HTTP access as needed. + +**Create a key pair** + +Create an EC2 key pair for SSH access to your container instance. + +``` +$ aws ec2 create-key-pair --key-name ecs-tutorial-key --query 'KeyMaterial' --output text > ecs-tutorial-key.pem +$ chmod 400 ecs-tutorial-key.pem +``` + +The private key is saved to your local machine with appropriate permissions for SSH access. + +**Launch the container instance** + +Launch an EC2 instance using the ECS-optimized AMI and configure it to join your cluster. + +``` +$ aws ec2 run-instances --image-id ami-abcd1234 --instance-type t3.micro --key-name ecs-tutorial-key --security-group-ids sg-abcd1234 --iam-instance-profile Name=ecsInstanceRole --user-data '#!/bin/bash +echo ECS_CLUSTER=MyCluster >> /etc/ecs/ecs.config' +``` + +Output: + +``` +{ + "Instances": [ + { + "InstanceId": "i-abcd1234", + "ImageId": "ami-abcd1234", + "State": { + "Code": 0, + "Name": "pending" + }, + "PrivateDnsName": "", + "PublicDnsName": "", + "StateReason": { + "Code": "pending", + "Message": "pending" + }, + "InstanceType": "t3.micro", + "KeyName": "ecs-tutorial-key", + "LaunchTime": "2025-01-13T10:30:00.000Z" + } + ] +} +``` + +The user data script configures the ECS agent to register the instance with your MyCluster. The instance uses the ecsInstanceRole IAM role, which provides the necessary permissions for the ECS agent. + +**Verify container instance registration** + +After a few minutes, check that your container instance has registered with the cluster. + +``` +$ aws ecs list-container-instances --cluster MyCluster +``` + +Output: + +``` +{ + "containerInstanceArns": [ + "arn:aws:ecs:us-east-1:123456789012:container-instance/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab" + ] +} +``` + +The output shows that one container instance has successfully registered with your cluster. + +**Get container instance details** + +You can get detailed information about your container instance, including available resources. + +``` +$ aws ecs describe-container-instances --cluster MyCluster --container-instances abcd1234-5678-90ab-cdef-1234567890ab +``` + +Output: + +``` +{ + "containerInstances": [ + { + "containerInstanceArn": "arn:aws:ecs:us-east-1:123456789012:container-instance/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab", + "ec2InstanceId": "i-abcd1234", + "status": "ACTIVE", + "runningTasksCount": 0, + "pendingTasksCount": 0, + "agentConnected": true, + "registeredResources": [ + { + "name": "CPU", + "type": "INTEGER", + "integerValue": 1024 + }, + { + "name": "MEMORY", + "type": "INTEGER", + "integerValue": 995 + } + ], + "remainingResources": [ + { + "name": "CPU", + "type": "INTEGER", + "integerValue": 1024 + }, + { + "name": "MEMORY", + "type": "INTEGER", + "integerValue": 995 + } + ] + } + ] +} +``` + +The output shows that your container instance is active and has 1024 CPU units and 995 MB of memory available for running tasks. + +## Register a task definition + +A task definition is a blueprint that describes how a container should run. In this section, you'll create a simple task definition that runs a busybox container. + +**Create a task definition file** + +Create a JSON file that defines your task. This example creates a simple task that runs an nginx web server container. + +``` +$ cat > nginx-task.json << 'EOF' +{ + "family": "nginx-task", + "containerDefinitions": [ + { + "name": "nginx", + "image": "public.ecr.aws/docker/library/nginx:latest", + "cpu": 256, + "memory": 512, + "essential": true, + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ] + } + ], + "requiresCompatibilities": ["EC2"], + "networkMode": "bridge" +} +EOF +``` + +This task definition specifies a container that runs an nginx web server, exposing port 80 for HTTP traffic. + +**Register the task definition** + +Register your task definition with Amazon ECS. + +``` +$ aws ecs register-task-definition --cli-input-json file://nginx-task.json +``` + +Output: + +``` +{ + "taskDefinition": { + "taskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:task-definition/nginx-task:1", + "family": "nginx-task", + "revision": 1, + "status": "ACTIVE", + "containerDefinitions": [ + { + "name": "nginx", + "image": "public.ecr.aws/docker/library/nginx:latest", + "cpu": 256, + "memory": 512, + "essential": true, + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ], + "environment": [], + "mountPoints": [], + "volumesFrom": [] + } + ], + "volumes": [], + "networkMode": "bridge", + "compatibilities": [ + "EC2" + ], + "requiresCompatibilities": [ + "EC2" + ] + } +} +``` + +The response shows that your task definition has been registered successfully with revision 1. You can now use this task definition to create a service. + +**List task definitions** + +You can list all task definitions in your account to verify registration. + +``` +$ aws ecs list-task-definitions +``` + +Output: + +``` +{ + "taskDefinitionArns": [ + "arn:aws:ecs:us-east-1:123456789012:task-definition/nginx-task:1" + ] +} +``` + +The output shows your newly registered task definition. + +## Create and monitor a service + +Now that you have a cluster, container instance, and task definition, you can create a service. An ECS service runs and maintains a desired number of tasks simultaneously and can replace unhealthy tasks automatically. + +**Create a service** + +Use the create-service command to create a service that maintains one running instance of your nginx task. + +``` +$ aws ecs create-service --cluster MyCluster --service-name nginx-service --task-definition nginx-task:1 --desired-count 1 +``` + +Output: + +``` +{ + "service": { + "serviceArn": "arn:aws:ecs:us-east-1:123456789012:service/MyCluster/nginx-service", + "serviceName": "nginx-service", + "clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster", + "taskDefinition": "arn:aws:ecs:us-east-1:123456789012:task-definition/nginx-task:1", + "desiredCount": 1, + "runningCount": 0, + "pendingCount": 0, + "launchType": "EC2", + "status": "ACTIVE", + "createdAt": "2025-01-13T10:45:00.000Z" + } +} +``` + +The response shows that your service has been created successfully. The service will automatically start one task and maintain it in a running state. + +**List services** + +Check the services running in your cluster. + +``` +$ aws ecs list-services --cluster MyCluster +``` + +Output: + +``` +{ + "serviceArns": [ + "arn:aws:ecs:us-east-1:123456789012:service/MyCluster/nginx-service" + ] +} +``` + +The output shows one service running in your cluster. + +**Get detailed service information** + +Get detailed information about your service, including the current status and task counts. + +``` +$ aws ecs describe-services --cluster MyCluster --services nginx-service +``` + +Output: + +``` +{ + "services": [ + { + "serviceArn": "arn:aws:ecs:us-east-1:123456789012:service/MyCluster/nginx-service", + "serviceName": "nginx-service", + "clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster", + "taskDefinition": "arn:aws:ecs:us-east-1:123456789012:task-definition/nginx-task:1", + "desiredCount": 1, + "runningCount": 1, + "pendingCount": 0, + "launchType": "EC2", + "status": "ACTIVE", + "createdAt": "2025-01-13T10:45:00.000Z", + "events": [ + { + "id": "abcd1234-5678-90ab-cdef-1234567890ab", + "createdAt": "2025-01-13T10:45:30.000Z", + "message": "(service nginx-service) has started 1 tasks: (task abcd1234-5678-90ab-cdef-1234567890ab)." + } + ] + } + ] +} +``` + +The output shows that your service is active with one running task. The events section provides information about recent service activities. + +**List tasks in the service** + +Check the tasks that are running as part of your service. + +``` +$ aws ecs list-tasks --cluster MyCluster --service-name nginx-service +``` + +Output: + +``` +{ + "taskArns": [ + "arn:aws:ecs:us-east-1:123456789012:task/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab" + ] +} +``` + +**Get detailed task information** + +Get detailed information about the task running in your service. + +``` +$ aws ecs describe-tasks --cluster MyCluster --tasks abcd1234-5678-90ab-cdef-1234567890ab +``` + +Output: + +``` +{ + "tasks": [ + { + "taskArn": "arn:aws:ecs:us-east-1:123456789012:task/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab", + "clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster", + "taskDefinitionArn": "arn:aws:ecs:us-east-1:123456789012:task-definition/nginx-task:1", + "containerInstanceArn": "arn:aws:ecs:us-east-1:123456789012:container-instance/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab", + "lastStatus": "RUNNING", + "desiredStatus": "RUNNING", + "containers": [ + { + "containerArn": "arn:aws:ecs:us-east-1:123456789012:container/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab/abcd1234-5678-90ab-cdef-1234567890ab", + "taskArn": "arn:aws:ecs:us-east-1:123456789012:task/MyCluster/abcd1234-5678-90ab-cdef-1234567890ab", + "name": "nginx", + "lastStatus": "RUNNING", + "networkBindings": [ + { + "bindIP": "0.0.0.0", + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ] + } + ], + "createdAt": "2025-01-13T10:45:00.000Z", + "startedAt": "2025-01-13T10:45:30.000Z" + } + ] +} +``` + +The output shows that your task is running and the nginx container is bound to port 80 on the host. You can access the web server using the public IP address of your EC2 instance. + +**Test the web server** + +Get the public IP address of your container instance and test the nginx web server. + +``` +$ aws ec2 describe-instances --instance-ids i-abcd1234 --query 'Reservations[0].Instances[0].PublicIpAddress' --output text +``` + +Output: + +``` +203.0.113.25 +``` + +``` +$ curl http://203.0.113.25 +``` + +Output: + +``` + + + +Welcome to nginx! +... + + +

Welcome to nginx!

+

If you can see this page, the nginx web server is successfully installed and working.

+... + + +``` + +The nginx welcome page confirms that your service is running successfully and accessible from the internet. + +## Clean up resources + +To avoid incurring charges, clean up the resources you created in this tutorial. + +**Delete the service** + +First, update the service to have zero desired tasks, then delete the service. + +``` +$ aws ecs update-service --cluster MyCluster --service nginx-service --desired-count 0 +``` + +Output: + +``` +{ + "service": { + "serviceArn": "arn:aws:ecs:us-east-1:123456789012:service/MyCluster/nginx-service", + "serviceName": "nginx-service", + "desiredCount": 0, + "runningCount": 1, + "pendingCount": 0, + "status": "ACTIVE" + } +} +``` + +Wait for the running tasks to stop, then delete the service. + +``` +$ aws ecs delete-service --cluster MyCluster --service nginx-service +``` + +Output: + +``` +{ + "service": { + "serviceArn": "arn:aws:ecs:us-east-1:123456789012:service/MyCluster/nginx-service", + "serviceName": "nginx-service", + "status": "DRAINING" + } +} +``` + +**Terminate the EC2 instance** + +Terminate the container instance you created. + +``` +$ aws ec2 terminate-instances --instance-ids i-abcd1234 +``` + +Output: + +``` +{ + "TerminatingInstances": [ + { + "InstanceId": "i-abcd1234", + "CurrentState": { + "Code": 32, + "Name": "shutting-down" + }, + "PreviousState": { + "Code": 16, + "Name": "running" + } + } + ] +} +``` + +**Delete the security group and key pair** + +Clean up the security group and key pair you created. + +``` +$ aws ec2 delete-security-group --group-id sg-abcd1234 +$ aws ec2 delete-key-pair --key-name ecs-tutorial-key +$ rm ecs-tutorial-key.pem +``` + +**Delete the ECS cluster** + +Finally, delete the ECS cluster. + +``` +$ aws ecs delete-cluster --cluster MyCluster +``` + +Output: + +``` +{ + "cluster": { + "clusterArn": "arn:aws:ecs:us-east-1:123456789012:cluster/MyCluster", + "clusterName": "MyCluster", + "status": "INACTIVE" + } +} +``` + +All resources have been successfully cleaned up. + +## Next steps + +Now that you've learned how to create and manage Amazon ECS services with the EC2 launch type, you can explore more advanced features: + +* [Use Application Load Balancers](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.html) to distribute traffic across multiple tasks in your service +* [Configure auto scaling](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-auto-scaling.html) to automatically adjust the number of running tasks based on demand +* [Set up CloudWatch logging](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_awslogs.html) to collect and monitor logs from your containers +* [Use Amazon ECR](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECR_on_ECS.html) to store and manage your container images +* [Deploy multi-container applications](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) using more complex task definitions +* [Configure service discovery](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html) to enable services to find and communicate with each other diff --git a/tuts/018-ecs-ec2/ecs-ec2-getting-started.sh b/tuts/018-ecs-ec2/ecs-ec2-getting-started.sh new file mode 100755 index 00000000..81b1bf13 --- /dev/null +++ b/tuts/018-ecs-ec2/ecs-ec2-getting-started.sh @@ -0,0 +1,597 @@ +#!/bin/bash + +# ECS EC2 Launch Type Tutorial Script - UPDATED VERSION +# This script demonstrates creating an ECS cluster, launching a container instance, +# registering a task definition, and creating a service using the EC2 launch type. +# Updated to match the tutorial draft with nginx web server and service creation. +# +# FIXES APPLIED: +# - HIGH SEVERITY: Improved cleanup error handling with proper logging and retry logic +# - MEDIUM SEVERITY: Dynamic region detection for CloudWatch logs +# - LOW SEVERITY: Enhanced IAM role creation timing +# - UPDATED: Changed from sleep task to nginx web server with service + +set -e # Exit on any error + +# Configuration +SCRIPT_NAME="ecs-ec2-tutorial" +LOG_FILE="${SCRIPT_NAME}-$(date +%Y%m%d-%H%M%S).log" +CLUSTER_NAME="tutorial-cluster-$(openssl rand -hex 4)" +TASK_FAMILY="nginx-task-$(openssl rand -hex 4)" +SERVICE_NAME="nginx-service-$(openssl rand -hex 4)" +KEY_PAIR_NAME="ecs-tutorial-key-$(openssl rand -hex 4)" +SECURITY_GROUP_NAME="ecs-tutorial-sg-$(openssl rand -hex 4)" + +# Get current AWS region dynamically +AWS_REGION=$(aws configure get region || echo "us-east-1") + +# Resource tracking arrays +CREATED_RESOURCES=() +CLEANUP_ORDER=() + +# Logging function +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" +} + +# Error handling function +handle_error() { + local exit_code=$? + log "ERROR: Script failed with exit code $exit_code" + log "ERROR: Last command: $BASH_COMMAND" + + echo "" + echo "===========================================" + echo "ERROR OCCURRED - ATTEMPTING CLEANUP" + echo "===========================================" + echo "Resources created before error:" + for resource in "${CREATED_RESOURCES[@]}"; do + echo " - $resource" + done + + cleanup_resources + exit $exit_code +} + +# Set error trap +trap handle_error ERR + +# FIXED: Enhanced cleanup function with proper error handling and logging +cleanup_resources() { + log "Starting cleanup process..." + local cleanup_errors=0 + + # Delete service first (this will stop tasks automatically) + if [[ -n "${SERVICE_ARN:-}" ]]; then + log "Updating service to desired count 0: $SERVICE_NAME" + if ! aws ecs update-service --cluster "$CLUSTER_NAME" --service "$SERVICE_NAME" --desired-count 0 2>>"$LOG_FILE"; then + log "WARNING: Failed to update service desired count to 0" + ((cleanup_errors++)) + else + log "Waiting for service tasks to stop..." + sleep 30 # Give time for tasks to stop + fi + + log "Deleting service: $SERVICE_NAME" + if ! aws ecs delete-service --cluster "$CLUSTER_NAME" --service "$SERVICE_NAME" 2>>"$LOG_FILE"; then + log "WARNING: Failed to delete service $SERVICE_NAME" + ((cleanup_errors++)) + fi + fi + + # Stop and delete any remaining tasks + if [[ -n "${TASK_ARN:-}" ]]; then + log "Stopping task: $TASK_ARN" + if ! aws ecs stop-task --cluster "$CLUSTER_NAME" --task "$TASK_ARN" --reason "Tutorial cleanup" 2>>"$LOG_FILE"; then + log "WARNING: Failed to stop task $TASK_ARN" + ((cleanup_errors++)) + else + log "Waiting for task to stop..." + if ! aws ecs wait tasks-stopped --cluster "$CLUSTER_NAME" --tasks "$TASK_ARN" 2>>"$LOG_FILE"; then + log "WARNING: Task stop wait failed for $TASK_ARN" + ((cleanup_errors++)) + fi + fi + fi + + # Deregister task definition + if [[ -n "${TASK_DEFINITION_ARN:-}" ]]; then + log "Deregistering task definition: $TASK_DEFINITION_ARN" + if ! aws ecs deregister-task-definition --task-definition "$TASK_DEFINITION_ARN" 2>>"$LOG_FILE"; then + log "WARNING: Failed to deregister task definition $TASK_DEFINITION_ARN" + ((cleanup_errors++)) + fi + fi + + # Terminate EC2 instance + if [[ -n "${INSTANCE_ID:-}" ]]; then + log "Terminating EC2 instance: $INSTANCE_ID" + if ! aws ec2 terminate-instances --instance-ids "$INSTANCE_ID" 2>>"$LOG_FILE"; then + log "WARNING: Failed to terminate instance $INSTANCE_ID" + ((cleanup_errors++)) + else + log "Waiting for instance to terminate..." + if ! aws ec2 wait instance-terminated --instance-ids "$INSTANCE_ID" 2>>"$LOG_FILE"; then + log "WARNING: Instance termination wait failed for $INSTANCE_ID" + ((cleanup_errors++)) + fi + fi + fi + + # Delete security group with retry logic + if [[ -n "${SECURITY_GROUP_ID:-}" ]]; then + log "Deleting security group: $SECURITY_GROUP_ID" + local retry_count=0 + local max_retries=3 + + while [[ $retry_count -lt $max_retries ]]; do + if aws ec2 delete-security-group --group-id "$SECURITY_GROUP_ID" 2>>"$LOG_FILE"; then + log "Successfully deleted security group" + break + else + ((retry_count++)) + if [[ $retry_count -lt $max_retries ]]; then + log "Retry $retry_count/$max_retries: Waiting 10 seconds before retrying security group deletion..." + sleep 10 + else + log "ERROR: Failed to delete security group after $max_retries attempts" + ((cleanup_errors++)) + fi + fi + done + fi + + # Delete key pair + if [[ -n "${KEY_PAIR_NAME:-}" ]]; then + log "Deleting key pair: $KEY_PAIR_NAME" + if ! aws ec2 delete-key-pair --key-name "$KEY_PAIR_NAME" 2>>"$LOG_FILE"; then + log "WARNING: Failed to delete key pair $KEY_PAIR_NAME" + ((cleanup_errors++)) + fi + rm -f "${KEY_PAIR_NAME}.pem" 2>>"$LOG_FILE" || log "WARNING: Failed to remove local key file" + fi + + # Delete ECS cluster + if [[ -n "${CLUSTER_NAME:-}" ]]; then + log "Deleting ECS cluster: $CLUSTER_NAME" + if ! aws ecs delete-cluster --cluster "$CLUSTER_NAME" 2>>"$LOG_FILE"; then + log "WARNING: Failed to delete cluster $CLUSTER_NAME" + ((cleanup_errors++)) + fi + fi + + if [[ $cleanup_errors -eq 0 ]]; then + log "Cleanup completed successfully" + else + log "Cleanup completed with $cleanup_errors warnings/errors. Check log file for details." + fi +} + +# Function to check prerequisites +check_prerequisites() { + log "Checking prerequisites..." + + # Check AWS CLI + if ! command -v aws &> /dev/null; then + log "ERROR: AWS CLI is not installed" + exit 1 + fi + + # Check AWS credentials + if ! aws sts get-caller-identity &> /dev/null; then + log "ERROR: AWS credentials not configured" + exit 1 + fi + + # Get caller identity + CALLER_IDENTITY=$(aws sts get-caller-identity --output text --query 'Account') + log "AWS Account: $CALLER_IDENTITY" + log "AWS Region: $AWS_REGION" + + # Check for default VPC + DEFAULT_VPC=$(aws ec2 describe-vpcs --filters "Name=is-default,Values=true" --query 'Vpcs[0].VpcId' --output text) + if [[ "$DEFAULT_VPC" == "None" ]]; then + log "ERROR: No default VPC found. Please create a VPC first." + exit 1 + fi + log "Using default VPC: $DEFAULT_VPC" + + # Get default subnet + DEFAULT_SUBNET=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$DEFAULT_VPC" "Name=default-for-az,Values=true" --query 'Subnets[0].SubnetId' --output text) + if [[ "$DEFAULT_SUBNET" == "None" ]]; then + log "ERROR: No default subnet found" + exit 1 + fi + log "Using default subnet: $DEFAULT_SUBNET" + + log "Prerequisites check completed successfully" +} + +# Function to create ECS cluster +create_cluster() { + log "Creating ECS cluster: $CLUSTER_NAME" + + CLUSTER_ARN=$(aws ecs create-cluster --cluster-name "$CLUSTER_NAME" --query 'cluster.clusterArn' --output text) + + if [[ -z "$CLUSTER_ARN" ]]; then + log "ERROR: Failed to create cluster" + exit 1 + fi + + log "Created cluster: $CLUSTER_ARN" + CREATED_RESOURCES+=("ECS Cluster: $CLUSTER_NAME") +} + +# Function to create key pair +create_key_pair() { + log "Creating EC2 key pair: $KEY_PAIR_NAME" + + # FIXED: Set secure umask before key creation + umask 077 + aws ec2 create-key-pair --key-name "$KEY_PAIR_NAME" --query 'KeyMaterial' --output text > "${KEY_PAIR_NAME}.pem" + chmod 400 "${KEY_PAIR_NAME}.pem" + umask 022 # Reset umask + + log "Created key pair: $KEY_PAIR_NAME" + CREATED_RESOURCES+=("EC2 Key Pair: $KEY_PAIR_NAME") +} + +# Function to create security group +create_security_group() { + log "Creating security group: $SECURITY_GROUP_NAME" + + SECURITY_GROUP_ID=$(aws ec2 create-security-group \ + --group-name "$SECURITY_GROUP_NAME" \ + --description "ECS tutorial security group" \ + --vpc-id "$DEFAULT_VPC" \ + --query 'GroupId' --output text) + + if [[ -z "$SECURITY_GROUP_ID" ]]; then + log "ERROR: Failed to create security group" + exit 1 + fi + + # Add HTTP access rule for nginx web server + aws ec2 authorize-security-group-ingress \ + --group-id "$SECURITY_GROUP_ID" \ + --protocol tcp \ + --port 80 \ + --cidr "0.0.0.0/0" + + log "Created security group: $SECURITY_GROUP_ID" + log "Added HTTP access on port 80" + CREATED_RESOURCES+=("Security Group: $SECURITY_GROUP_ID") +} + +# Function to get ECS optimized AMI +get_ecs_ami() { + log "Getting ECS-optimized AMI ID..." + + ECS_AMI_ID=$(aws ssm get-parameters \ + --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended \ + --query 'Parameters[0].Value' --output text | jq -r '.image_id') + + if [[ -z "$ECS_AMI_ID" ]]; then + log "ERROR: Failed to get ECS-optimized AMI ID" + exit 1 + fi + + log "ECS-optimized AMI ID: $ECS_AMI_ID" +} + +# Function to create IAM role for ECS instance (if it doesn't exist) +ensure_ecs_instance_role() { + log "Checking for ecsInstanceRole..." + + if ! aws iam get-role --role-name ecsInstanceRole &> /dev/null; then + log "Creating ecsInstanceRole..." + + # Create trust policy + cat > ecs-instance-trust-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ec2.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EOF + + # Create role + aws iam create-role \ + --role-name ecsInstanceRole \ + --assume-role-policy-document file://ecs-instance-trust-policy.json + + # Attach managed policy + aws iam attach-role-policy \ + --role-name ecsInstanceRole \ + --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role + + # Create instance profile + aws iam create-instance-profile --instance-profile-name ecsInstanceRole + + # Add role to instance profile + aws iam add-role-to-instance-profile \ + --instance-profile-name ecsInstanceRole \ + --role-name ecsInstanceRole + + # FIXED: Enhanced wait for role to be ready + log "Waiting for IAM role to be ready..." + aws iam wait role-exists --role-name ecsInstanceRole + sleep 30 # Additional buffer for eventual consistency + + rm -f ecs-instance-trust-policy.json + log "Created ecsInstanceRole" + CREATED_RESOURCES+=("IAM Role: ecsInstanceRole") + else + log "ecsInstanceRole already exists" + fi +} + +# Function to launch container instance +launch_container_instance() { + log "Launching ECS container instance..." + + # Create user data script + cat > ecs-user-data.sh << EOF +#!/bin/bash +echo ECS_CLUSTER=$CLUSTER_NAME >> /etc/ecs/ecs.config +EOF + + INSTANCE_ID=$(aws ec2 run-instances \ + --image-id "$ECS_AMI_ID" \ + --instance-type t3.micro \ + --key-name "$KEY_PAIR_NAME" \ + --security-group-ids "$SECURITY_GROUP_ID" \ + --subnet-id "$DEFAULT_SUBNET" \ + --iam-instance-profile Name=ecsInstanceRole \ + --user-data file://ecs-user-data.sh \ + --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=ecs-tutorial-instance}]" \ + --query 'Instances[0].InstanceId' --output text) + + if [[ -z "$INSTANCE_ID" ]]; then + log "ERROR: Failed to launch EC2 instance" + exit 1 + fi + + log "Launched EC2 instance: $INSTANCE_ID" + CREATED_RESOURCES+=("EC2 Instance: $INSTANCE_ID") + + # Wait for instance to be running + log "Waiting for instance to be running..." + aws ec2 wait instance-running --instance-ids "$INSTANCE_ID" + + # Wait for ECS agent to register + log "Waiting for ECS agent to register with cluster..." + local max_attempts=30 + local attempt=0 + + while [[ $attempt -lt $max_attempts ]]; do + CONTAINER_INSTANCES=$(aws ecs list-container-instances --cluster "$CLUSTER_NAME" --query 'containerInstanceArns' --output text) + if [[ -n "$CONTAINER_INSTANCES" && "$CONTAINER_INSTANCES" != "None" ]]; then + log "Container instance registered successfully" + break + fi + + attempt=$((attempt + 1)) + log "Waiting for container instance registration... (attempt $attempt/$max_attempts)" + sleep 10 + done + + if [[ $attempt -eq $max_attempts ]]; then + log "ERROR: Container instance failed to register within expected time" + exit 1 + fi + + rm -f ecs-user-data.sh +} + +# Function to register task definition +register_task_definition() { + log "Creating task definition..." + + # Create nginx task definition JSON matching the tutorial + cat > task-definition.json << EOF +{ + "family": "$TASK_FAMILY", + "containerDefinitions": [ + { + "name": "nginx", + "image": "public.ecr.aws/docker/library/nginx:latest", + "cpu": 256, + "memory": 512, + "essential": true, + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ] + } + ], + "requiresCompatibilities": ["EC2"], + "networkMode": "bridge" +} +EOF + + # FIXED: Validate JSON before registration + if ! jq empty task-definition.json 2>/dev/null; then + log "ERROR: Invalid JSON in task definition" + exit 1 + fi + + TASK_DEFINITION_ARN=$(aws ecs register-task-definition \ + --cli-input-json file://task-definition.json \ + --query 'taskDefinition.taskDefinitionArn' --output text) + + if [[ -z "$TASK_DEFINITION_ARN" ]]; then + log "ERROR: Failed to register task definition" + exit 1 + fi + + log "Registered task definition: $TASK_DEFINITION_ARN" + CREATED_RESOURCES+=("Task Definition: $TASK_DEFINITION_ARN") + + rm -f task-definition.json +} + +# Function to create service +create_service() { + log "Creating ECS service..." + + SERVICE_ARN=$(aws ecs create-service \ + --cluster "$CLUSTER_NAME" \ + --service-name "$SERVICE_NAME" \ + --task-definition "$TASK_FAMILY" \ + --desired-count 1 \ + --query 'service.serviceArn' --output text) + + if [[ -z "$SERVICE_ARN" ]]; then + log "ERROR: Failed to create service" + exit 1 + fi + + log "Created service: $SERVICE_ARN" + CREATED_RESOURCES+=("ECS Service: $SERVICE_NAME") + + # Wait for service to be stable + log "Waiting for service to be stable..." + aws ecs wait services-stable --cluster "$CLUSTER_NAME" --services "$SERVICE_NAME" + + log "Service is now stable and running" + + # Get the task ARN for monitoring + TASK_ARN=$(aws ecs list-tasks --cluster "$CLUSTER_NAME" --service-name "$SERVICE_NAME" --query 'taskArns[0]' --output text) + if [[ -n "$TASK_ARN" && "$TASK_ARN" != "None" ]]; then + log "Service task: $TASK_ARN" + CREATED_RESOURCES+=("ECS Task: $TASK_ARN") + fi +} + +# Function to demonstrate monitoring and testing +demonstrate_monitoring() { + log "Demonstrating monitoring capabilities..." + + # List services + log "Listing services in cluster:" + aws ecs list-services --cluster "$CLUSTER_NAME" --output table + + # Describe service + log "Service details:" + aws ecs describe-services --cluster "$CLUSTER_NAME" --services "$SERVICE_NAME" --output table --query 'services[0].{ServiceName:serviceName,Status:status,RunningCount:runningCount,DesiredCount:desiredCount,TaskDefinition:taskDefinition}' + + # List tasks + log "Listing tasks in service:" + aws ecs list-tasks --cluster "$CLUSTER_NAME" --service-name "$SERVICE_NAME" --output table + + # Describe task + if [[ -n "$TASK_ARN" && "$TASK_ARN" != "None" ]]; then + log "Task details:" + aws ecs describe-tasks --cluster "$CLUSTER_NAME" --tasks "$TASK_ARN" --output table --query 'tasks[0].{TaskArn:taskArn,LastStatus:lastStatus,DesiredStatus:desiredStatus,CreatedAt:createdAt}' + fi + + # List container instances + log "Container instances in cluster:" + aws ecs list-container-instances --cluster "$CLUSTER_NAME" --output table + + # Describe container instance + CONTAINER_INSTANCE_ARN=$(aws ecs list-container-instances --cluster "$CLUSTER_NAME" --query 'containerInstanceArns[0]' --output text) + if [[ -n "$CONTAINER_INSTANCE_ARN" && "$CONTAINER_INSTANCE_ARN" != "None" ]]; then + log "Container instance details:" + aws ecs describe-container-instances --cluster "$CLUSTER_NAME" --container-instances "$CONTAINER_INSTANCE_ARN" --output table --query 'containerInstances[0].{Arn:containerInstanceArn,Status:status,RunningTasks:runningTasksCount,PendingTasks:pendingTasksCount}' + fi + + # Test the nginx web server + log "Testing nginx web server..." + PUBLIC_IP=$(aws ec2 describe-instances --instance-ids "$INSTANCE_ID" --query 'Reservations[0].Instances[0].PublicIpAddress' --output text) + + if [[ -n "$PUBLIC_IP" && "$PUBLIC_IP" != "None" ]]; then + log "Container instance public IP: $PUBLIC_IP" + log "Testing HTTP connection to nginx..." + + # Wait a moment for nginx to be fully ready + sleep 10 + + if curl -s --connect-timeout 10 "http://$PUBLIC_IP" | grep -q "Welcome to nginx"; then + log "SUCCESS: Nginx web server is responding correctly" + echo "" + echo "===========================================" + echo "WEB SERVER TEST SUCCESSFUL" + echo "===========================================" + echo "You can access your nginx web server at: http://$PUBLIC_IP" + echo "The nginx welcome page should be visible in your browser." + else + log "WARNING: Nginx web server may not be fully ready yet. Try accessing http://$PUBLIC_IP in a few minutes." + fi + else + log "WARNING: Could not retrieve public IP address" + fi +} + +# Main execution +main() { + log "Starting ECS EC2 Launch Type Tutorial (UPDATED VERSION)" + log "Log file: $LOG_FILE" + + check_prerequisites + create_cluster + create_key_pair + create_security_group + get_ecs_ami + ensure_ecs_instance_role + launch_container_instance + register_task_definition + create_service + demonstrate_monitoring + + log "Tutorial completed successfully!" + + echo "" + echo "===========================================" + echo "TUTORIAL COMPLETED SUCCESSFULLY" + echo "===========================================" + echo "Resources created:" + for resource in "${CREATED_RESOURCES[@]}"; do + echo " - $resource" + done + echo "" + echo "The nginx service will continue running and maintain the desired task count." + echo "You can monitor the service status using:" + echo " aws ecs describe-services --cluster $CLUSTER_NAME --services $SERVICE_NAME" + echo "" + if [[ -n "${PUBLIC_IP:-}" ]]; then + echo "Access your web server at: http://$PUBLIC_IP" + echo "" + fi + echo "===========================================" + echo "CLEANUP CONFIRMATION" + echo "===========================================" + echo "Do you want to clean up all created resources? (y/n): " + read -r CLEANUP_CHOICE + + if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + cleanup_resources + log "All resources have been cleaned up" + else + log "Resources left running. Remember to clean them up manually to avoid charges." + echo "" + echo "To clean up manually later, run these commands:" + echo " aws ecs update-service --cluster $CLUSTER_NAME --service $SERVICE_NAME --desired-count 0" + echo " aws ecs delete-service --cluster $CLUSTER_NAME --service $SERVICE_NAME" + echo " aws ecs delete-cluster --cluster $CLUSTER_NAME" + echo " aws ec2 terminate-instances --instance-ids $INSTANCE_ID" + echo " aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID" + echo " aws ec2 delete-key-pair --key-name $KEY_PAIR_NAME" + fi + + log "Script execution completed" +} + +# Run main function +main "$@" diff --git a/tuts/020-ebs-gs-volumes/README.md b/tuts/020-ebs-gs-volumes/README.md new file mode 100644 index 00000000..9d5bac47 --- /dev/null +++ b/tuts/020-ebs-gs-volumes/README.md @@ -0,0 +1,5 @@ +# Amazon EBS getting started volumes + +This tutorial guides you through the process of creating and managing Amazon Elastic Block Store (Amazon EBS) volumes using the AWS Command Line Interface (AWS CLI). You'll learn how to create volumes, check their status, attach them to Amazon EC2 instances, and clean up resources when you're done. + +You can either run the automated shell script (`ebs-gs-volumes.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`ebs-gs-volumes.md`) to understand each component in detail. Both approaches will help you understand how to work with Amazon EBS volumes for persistent storage with your Amazon EC2 instances. diff --git a/tuts/020-ebs-gs-volumes/ebs-gs-volumes.md b/tuts/020-ebs-gs-volumes/ebs-gs-volumes.md new file mode 100644 index 00000000..b891db9a --- /dev/null +++ b/tuts/020-ebs-gs-volumes/ebs-gs-volumes.md @@ -0,0 +1,517 @@ +# Creating and managing Amazon EBS volumes using the AWS CLI + +This tutorial guides you through the process of creating and managing Amazon Elastic Block Store (EBS) volumes using the AWS Command Line Interface (AWS CLI). You'll learn how to create volumes, check their status, attach them to EC2 instances, and clean up resources when you're done. + +## Prerequisites + +Before you begin this tutorial, make sure you have the following: + +1. The AWS CLI installed and configured. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). +2. Configured your AWS CLI with appropriate credentials. Run `aws configure` if you haven't set up your credentials yet. +3. Permissions to create and manage EBS volumes and EC2 instances in your AWS account. At minimum, you need the following permissions: `ec2:CreateVolume`, `ec2:DescribeVolumes`, `ec2:AttachVolume`, `ec2:DetachVolume`, and `ec2:DeleteVolume`. +4. At least one running EC2 instance if you want to attach your volume (optional - the script provided with this tutorial can create a test instance for you if needed). + +**Time to complete:** Approximately 30 minutes + +**Cost:** The resources created in this tutorial will incur charges while they exist. The approximate costs for running the resources for one hour in the US East (N. Virginia) region are: +- Standard gp3 volume (10 GiB): $0.01/hour +- EC2 t2.micro instance (if created): $0.0116/hour + +To avoid ongoing charges, follow the cleanup instructions at the end of the tutorial. For more information about pricing, see [Amazon EBS pricing](https://aws.amazon.com/ebs/pricing/) and [Amazon EC2 pricing](https://aws.amazon.com/ec2/pricing/). + +## Automated script option + +If you prefer to automate the steps in this tutorial, you can use the provided script. The script creates an EBS volume, optionally creates an EC2 instance if you don't have one available, attaches the volume to the instance, and provides cleanup options. + +To run the script: + +```bash +./ebs-gs-volumes.sh +``` + +The script will guide you through the process with interactive prompts. For a more hands-on experience, continue with the manual steps below. + +## Create an EBS volume + +Amazon EBS provides block-level storage volumes that you can attach to EC2 instances. In this section, you'll create a new EBS volume that you can later attach to an instance. + +**Create a basic gp3 volume** + +The following command creates a 100 GiB General Purpose SSD (gp3) volume in the specified Availability Zone. The gp3 volume type provides a baseline performance of 3,000 IOPS and 125 MiB/s throughput. + +```bash +aws ec2 create-volume \ + --volume-type gp3 \ + --size 100 \ + --region us-east-1 \ + --availability-zone us-east-1a +``` + +The command returns details about the newly created volume, including its ID, size, type, and state: + +```json +{ + "AvailabilityZone": "us-east-1a", + "Tags": [], + "Encrypted": false, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "creating", + "Iops": 3000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:15:30.000Z", + "Size": 100, + "Throughput": 125 +} +``` + +When you create a volume, it starts in the `creating` state and transitions to the `available` state when it's ready to use. Make note of the `VolumeId` as you'll need it for subsequent commands. + +**Create a volume with custom performance settings** + +If you need higher performance than the gp3 baseline provides, you can specify custom IOPS and throughput values. The following command creates a gp3 volume with 4,000 IOPS and 250 MiB/s throughput: + +```bash +aws ec2 create-volume \ + --volume-type gp3 \ + --size 100 \ + --iops 4000 \ + --throughput 250 \ + --region us-east-1 \ + --availability-zone us-east-1a +``` + +This command returns similar output to the previous one, but with the specified IOPS and throughput values: + +```json +{ + "AvailabilityZone": "us-east-1a", + "Tags": [], + "Encrypted": false, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678j", + "State": "creating", + "Iops": 4000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:20:45.000Z", + "Size": 100, + "Throughput": 250 +} +``` + +**Create an encrypted volume** + +For sensitive data, you should create encrypted volumes. The following command creates an encrypted gp3 volume using the default AWS managed key for EBS: + +```bash +aws ec2 create-volume \ + --volume-type gp3 \ + --size 100 \ + --encrypted \ + --region us-east-1 \ + --availability-zone us-east-1a +``` + +The response shows that the volume is encrypted: + +```json +{ + "AvailabilityZone": "us-east-1a", + "Tags": [], + "Encrypted": true, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678k", + "State": "creating", + "Iops": 3000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:25:15.000Z", + "Size": 100, + "Throughput": 125, + "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcd1234-a123-456a-a12b-a123b4cd56ef" +} +``` + +**Create a volume with tags** + +Tags help you organize and manage your AWS resources. The following command creates a volume with tags for Name and Environment: + +```bash +aws ec2 create-volume \ + --volume-type gp3 \ + --size 100 \ + --region us-east-1 \ + --availability-zone us-east-1a \ + --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=MyDataVolume},{Key=Environment,Value=Production}]' +``` + +The response includes the tags you specified: + +```json +{ + "AvailabilityZone": "us-east-1a", + "Tags": [ + { + "Key": "Name", + "Value": "MyDataVolume" + }, + { + "Key": "Environment", + "Value": "Production" + } + ], + "Encrypted": false, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678l", + "State": "creating", + "Iops": 3000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:30:00.000Z", + "Size": 100, + "Throughput": 125 +} +``` + +## Check volume status + +Before you can use a volume, it must be in the `available` state. This section shows you how to check the status of your volumes. + +**Check a specific volume's status** + +To check the status of a specific volume, use the `describe-volumes` command with the volume ID: + +```bash +aws ec2 describe-volumes \ + --region us-east-1 \ + --volume-ids vol-abcd1234efgh5678i +``` + +The command returns detailed information about the volume: + +```json +{ + "Volumes": [ + { + "AvailabilityZone": "us-east-1a", + "Attachments": [], + "Encrypted": false, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "available", + "Iops": 3000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:15:30.000Z", + "Size": 100, + "Throughput": 125 + } + ] +} +``` + +The `State` field shows that the volume is now `available`, which means it's ready to be attached to an instance. + +**Filter volumes by state** + +You can also list all volumes in a specific state, such as `available`: + +```bash +aws ec2 describe-volumes \ + --region us-east-1 \ + --filters Name=status,Values=available +``` + +This command returns information about all available volumes in your account in the current region. + +## Create an EC2 instance (optional) + +If you don't have an EC2 instance available for attaching your volume, you can create one using the following command. Make sure to use a subnet in the same Availability Zone as the volume that you want to attach. + +```bash +# Get the latest Amazon Linux 2 AMI +AMI_ID=$(aws ec2 describe-images \ + --owners amazon \ + --filters "Name=name,Values=amzn2-ami-hvm-*-x86_64-gp2" "Name=state,Values=available" \ + --query "sort_by(Images, &CreationDate)[-1].ImageId" \ + --region us-east-1 \ + --output text) + +# Create the instance +aws ec2 run-instances \ + --image-id "$AMI_ID" \ + --instance-type "t2.micro" \ + --subnet-id "subnet-01234567890abcdef" \ + --security-group-ids "sg-abcdef01234567890" \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=EBSTutorialInstance}]' \ + --query "Instances[0].InstanceId" \ + --region us-east-1 \ + --output text + +``` + +Wait for the instance to be in the `running` state before proceeding: + +```bash +aws ec2 wait instance-running --instance-ids i-abcd1234efgh5678m --region us-east-1 +``` + +## Attach a volume to an EC2 instance + +Once your volume is in the `available` state, you can attach it to an EC2 instance in the same Availability Zone. This section shows you how to attach a volume and verify the attachment. + +**Attach the volume to an instance** + +To attach a volume to an instance, you need the volume ID, instance ID, and a device name. The following command attaches a volume to an instance as `/dev/sdf`: + +```bash +aws ec2 attach-volume \ + --volume-id vol-abcd1234efgh5678i \ + --instance-id i-abcd1234efgh5678m \ + --region us-east-1 \ + --device /dev/sdf +``` + +The command returns information about the attachment: + +```json +{ + "AttachTime": "2025-01-13T10:45:30.000Z", + "InstanceId": "i-abcd1234efgh5678m", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "attaching", + "Device": "/dev/sdf" +} +``` + +The `State` field shows that the volume is in the process of being attached. It will change to `attached` when the process is complete. + +**Verify the attachment** + +To verify that the volume is attached to the instance, you can describe the volume again: + +```bash +aws ec2 describe-volumes \ + --region us-east-1 \ + --volume-ids vol-abcd1234efgh5678i +``` + +The response now includes attachment information: + +```json +{ + "Volumes": [ + { + "AvailabilityZone": "us-east-1a", + "Attachments": [ + { + "AttachTime": "2025-01-13T10:45:30.000Z", + "InstanceId": "i-abcd1234efgh5678m", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "attached", + "Device": "/dev/sdf", + "DeleteOnTermination": false + } + ], + "Encrypted": false, + "VolumeType": "gp3", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "in-use", + "Iops": 3000, + "SnapshotId": "", + "CreateTime": "2025-01-13T10:15:30.000Z", + "Size": 100, + "Throughput": 125 + } + ] +} +``` + +The `State` field now shows `in-use`, and the `Attachments` array contains information about the attachment. + +**List volumes attached to a specific instance** + +You can also list all volumes attached to a specific instance: + +```bash +aws ec2 describe-volumes \ + --region us-east-1 \ + --filters Name=attachment.instance-id,Values=i-abcd1234efgh5678m +``` + +This command returns information about all volumes attached to the specified instance. + +## Modify a volume (optional) + +You can modify the size, IOPS, or throughput of an existing EBS volume without detaching it from an instance. This section shows you how to modify a volume's attributes. + +**Increase volume size and performance** + +The following command increases a volume's size to 200 GiB and changes its type to io2 with 10,000 IOPS: + +```bash +aws ec2 modify-volume \ + --volume-id vol-abcd1234efgh5678i \ + --size 200 \ + --volume-type io2 \ + --region us-east-1 \ + --iops 10000 +``` + +The command returns information about the modification: + +```json +{ + "VolumeModification": { + "VolumeId": "vol-abcd1234efgh5678i", + "ModificationState": "modifying", + "TargetSize": 200, + "TargetIops": 10000, + "TargetVolumeType": "io2", + "OriginalSize": 100, + "OriginalIops": 3000, + "OriginalVolumeType": "gp3", + "Progress": 0, + "StartTime": "2025-01-13T11:00:00.000Z" + } +} +``` + +The `ModificationState` field shows that the volume is being modified. The modification process can take some time to complete, especially for larger volumes. + +## Clean up resources + +When you're finished with your EBS volumes, you should clean them up to avoid incurring additional charges. This section shows you how to detach and delete volumes. + +**Detach a volume** + +Before you can delete a volume, you must detach it from any instances: + +```bash +aws ec2 detach-volume \ + --region us-east-1 \ + --volume-id vol-abcd1234efgh5678i +``` + +The command returns information about the detachment: + +```json +{ + "AttachTime": "2025-01-13T10:45:30.000Z", + "InstanceId": "i-abcd1234efgh5678m", + "VolumeId": "vol-abcd1234efgh5678i", + "State": "detaching", + "Device": "/dev/sdf" +} +``` + +The `State` field shows that the volume is in the process of being detached. You should wait until the volume is fully detached before deleting it. + +**Delete a volume** + +Once a volume is detached and in the `available` state, you can delete it: + +```bash +aws ec2 delete-volume \ + --region us-east-1 \ + --volume-id vol-abcd1234efgh5678i +``` + +This command doesn't return any output if successful. You can verify that the volume has been deleted by trying to describe it: + +```bash +aws ec2 describe-volumes \ + --region us-east-1 \ + --volume-ids vol-abcd1234efgh5678i +``` + +If the volume has been deleted, you'll receive an error message indicating that the volume doesn't exist. + +**Terminate the EC2 instance (if created for this tutorial)** + +If you created an EC2 instance specifically for this tutorial, you should terminate it to avoid ongoing charges: + +```bash +aws ec2 terminate-instances \ + --region us-east-1 \ + --instance-ids i-abcd1234efgh5678m +``` + +Wait for the instance to terminate: + +```bash +aws ec2 wait instance-terminated \ + --region us-east-1 \ + --instance-ids i-abcd1234efgh5678m +``` + +## Troubleshooting + +Here are some common issues you might encounter when working with EBS volumes and how to resolve them: + +**Volume stuck in "creating" state** + +If a volume remains in the `creating` state for an extended period: +- Check your AWS service health dashboard for any EBS issues in your region +- Try creating the volume in a different Availability Zone +- Contact AWS Support if the issue persists + +**Permission errors** + +If you receive permission errors: +- Verify that your IAM user or role has the necessary permissions +- Check for any resource-based policies that might be restricting access +- Ensure you're operating in the correct AWS account and region + +**Availability Zone mismatch** + +If you can't attach a volume to an instance: +- Verify that the volume and instance are in the same Availability Zone +- You cannot attach a volume to an instance in a different Availability Zone + +**Volume limit exceeded** + +If you receive a "volume limit exceeded" error: +- Check your current EBS volume limits in the EC2 console +- Request a limit increase through the Service Quotas console if needed + + +## Going to production + +This tutorial is designed to help you learn how to use the EBS API through the AWS CLI. When implementing EBS volumes in a production environment, consider the following additional best practices: + +### Security considerations + +1. **IAM permissions**: Implement least privilege access by creating specific IAM policies that grant only the permissions needed for each role or user. + +2. **KMS key management**: For sensitive data, create and manage your own customer managed keys (CMKs) instead of using the default AWS managed key. + +3. **Encryption**: Enable default encryption for all EBS volumes in your account. + +4. **Monitoring**: Set up CloudTrail to monitor API calls related to your EBS volumes. + +### Architecture best practices + +1. **Backup strategy**: Implement regular snapshots of your volumes to protect against data loss. + +2. **Multi-AZ resilience**: Design your application to span multiple Availability Zones for higher availability. + +3. **Performance monitoring**: Use CloudWatch to monitor volume performance metrics and set up alarms for potential issues. + +4. **Automation**: Use infrastructure as code tools like AWS CloudFormation or Terraform to manage your volumes consistently. + +5. **Cost optimization**: Implement lifecycle policies to manage snapshots and consider using gp3 volumes instead of gp2 for better price-performance. + +For more information on building production-ready systems on AWS, refer to: +- [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) +- [AWS Security Best Practices](https://aws.amazon.com/architecture/security-identity-compliance/) +- [EBS Best Practices](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html) + +## Next steps + +Now that you've learned how to create and manage EBS volumes using the AWS CLI, you can explore other EBS features: + +1. **Snapshots** - [Create point-in-time backups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSSnapshots.html) of your EBS volumes. +2. **Volume initialization** - [Improve performance](https://docs.aws.amazon.com/ebs/latest/userguide/initalize-volume.html) when creating volumes from snapshots. +3. **Multi-Attach** - [Attach a single volume to multiple instances](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes-multi.html) (for io1 and io2 volumes only). +4. **Volume metrics** - [Monitor your volumes](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cloudwatch_ebs.html) using Amazon CloudWatch. +5. **Data lifecycle management** - [Automate snapshot creation and deletion](https://docs.aws.amazon.com/ebs/latest/userguide/dlm-overview.html) using Amazon Data Lifecycle Manager. + +For more information about available AWS CLI commands for EBS, see the [AWS CLI Command Reference for EC2](https://docs.aws.amazon.com/cli/latest/reference/ec2/index.html). diff --git a/tuts/020-ebs-gs-volumes/ebs-gs-volumes.sh b/tuts/020-ebs-gs-volumes/ebs-gs-volumes.sh new file mode 100755 index 00000000..1834c816 --- /dev/null +++ b/tuts/020-ebs-gs-volumes/ebs-gs-volumes.sh @@ -0,0 +1,376 @@ +#!/bin/bash + +# Script to create and manage Amazon EBS volumes +# This script demonstrates how to create an EBS volume and attach it to an EC2 instance +# It can also create a test EC2 instance if needed + +# Set up logging +LOG_FILE="ebs-volume-creation.log" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "Starting EBS volume creation script at $(date)" +echo "==============================================" + +# Function to handle errors +handle_error() { + echo "ERROR: $1" + echo "Resources created:" + if [ -n "$VOLUME_ID" ]; then + echo "- EBS Volume: $VOLUME_ID" + fi + if [ -n "$INSTANCE_ID" ]; then + echo "- EC2 Instance: $INSTANCE_ID" + fi + if [ -n "$SG_ID" ]; then + echo "- Security Group: $SG_ID" + fi + + echo "" + echo "===========================================" + echo "CLEANUP CONFIRMATION" + echo "===========================================" + echo "An error occurred. Do you want to clean up created resources? (y/n): " + read -r CLEANUP_CHOICE + + if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + cleanup_resources + else + echo "Resources were not cleaned up. You will need to delete them manually." + fi + + exit 1 +} + +# Function to clean up resources +cleanup_resources() { + echo "Cleaning up resources..." + + if [ -n "$VOLUME_ID" ] && [ "$ATTACHED" = true ]; then + echo "Detaching volume $VOLUME_ID..." + aws ec2 detach-volume --volume-id "$VOLUME_ID" + + # Wait for volume to be detached + echo "Waiting for volume to be detached..." + aws ec2 wait volume-available --volume-ids "$VOLUME_ID" + fi + + if [ -n "$VOLUME_ID" ]; then + echo "Deleting volume $VOLUME_ID..." + aws ec2 delete-volume --volume-id "$VOLUME_ID" + fi + + if [ -n "$INSTANCE_ID" ] && [ "$CREATED_INSTANCE" = true ]; then + echo "Terminating instance $INSTANCE_ID..." + aws ec2 terminate-instances --instance-ids "$INSTANCE_ID" + echo "Waiting for instance to terminate..." + aws ec2 wait instance-terminated --instance-ids "$INSTANCE_ID" + fi + + # Clean up security group if created + if [ -n "$SG_ID" ] && [ "$CREATED_INSTANCE" = true ]; then + echo "Deleting security group $SG_ID..." + # Wait a bit for instance termination to complete + sleep 10 + aws ec2 delete-security-group --group-id "$SG_ID" 2>/dev/null || echo "Security group may have dependencies, delete manually if needed" + fi + + echo "Cleanup completed." +} + +# Function to get available instance type +get_available_instance_type() { + local region=$1 + + # Try instance types in order of preference (cheapest first) + local instance_types=("t3.nano" "t3.micro" "t2.micro" "t2.nano") + + for instance_type in "${instance_types[@]}"; do + local available=$(aws ec2 describe-instance-type-offerings \ + --region "$region" \ + --filters "Name=instance-type,Values=$instance_type" \ + --query "length(InstanceTypeOfferings)" \ + --output text) + + if [ "$available" -gt 0 ]; then + echo "$instance_type" + return 0 + fi + done + + # If none of the preferred types are available, get any available type + local fallback_type=$(aws ec2 describe-instance-type-offerings \ + --region "$region" \ + --query "InstanceTypeOfferings[0].InstanceType" \ + --output text) + + if [ "$fallback_type" != "None" ] && [ -n "$fallback_type" ]; then + echo "$fallback_type" + return 0 + fi + + return 1 +} + +# Get current region +REGION=$(aws configure get region) +if [ -z "$REGION" ]; then + REGION=$(aws ec2 describe-availability-zones --query "AvailabilityZones[0].RegionName" --output text) +fi + +echo "Using region: $REGION" + +# Get available Availability Zones +echo "Retrieving available Availability Zones..." +AZ=$(aws ec2 describe-availability-zones --filters "Name=state,Values=available" --query "AvailabilityZones[0].ZoneName" --output text) +if [ -z "$AZ" ]; then + handle_error "Failed to retrieve Availability Zones" +fi +echo "Using Availability Zone: $AZ" + +# Create a gp3 volume +echo "Creating a 10 GiB gp3 volume in $AZ..." +VOLUME_ID=$(aws ec2 create-volume \ + --volume-type gp3 \ + --size 10 \ + --availability-zone "$AZ" \ + --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=EBSTutorialVolume},{Key=Purpose,Value=Tutorial}]' \ + --query 'VolumeId' \ + --output text) + +if [ -z "$VOLUME_ID" ]; then + handle_error "Failed to create EBS volume" +fi + +echo "Volume created with ID: $VOLUME_ID" + +# Wait for volume to become available +echo "Waiting for volume to become available..." +aws ec2 wait volume-available --volume-ids "$VOLUME_ID" +if [ $? -ne 0 ]; then + handle_error "Volume did not become available" +fi + +# Check volume details +echo "Retrieving volume details..." +aws ec2 describe-volumes --volume-ids "$VOLUME_ID" + +# Ask if user wants to attach the volume to an instance +echo "" +echo "===========================================" +echo "VOLUME ATTACHMENT" +echo "===========================================" +echo "Do you want to attach this volume to an EC2 instance? (y/n): " +read -r ATTACH_CHOICE + +ATTACHED=false +CREATED_INSTANCE=false +INSTANCE_ID="" +SG_ID="" + +if [[ "$ATTACH_CHOICE" =~ ^[Yy]$ ]]; then + # List available instances in the same AZ + echo "Retrieving EC2 instances in $AZ..." + INSTANCES_COUNT=$(aws ec2 describe-instances \ + --filters "Name=availability-zone,Values=$AZ" "Name=instance-state-name,Values=running" \ + --query "length(Reservations[].Instances[])" \ + --output text) + + # Check if there are any running instances in the AZ + if [ "$INSTANCES_COUNT" -eq 0 ]; then + echo "No running instances found in $AZ." + echo "" + echo "Would you like to create a test EC2 instance? (y/n): " + read -r CREATE_INSTANCE_CHOICE + + if [[ "$CREATE_INSTANCE_CHOICE" =~ ^[Yy]$ ]]; then + # Get available instance type + echo "Finding available instance type for region $REGION..." + INSTANCE_TYPE=$(get_available_instance_type "$REGION") + if [ $? -ne 0 ] || [ -z "$INSTANCE_TYPE" ]; then + handle_error "No suitable instance type found in region $REGION" + fi + echo "Using instance type: $INSTANCE_TYPE" + + # Get the latest Amazon Linux 2 AMI + echo "Finding the latest Amazon Linux 2 AMI..." + AMI_ID=$(aws ec2 describe-images \ + --owners amazon \ + --filters "Name=name,Values=amzn2-ami-hvm-*-x86_64-gp2" "Name=state,Values=available" \ + --query "sort_by(Images, &CreationDate)[-1].ImageId" \ + --output text) + + if [ -z "$AMI_ID" ]; then + handle_error "Failed to find a suitable AMI" + fi + + echo "Using AMI: $AMI_ID" + + # Check if a default VPC exists + DEFAULT_VPC_ID=$(aws ec2 describe-vpcs \ + --filters "Name=isDefault,Values=true" \ + --query "Vpcs[0].VpcId" \ + --output text) + + if [ "$DEFAULT_VPC_ID" = "None" ] || [ -z "$DEFAULT_VPC_ID" ]; then + handle_error "No default VPC found. Please create a VPC and subnet before running this script." + fi + + # Get a subnet in the selected AZ + SUBNET_ID=$(aws ec2 describe-subnets \ + --filters "Name=vpc-id,Values=$DEFAULT_VPC_ID" "Name=availability-zone,Values=$AZ" \ + --query "Subnets[0].SubnetId" \ + --output text) + + if [ -z "$SUBNET_ID" ] || [ "$SUBNET_ID" = "None" ]; then + handle_error "No subnet found in $AZ. Please create a subnet before running this script." + fi + + echo "Using subnet: $SUBNET_ID" + + # Create a security group that allows SSH + SG_NAME="EBSTutorialSG-$(date +%s)" + SG_ID=$(aws ec2 create-security-group \ + --group-name "$SG_NAME" \ + --description "Security group for EBS tutorial" \ + --vpc-id "$DEFAULT_VPC_ID" \ + --query "GroupId" \ + --output text) + + if [ -z "$SG_ID" ]; then + handle_error "Failed to create security group" + fi + + echo "Created security group: $SG_ID" + + # Add a rule to allow SSH + aws ec2 authorize-security-group-ingress \ + --group-id "$SG_ID" \ + --protocol tcp \ + --port 22 \ + --cidr 0.0.0.0/0 + + echo "Added SSH rule to security group" + + # Create the instance + echo "Creating EC2 instance in $AZ with instance type $INSTANCE_TYPE..." + INSTANCE_ID=$(aws ec2 run-instances \ + --image-id "$AMI_ID" \ + --instance-type "$INSTANCE_TYPE" \ + --subnet-id "$SUBNET_ID" \ + --security-group-ids "$SG_ID" \ + --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=EBSTutorialInstance},{Key=Purpose,Value=Tutorial}]' \ + --query "Instances[0].InstanceId" \ + --output text) + + if [ -z "$INSTANCE_ID" ]; then + handle_error "Failed to create EC2 instance" + fi + + CREATED_INSTANCE=true + echo "Instance created with ID: $INSTANCE_ID" + + # Wait for the instance to be running + echo "Waiting for instance to be running..." + aws ec2 wait instance-running --instance-ids "$INSTANCE_ID" + + # Wait a bit more for the instance to initialize + echo "Waiting for instance initialization (30 seconds)..." + sleep 30 + else + echo "Skipping instance creation and volume attachment." + INSTANCE_ID="" + fi + else + # Display available instances + echo "Available instances in $AZ:" + aws ec2 describe-instances \ + --filters "Name=availability-zone,Values=$AZ" "Name=instance-state-name,Values=running" \ + --query "Reservations[*].Instances[*].[InstanceId,Tags[?Key=='Name'].Value|[0],InstanceType]" \ + --output table + + # Ask for instance ID + echo "" + echo "Enter the instance ID to attach the volume to (or press Enter to skip): " + read -r INSTANCE_ID + fi + + if [ -n "$INSTANCE_ID" ]; then + # Attach volume to the instance + echo "Attaching volume $VOLUME_ID to instance $INSTANCE_ID..." + ATTACH_RESULT=$(aws ec2 attach-volume \ + --volume-id "$VOLUME_ID" \ + --instance-id "$INSTANCE_ID" \ + --device "/dev/sdf" \ + --query 'State' \ + --output text) + + if [ $? -ne 0 ] || [ -z "$ATTACH_RESULT" ]; then + handle_error "Failed to attach volume to instance" + fi + + ATTACHED=true + echo "Volume attached successfully. Device: /dev/sdf" + + # Verify attachment + echo "Verifying attachment..." + aws ec2 describe-volumes \ + --volume-ids "$VOLUME_ID" \ + --query "Volumes[0].Attachments" + else + echo "Skipping volume attachment." + fi +else + echo "Skipping volume attachment." +fi + +# Display summary of created resources +echo "" +echo "===========================================" +echo "RESOURCE SUMMARY" +echo "===========================================" +echo "Created resources:" +echo "- EBS Volume: $VOLUME_ID" +if [ "$ATTACHED" = true ]; then + echo " - Attached to: $INSTANCE_ID as /dev/sdf" +fi +if [ "$CREATED_INSTANCE" = true ]; then + echo "- EC2 Instance: $INSTANCE_ID (type: $INSTANCE_TYPE)" +fi +if [ -n "$SG_ID" ]; then + echo "- Security Group: $SG_ID" +fi + +# Ask if user wants to clean up resources +echo "" +echo "===========================================" +echo "CLEANUP CONFIRMATION" +echo "===========================================" +echo "Do you want to clean up all created resources? (y/n): " +read -r CLEANUP_CHOICE + +if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + cleanup_resources +else + echo "" + echo "Resources were not cleaned up. You can manually delete them later." + if [ -n "$VOLUME_ID" ]; then + if [ "$ATTACHED" = true ]; then + echo "To detach the volume:" + echo " aws ec2 detach-volume --volume-id $VOLUME_ID" + fi + echo "To delete the volume:" + echo " aws ec2 delete-volume --volume-id $VOLUME_ID" + fi + if [ "$CREATED_INSTANCE" = true ]; then + echo "To terminate the instance:" + echo " aws ec2 terminate-instances --instance-ids $INSTANCE_ID" + fi + if [ -n "$SG_ID" ]; then + echo "To delete the security group (after instance termination):" + echo " aws ec2 delete-security-group --group-id $SG_ID" + fi +fi + +echo "" +echo "Script completed at $(date)" +echo "==============================================" diff --git a/tuts/058-elastic-load-balancing-gs/README.md b/tuts/058-elastic-load-balancing-gs/README.md new file mode 100644 index 00000000..95bfad69 --- /dev/null +++ b/tuts/058-elastic-load-balancing-gs/README.md @@ -0,0 +1,5 @@ +# Elastic Load Balancing getting started + +This tutorial guides you through creating and configuring an Application Load Balancer using the AWS Command Line Interface (AWS CLI). You'll learn how to create a load balancer, configure target groups, register targets, and set up listeners to distribute traffic to your applications. + +You can either run the automated shell script (`elastic-load-balancing-gs.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`elastic-load-balancing-gs.md`) to understand each component in detail. Both approaches will help you understand how to use Elastic Load Balancing to distribute incoming traffic across multiple targets for improved availability and fault tolerance. diff --git a/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.md b/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.md new file mode 100644 index 00000000..8a4867ad --- /dev/null +++ b/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.md @@ -0,0 +1,349 @@ +# Getting started with Elastic Load Balancing using the AWS CLI + +This tutorial guides you through creating and configuring an Application Load Balancer using the AWS Command Line Interface (AWS CLI). You'll learn how to create a load balancer, configure target groups, register targets, and set up listeners to distribute traffic to your applications. + +## Topics + +* [Prerequisites](#prerequisites) +* [Create an Application Load Balancer](#create-an-application-load-balancer) +* [Create a target group](#create-a-target-group) +* [Register targets](#register-targets) +* [Create a listener](#create-a-listener) +* [Verify your configuration](#verify-your-configuration) +* [Add an HTTPS listener (optional)](#add-an-https-listener-optional) +* [Add path-based routing (optional)](#add-path-based-routing-optional) +* [Going to production](#going-to-production) +* [Clean up resources](#clean-up-resources) +* [Next steps](#next-steps) + +## Prerequisites + +Before you begin this tutorial, make sure you have the following: + +1. The AWS CLI installed and configured with appropriate credentials. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). +2. EC2 instances running in a VPC with a web server (such as Apache or IIS) installed. +3. Security groups configured to allow HTTP access on port 80 for your instances. +4. At least two subnets in different Availability Zones within your VPC. +5. [Sufficient permissions](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-authentication-access-control.html) to create and manage Elastic Load Balancing resources. + +**Cost information**: The resources created in this tutorial will incur charges as long as they exist. The estimated cost for running an Application Load Balancer is approximately $0.0305 per hour, plus any charges for your EC2 instances (approximately $0.0116 per hour per t2.micro instance). The total estimated cost for completing this tutorial (assuming 1 hour with 2 t2.micro instances) is about $0.05. Make sure to follow the cleanup instructions to avoid ongoing charges. For more information about pricing, see [Elastic Load Balancing pricing](https://aws.amazon.com/elasticloadbalancing/pricing/) and [Amazon EC2 pricing](https://aws.amazon.com/ec2/pricing/). + +First, verify that your AWS CLI version supports Elastic Load Balancing v2 commands: + +``` +$ aws elbv2 help +``` + +If you get an error message, update your AWS CLI to the latest version. + +## Create an Application Load Balancer + +An Application Load Balancer operates at the application layer (layer 7) and routes traffic based on the content of the request. In this section, you'll create an Application Load Balancer that distributes traffic across multiple EC2 instances. + +**Retrieve your VPC and subnet information** + +Before creating a load balancer, you need to identify your VPC and subnets. The following commands help you retrieve this information: + +``` +$ VPC_ID=$(aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" --query "Vpcs[0].VpcId" --output text) +$ echo "Using VPC: $VPC_ID" + +$ SUBNETS=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query "Subnets[0:2].SubnetId" --output text) +$ read -r SUBNET1 SUBNET2 <<< "$SUBNETS" +$ echo "Using subnets: $SUBNET1 and $SUBNET2" +``` + +These commands find your default VPC and retrieve two subnets within it. For a production environment, you should select specific subnets in different Availability Zones. + +**Create a security group for the load balancer** + +Next, create a security group that allows HTTP traffic to your load balancer: + +``` +$ SECURITY_GROUP_ID=$(aws ec2 create-security-group \ + --group-name "elb-demo-sg" \ + --description "Security group for ELB demo" \ + --vpc-id "$VPC_ID" \ + --query "GroupId" --output text) +$ echo "Created security group: $SECURITY_GROUP_ID" + +$ aws ec2 authorize-security-group-ingress \ + --group-id "$SECURITY_GROUP_ID" \ + --protocol tcp \ + --port 80 \ + --cidr "203.0.113.0/24" # Replace with your specific IP range +``` + +This security group allows inbound HTTP traffic from a specific IP range. For the tutorial, you can use your own IP address or range. In a production environment, you should restrict this to only the IP ranges that need access to your application. + +**Create the load balancer** + +Now, create the Application Load Balancer using the subnets and security group: + +``` +$ LOAD_BALANCER_ARN=$(aws elbv2 create-load-balancer \ + --name "my-load-balancer" \ + --subnets "$SUBNET1" "$SUBNET2" \ + --security-groups "$SECURITY_GROUP_ID" \ + --query "LoadBalancers[0].LoadBalancerArn" --output text) +$ echo "Created load balancer: $LOAD_BALANCER_ARN" +``` + +This command creates an internet-facing Application Load Balancer in the specified subnets with the security group you created. The load balancer ARN is stored in the `LOAD_BALANCER_ARN` variable for later use. + +Wait for the load balancer to become active before proceeding: + +``` +$ aws elbv2 wait load-balancer-available --load-balancer-arns "$LOAD_BALANCER_ARN" +``` + +## Create a target group + +A target group routes requests to registered targets (such as EC2 instances) using the protocol and port you specify. In this section, you'll create a target group for your Application Load Balancer. + +``` +$ TARGET_GROUP_ARN=$(aws elbv2 create-target-group \ + --name "my-targets" \ + --protocol HTTP \ + --port 80 \ + --vpc-id "$VPC_ID" \ + --target-type instance \ + --query "TargetGroups[0].TargetGroupArn" --output text) +$ echo "Created target group: $TARGET_GROUP_ARN" +``` + +This command creates a target group that uses the HTTP protocol on port 80. The target type is set to `instance`, which means you'll register EC2 instances by their instance IDs. + +You can customize the health check settings for your target group to better suit your application: + +``` +$ aws elbv2 modify-target-group \ + --target-group-arn "$TARGET_GROUP_ARN" \ + --health-check-path "/health" \ + --health-check-interval-seconds 15 \ + --healthy-threshold-count 3 \ + --unhealthy-threshold-count 3 +``` + +This command configures the health check to check the `/health` path every 15 seconds and requires 3 consecutive successful or failed checks to change the health status. + +## Register targets + +After creating a target group, you need to register targets with it. In this section, you'll register EC2 instances with your target group. + +First, find available EC2 instances in your VPC: + +``` +$ INSTANCES=$(aws ec2 describe-instances \ + --filters "Name=vpc-id,Values=$VPC_ID" "Name=instance-state-name,Values=running" \ + --query "Reservations[*].Instances[*].InstanceId" --output text) +``` + +Then, register the instances with your target group: + +``` +$ aws elbv2 register-targets \ + --target-group-arn "$TARGET_GROUP_ARN" \ + --targets Id=$INSTANCES +``` + +Replace the instance IDs with your actual instance IDs. You can register multiple instances at once by specifying multiple `Id` parameters. + +## Create a listener + +A listener checks for connection requests using the protocol and port you configure. In this section, you'll create an HTTP listener for your load balancer. + +``` +$ LISTENER_ARN=$(aws elbv2 create-listener \ + --load-balancer-arn "$LOAD_BALANCER_ARN" \ + --protocol HTTP \ + --port 80 \ + --default-actions Type=forward,TargetGroupArn="$TARGET_GROUP_ARN" \ + --query "Listeners[0].ListenerArn" --output text) +$ echo "Created listener: $LISTENER_ARN" +``` + +This command creates an HTTP listener on port 80 that forwards requests to your target group. The listener ARN is stored in the `LISTENER_ARN` variable for later use. + +## Verify your configuration + +After setting up your load balancer, target group, and listener, you should verify that everything is working correctly. + +**Check target health** + +Verify the health of your registered targets: + +``` +$ aws elbv2 describe-target-health --target-group-arn "$TARGET_GROUP_ARN" +``` + +This command shows the health status of each registered target. If your targets are healthy, you should see a status of `healthy`. If they're unhealthy, check that your instances are running and that the security groups allow traffic on port 80. + +**Troubleshooting unhealthy targets** + +If your targets are unhealthy, check the following: + +1. Ensure your instances are running and the web server is active +2. Verify that the security group for your instances allows inbound traffic from the load balancer +3. Check that the health check path exists and returns a 200 OK response +4. Review the health check settings to ensure they're appropriate for your application + +**Get the load balancer DNS name** + +Retrieve the DNS name of your load balancer: + +``` +$ LB_DNS=$(aws elbv2 describe-load-balancers \ + --load-balancer-arns "$LOAD_BALANCER_ARN" \ + --query "LoadBalancers[0].DNSName" --output text) +$ echo "Load Balancer DNS Name: $LB_DNS" +``` + +You can use this DNS name to access your application through the load balancer. Open a web browser and enter the DNS name to test your load balancer. + +## Add an HTTPS listener (optional) + +For secure communication, you can add an HTTPS listener to your load balancer. This requires an SSL/TLS certificate. + +**Create or import an SSL certificate** + +Before creating an HTTPS listener, you need an SSL certificate. You can create or import a certificate using AWS Certificate Manager (ACM): + +``` +$ CERTIFICATE_ARN=$(aws acm request-certificate \ + --domain-name example.com \ + --validation-method DNS \ + --query "CertificateArn" --output text) +$ echo "Certificate ARN: $CERTIFICATE_ARN" +``` + +Replace `example.com` with your domain name. You'll need to complete the domain validation process before using the certificate. + +**Create an HTTPS listener** + +After obtaining a certificate, create an HTTPS listener: + +``` +$ aws elbv2 create-listener \ + --load-balancer-arn "$LOAD_BALANCER_ARN" \ + --protocol HTTPS \ + --port 443 \ + --certificates CertificateArn="$CERTIFICATE_ARN" \ + --default-actions Type=forward,TargetGroupArn="$TARGET_GROUP_ARN" +``` + +This command creates an HTTPS listener on port 443 that uses your SSL certificate and forwards requests to your target group. + +## Add path-based routing (optional) + +Path-based routing allows you to route requests to different target groups based on the URL path. In this section, you'll create a rule that forwards requests with a specific path pattern to a different target group. + +**Create another target group** + +First, create a new target group for handling specific paths: + +``` +$ IMAGE_TARGET_GROUP_ARN=$(aws elbv2 create-target-group \ + --name "my-images" \ + --protocol HTTP \ + --port 80 \ + --vpc-id "$VPC_ID" \ + --query "TargetGroups[0].TargetGroupArn" --output text) +$ echo "Created image target group: $IMAGE_TARGET_GROUP_ARN" +``` + +**Register targets with the new target group** + +Register your instance with the new target group: + +``` +$ aws elbv2 register-targets \ + --target-group-arn "$IMAGE_TARGET_GROUP_ARN" \ + --targets Id=$INSTANCES +``` + +**Create a rule for path-based routing** + +Create a rule that forwards requests with a specific path pattern to the new target group: + +``` +$ aws elbv2 create-rule \ + --listener-arn "$LISTENER_ARN" \ + --priority 10 \ + --conditions Field=path-pattern,Values='/img/*' \ + --actions Type=forward,TargetGroupArn="$IMAGE_TARGET_GROUP_ARN" +``` + +This rule forwards requests with paths that start with `/img/` to your image target group. All other requests are handled by the default action defined in the listener. + +## Clean up resources + +When you're done with this tutorial, you should clean up the resources you created to avoid incurring charges. + +**Delete the listener** + +``` +$ aws elbv2 delete-listener --listener-arn "$LISTENER_ARN" +``` + +**Delete the load balancer** + +``` +$ aws elbv2 delete-load-balancer --load-balancer-arn "$LOAD_BALANCER_ARN" +``` + +Wait for the load balancer to be deleted: + +``` +$ aws elbv2 wait load-balancers-deleted --load-balancer-arns "$LOAD_BALANCER_ARN" +``` + +**Delete the target groups** + +``` +$ aws elbv2 delete-target-group --target-group-arn "$TARGET_GROUP_ARN" + +# If you created an image target group +$ aws elbv2 delete-target-group --target-group-arn "$IMAGE_TARGET_GROUP_ARN" +``` + +## Going to production + +This tutorial is designed to help you learn how to use the Elastic Load Balancing API, not to create a production-ready deployment. Before using these concepts in a production environment, consider the following best practices: + +### Security considerations + +1. **Use HTTPS instead of HTTP**: Always use HTTPS in production to encrypt data in transit. +2. **Restrict security group rules**: Limit inbound traffic to specific IP ranges or security groups. +3. **Implement AWS WAF**: Consider using AWS WAF with your Application Load Balancer for additional protection against common web exploits. +4. **Enable access logging**: Configure access logs to track requests to your load balancer. + +### Reliability and scalability considerations + +1. **Use Auto Scaling groups**: Integrate with Auto Scaling to automatically adjust capacity based on demand. +2. **Configure appropriate health checks**: Customize health checks to accurately reflect the health of your application. +3. **Enable cross-zone load balancing**: Ensure traffic is distributed evenly across all Availability Zones. +4. **Set appropriate deregistration delays**: Configure connection draining to allow in-flight requests to complete when instances are deregistered. + +### Monitoring considerations + +1. **Set up CloudWatch alarms**: Create alarms for key metrics like unhealthy host count and request count. +2. **Enable detailed monitoring**: Consider enabling detailed monitoring for more granular metrics. +3. **Implement request tracing**: Use AWS X-Ray to trace requests through your application. + +For more information on building production-ready architectures, refer to: +- [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) +- [AWS Security Best Practices](https://aws.amazon.com/architecture/security-identity-compliance/) +- [Elastic Load Balancing Best Practices](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers-best-practices.html) + + +## Next steps + +Now that you've learned how to create and configure an Application Load Balancer using the AWS CLI, you might want to explore these related topics: + +* [Configure health checks for your target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-health-checks.html) +* [Use sticky sessions with your load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/sticky-sessions.html) +* [Configure access logs for your load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html) +* [Monitor your load balancer with CloudWatch](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-cloudwatch-metrics.html) +* [Create a Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/network-load-balancer-getting-started.html) +* [Create a Gateway Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/gateway/getting-started.html) diff --git a/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.sh b/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.sh new file mode 100755 index 00000000..d88e6a75 --- /dev/null +++ b/tuts/058-elastic-load-balancing-gs/elastic-load-balancing-gs.sh @@ -0,0 +1,245 @@ +#!/bin/bash + +# Elastic Load Balancing Getting Started Script - v2 +# This script creates an Application Load Balancer with HTTP listener and target group + +# Set up logging +LOG_FILE="elb-script-v2.log" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "Starting Elastic Load Balancing setup script at $(date)" +echo "All commands and outputs will be logged to $LOG_FILE" + +# Function to handle errors +handle_error() { + echo "ERROR: $1" + echo "Attempting to clean up resources..." + cleanup_resources + exit 1 +} + +# Function to check command success +check_command() { + if echo "$1" | grep -i "error" > /dev/null; then + handle_error "$1" + fi +} + +# Function to clean up resources +cleanup_resources() { + echo "Cleaning up resources in reverse order..." + + if [ -n "$LISTENER_ARN" ]; then + echo "Deleting listener: $LISTENER_ARN" + aws elbv2 delete-listener --listener-arn "$LISTENER_ARN" + fi + + if [ -n "$LOAD_BALANCER_ARN" ]; then + echo "Deleting load balancer: $LOAD_BALANCER_ARN" + aws elbv2 delete-load-balancer --load-balancer-arn "$LOAD_BALANCER_ARN" + + # Wait for load balancer to be deleted before deleting target group + echo "Waiting for load balancer to be deleted..." + aws elbv2 wait load-balancers-deleted --load-balancer-arns "$LOAD_BALANCER_ARN" + fi + + if [ -n "$TARGET_GROUP_ARN" ]; then + echo "Deleting target group: $TARGET_GROUP_ARN" + aws elbv2 delete-target-group --target-group-arn "$TARGET_GROUP_ARN" + fi + + # Add a delay before attempting to delete the security group + # to ensure all ELB resources are fully deleted + if [ -n "$SECURITY_GROUP_ID" ]; then + echo "Waiting 30 seconds before deleting security group to ensure all dependencies are removed..." + sleep 30 + + echo "Deleting security group: $SECURITY_GROUP_ID" + SG_DELETE_OUTPUT=$(aws ec2 delete-security-group --group-id "$SECURITY_GROUP_ID" 2>&1) + + # If there's still a dependency issue, retry a few times + RETRY_COUNT=0 + MAX_RETRIES=5 + while echo "$SG_DELETE_OUTPUT" | grep -i "DependencyViolation" > /dev/null && [ $RETRY_COUNT -lt $MAX_RETRIES ]; do + RETRY_COUNT=$((RETRY_COUNT+1)) + echo "Security group still has dependencies. Retrying in 30 seconds... (Attempt $RETRY_COUNT of $MAX_RETRIES)" + sleep 30 + SG_DELETE_OUTPUT=$(aws ec2 delete-security-group --group-id "$SECURITY_GROUP_ID" 2>&1) + done + + if echo "$SG_DELETE_OUTPUT" | grep -i "error" > /dev/null; then + echo "WARNING: Could not delete security group: $SECURITY_GROUP_ID" + echo "You may need to delete it manually using: aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID" + else + echo "Security group deleted successfully." + fi + fi +} + +# Generate a random identifier for resource names +RANDOM_ID=$(openssl rand -hex 4) +RESOURCE_PREFIX="elb-demo-${RANDOM_ID}" + +# Step 1: Verify AWS CLI support for Elastic Load Balancing +echo "Verifying AWS CLI support for Elastic Load Balancing..." +aws elbv2 help > /dev/null 2>&1 +if [ $? -ne 0 ]; then + handle_error "AWS CLI does not support elbv2 commands. Please update your AWS CLI." +fi + +# Step 2: Get VPC ID and subnet information +echo "Retrieving VPC information..." +VPC_INFO=$(aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" --query "Vpcs[0].VpcId" --output text) +check_command "$VPC_INFO" +VPC_ID=$VPC_INFO +echo "Using VPC: $VPC_ID" + +# Get two subnets from different Availability Zones +echo "Retrieving subnet information..." +SUBNET_INFO=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query "Subnets[0:2].SubnetId" --output text) +check_command "$SUBNET_INFO" + +# Convert space-separated list to array +read -r -a SUBNETS <<< "$SUBNET_INFO" +if [ ${#SUBNETS[@]} -lt 2 ]; then + handle_error "Need at least 2 subnets in different Availability Zones. Found: ${#SUBNETS[@]}" +fi + +echo "Using subnets: ${SUBNETS[0]} and ${SUBNETS[1]}" + +# Step 3: Create a security group for the load balancer +echo "Creating security group for the load balancer..." +SG_INFO=$(aws ec2 create-security-group \ + --group-name "${RESOURCE_PREFIX}-sg" \ + --description "Security group for ELB demo" \ + --vpc-id "$VPC_ID" \ + --query "GroupId" --output text) +check_command "$SG_INFO" +SECURITY_GROUP_ID=$SG_INFO +echo "Created security group: $SECURITY_GROUP_ID" + +# Add inbound rule to allow HTTP traffic +echo "Adding inbound rule to allow HTTP traffic..." +aws ec2 authorize-security-group-ingress \ + --group-id "$SECURITY_GROUP_ID" \ + --protocol tcp \ + --port 80 \ + --cidr "0.0.0.0/0" > /dev/null +# Note: In production, you should restrict the CIDR range to specific IP addresses + +# Step 4: Create the load balancer +echo "Creating Application Load Balancer..." +LB_INFO=$(aws elbv2 create-load-balancer \ + --name "${RESOURCE_PREFIX}-lb" \ + --subnets "${SUBNETS[0]}" "${SUBNETS[1]}" \ + --security-groups "$SECURITY_GROUP_ID" \ + --query "LoadBalancers[0].LoadBalancerArn" --output text) +check_command "$LB_INFO" +LOAD_BALANCER_ARN=$LB_INFO +echo "Created load balancer: $LOAD_BALANCER_ARN" + +# Wait for the load balancer to be active +echo "Waiting for load balancer to become active..." +aws elbv2 wait load-balancer-available --load-balancer-arns "$LOAD_BALANCER_ARN" + +# Step 5: Create a target group +echo "Creating target group..." +TG_INFO=$(aws elbv2 create-target-group \ + --name "${RESOURCE_PREFIX}-targets" \ + --protocol HTTP \ + --port 80 \ + --vpc-id "$VPC_ID" \ + --target-type instance \ + --query "TargetGroups[0].TargetGroupArn" --output text) +check_command "$TG_INFO" +TARGET_GROUP_ARN=$TG_INFO +echo "Created target group: $TARGET_GROUP_ARN" + +# Step 6: Find EC2 instances to register as targets +echo "Looking for available EC2 instances to register as targets..." +INSTANCES=$(aws ec2 describe-instances \ + --filters "Name=vpc-id,Values=$VPC_ID" "Name=instance-state-name,Values=running" \ + --query "Reservations[*].Instances[*].InstanceId" --output text) +check_command "$INSTANCES" + +# Convert space-separated list to array +read -r -a INSTANCE_IDS <<< "$INSTANCES" + +if [ ${#INSTANCE_IDS[@]} -eq 0 ]; then + echo "No running instances found in VPC $VPC_ID." + echo "You will need to register targets manually after launching instances." +else + # Step 7: Register targets with the target group (up to 2 instances) + echo "Registering targets with the target group..." + TARGET_ARGS="" + for i in "${!INSTANCE_IDS[@]}"; do + if [ "$i" -lt 2 ]; then # Register up to 2 instances + TARGET_ARGS="$TARGET_ARGS Id=${INSTANCE_IDS[$i]} " + fi + done + + if [ -n "$TARGET_ARGS" ]; then + aws elbv2 register-targets \ + --target-group-arn "$TARGET_GROUP_ARN" \ + --targets $TARGET_ARGS + echo "Registered instances: $TARGET_ARGS" + fi +fi + +# Step 8: Create a listener +echo "Creating HTTP listener..." +LISTENER_INFO=$(aws elbv2 create-listener \ + --load-balancer-arn "$LOAD_BALANCER_ARN" \ + --protocol HTTP \ + --port 80 \ + --default-actions Type=forward,TargetGroupArn="$TARGET_GROUP_ARN" \ + --query "Listeners[0].ListenerArn" --output text) +check_command "$LISTENER_INFO" +LISTENER_ARN=$LISTENER_INFO +echo "Created listener: $LISTENER_ARN" + +# Step 9: Verify target health +echo "Verifying target health..." +aws elbv2 describe-target-health --target-group-arn "$TARGET_GROUP_ARN" + +# Display load balancer DNS name +LB_DNS=$(aws elbv2 describe-load-balancers \ + --load-balancer-arns "$LOAD_BALANCER_ARN" \ + --query "LoadBalancers[0].DNSName" --output text) +check_command "$LB_DNS" + +echo "" +echo "==============================================" +echo "SETUP COMPLETE" +echo "==============================================" +echo "Load Balancer DNS Name: $LB_DNS" +echo "" +echo "Resources created:" +echo "- Load Balancer: $LOAD_BALANCER_ARN" +echo "- Target Group: $TARGET_GROUP_ARN" +echo "- Listener: $LISTENER_ARN" +echo "- Security Group: $SECURITY_GROUP_ID" +echo "" + +# Ask user if they want to clean up resources +echo "==============================================" +echo "CLEANUP CONFIRMATION" +echo "==============================================" +echo "Do you want to clean up all created resources? (y/n): " +read -r CLEANUP_CHOICE + +if [[ "$CLEANUP_CHOICE" =~ ^[Yy] ]]; then + echo "Starting cleanup process..." + cleanup_resources + echo "Cleanup completed." +else + echo "Resources have been preserved." + echo "To clean up later, run the following commands:" + echo "aws elbv2 delete-listener --listener-arn $LISTENER_ARN" + echo "aws elbv2 delete-load-balancer --load-balancer-arn $LOAD_BALANCER_ARN" + echo "aws elbv2 wait load-balancers-deleted --load-balancer-arns $LOAD_BALANCER_ARN" + echo "aws elbv2 delete-target-group --target-group-arn $TARGET_GROUP_ARN" + echo "aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID" +fi + +echo "Script completed at $(date)" diff --git a/tuts/078-amazon-elastic-container-registry-gs/README.md b/tuts/078-amazon-elastic-container-registry-gs/README.md new file mode 100644 index 00000000..2293f620 --- /dev/null +++ b/tuts/078-amazon-elastic-container-registry-gs/README.md @@ -0,0 +1,5 @@ +# Amazon Elastic Container Registry getting started + +This tutorial guides you through the process of creating, pushing, pulling, and managing Docker container images with Amazon Elastic Container Registry (Amazon ECR) using the AWS Command Line Interface (AWS CLI). You'll learn how to create repositories, authenticate with Docker, and manage container images in a secure registry. + +You can either run the automated shell script (`amazon-elastic-container-registry-gs.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`amazon-elastic-container-registry-gs.md`) to understand each component in detail. Both approaches will help you understand how to use Amazon ECR as a fully managed container registry for storing and managing your Docker images. diff --git a/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.md b/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.md new file mode 100644 index 00000000..93464cae --- /dev/null +++ b/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.md @@ -0,0 +1,302 @@ +# Getting started with Amazon ECR using the AWS CLI + +This tutorial guides you through the process of creating, pushing, pulling, and managing Docker container images with Amazon Elastic Container Registry (ECR) using the AWS Command Line Interface (AWS CLI). + +## Prerequisites + +Before you begin this tutorial, make sure you have the following: + +1. The AWS CLI. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). +2. Configured your AWS CLI with appropriate credentials. Run `aws configure` if you haven't set up your credentials yet. +3. Docker installed on your local machine or EC2 instance. Visit the [Docker installation guide](https://docs.docker.com/engine/installation/) for instructions specific to your operating system. +4. Basic familiarity with Docker concepts and commands. +5. [Sufficient permissions](https://docs.aws.amazon.com/AmazonECR/latest/userguide/security-iam.html) to create and manage ECR resources in your AWS account. + +**Cost Information**: The cost of running this tutorial is minimal. Amazon ECR charges $0.10 per GB-month for storage and has data transfer costs that vary by region. For this tutorial, with a ~200MB image stored for a short time, the cost is less than $0.01. AWS Free Tier includes 500MB of storage per month for private repositories, which would cover this tutorial. See [Amazon ECR pricing](https://aws.amazon.com/ecr/pricing/) for more details. + +**Time to Complete**: Approximately 20-30 minutes. + +Before you start, set the environment variables for your AWS account ID and region: + +``` +$ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) +$ export AWS_REGION=$(aws configure get region) +``` + +## Create a Docker image + +First, you'll create a simple web application Docker image that you'll later push to Amazon ECR. + +**Create a Dockerfile** + +Create a file named `Dockerfile` with the following content: + +``` +FROM public.ecr.aws/amazonlinux/amazonlinux:latest + +# Install dependencies +RUN yum update -y && \ + yum install -y httpd + +# Install apache and write hello world message +RUN echo 'Hello World!' > /var/www/html/index.html + +# Configure apache +RUN echo 'mkdir -p /var/run/httpd' >> /root/run_apache.sh && \ + echo 'mkdir -p /var/lock/httpd' >> /root/run_apache.sh && \ + echo '/usr/sbin/httpd -D FOREGROUND' >> /root/run_apache.sh && \ + chmod 755 /root/run_apache.sh + +EXPOSE 80 + +CMD /root/run_apache.sh +``` + +This Dockerfile uses the Amazon Linux image from Amazon ECR Public Gallery as a base. It installs the Apache web server, creates a simple "Hello World" web page, and configures Apache to run in the foreground. + +**Build the Docker image** + +Build the Docker image using the following command: + +``` +$ docker build -t hello-app . +``` + +The `-t` flag tags your image with the name "hello-app". The dot at the end specifies that the Dockerfile is in the current directory. + +**Verify the image was created** + +List your Docker images to confirm that the image was created successfully: + +``` +$ docker images --filter reference=hello-app +``` + +You should see output similar to: + +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +hello-app latest abcd1234e567 1 minute ago 194MB +``` + +## Create an Amazon ECR repository + +Now that you have a Docker image, you need to create an Amazon ECR repository to store it. + +**Create a repository** + +Create a repository named "hello-app-repository" using the following command: + +``` +$ aws ecr create-repository --repository-name hello-app-repository +``` + +The command returns information about the newly created repository, including its Amazon Resource Name (ARN) and URI: + +``` +{ + "repository": { + "repositoryArn": "arn:aws:ecr:us-east-1:123456789012:repository/hello-app-repository", + "registryId": "123456789012", + "repositoryName": "hello-app-repository", + "repositoryUri": "123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-app-repository", + "createdAt": "2025-01-13T12:00:00.000Z", + "imageTagMutability": "MUTABLE", + "imageScanningConfiguration": { + "scanOnPush": false + }, + "encryptionConfiguration": { + "encryptionType": "AES256" + } + } +} +``` + +Take note of the `repositoryUri` value, as you'll need it in the next steps. + +## Authenticate to Amazon ECR + +Before you can push or pull images, you need to authenticate your Docker client to your Amazon ECR registry. + +**Get authentication token** + +Use the `get-login-password` command to retrieve an authentication token and authenticate your Docker client: + +``` +$ aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com +``` + +If successful, you'll see the output: + +``` +Login Succeeded +``` + +This authentication is valid for 12 hours, after which you'll need to authenticate again. + +## Push an image to Amazon ECR + +Now that you've authenticated to Amazon ECR, you can tag and push your Docker image to your repository. + +**Tag the image** + +Tag your local image with the Amazon ECR repository URI: + +``` +$ docker tag hello-app:latest $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/hello-app-repository:latest +``` + +This command doesn't produce any output, but it creates a new tag for your image that points to your Amazon ECR repository. + +**Push the image** + +Push the tagged image to Amazon ECR: + +``` +$ docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/hello-app-repository:latest +``` + +You'll see output showing the progress of the push operation: + +``` +The push refers to repository [123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-app-repository] +abcd1234: Pushed +5678efgh: Pushed +90ijklmn: Pushed +opqr1234: Pushed +latest: digest: sha256:abcd1234efgh5678ijkl90mnopqr1234stuvwxyz1234567890abcdefghijkl size: 6774 +``` + +## Pull an image from Amazon ECR + +After pushing an image to Amazon ECR, you can pull it to any machine that has Docker installed and appropriate permissions. + +**Remove the local image** + +To demonstrate pulling from ECR, first remove the local tagged image: + +``` +$ docker rmi $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/hello-app-repository:latest +``` + +**Pull the image** + +Now pull the image from Amazon ECR: + +``` +$ docker pull $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/hello-app-repository:latest +``` + +You'll see output showing the progress of the pull operation: + +``` +latest: Pulling from hello-app-repository +abcd1234: Pull complete +5678efgh: Pull complete +90ijklmn: Pull complete +opqr1234: Pull complete +Digest: sha256:abcd1234efgh5678ijkl90mnopqr1234stuvwxyz1234567890abcdefghijkl +Status: Downloaded newer image for 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-app-repository:latest +``` + +## Delete resources + +When you're done with the tutorial, you should clean up the resources you created to avoid incurring any unnecessary charges. + +**Delete the image** + +Delete the image from your Amazon ECR repository: + +``` +$ aws ecr batch-delete-image --repository-name hello-app-repository --image-ids imageTag=latest +``` + +The command returns information about the deleted image: + +``` +{ + "imageIds": [ + { + "imageDigest": "sha256:abcd1234efgh5678ijkl90mnopqr1234stuvwxyz1234567890abcdefghijkl", + "imageTag": "latest" + } + ], + "failures": [] +} +``` + +**Delete the repository** + +Delete the Amazon ECR repository: + +``` +$ aws ecr delete-repository --repository-name hello-app-repository --force +``` + +The `--force` flag is required because the repository contained images. The command returns information about the deleted repository: + +``` +{ + "repository": { + "repositoryArn": "arn:aws:ecr:us-east-1:123456789012:repository/hello-app-repository", + "registryId": "123456789012", + "repositoryName": "hello-app-repository", + "repositoryUri": "123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-app-repository", + "createdAt": "2025-01-13T12:00:00.000Z", + "imageTagMutability": "MUTABLE" + } +} +``` + +**Remove local Docker images** + +Finally, remove the local Docker image: + +``` +$ docker rmi hello-app:latest +``` + +## Going to production + +This tutorial is designed to teach you the basics of using Amazon ECR with the AWS CLI. For production environments, consider these additional best practices: + +### Security best practices + +1. **Enable image scanning** - Scan your container images for security vulnerabilities: + ``` + $ aws ecr create-repository --repository-name hello-app-repository --image-scanning-configuration scanOnPush=true + ``` + +2. **Configure image tag immutability** - Prevent overwriting existing image tags: + ``` + $ aws ecr create-repository --repository-name hello-app-repository --image-tag-mutability IMMUTABLE + ``` + +3. **Implement repository policies** - Restrict access to your repositories using IAM policies. + +4. **Use non-root users** - In production Dockerfiles, create and use non-root users to run your applications. + +### Architecture best practices + +1. **Implement lifecycle policies** - Automatically manage image retention to control costs and repository size. + +2. **Set up cross-region replication** - For disaster recovery, replicate critical images across regions. + +3. **Optimize image size** - Use multi-stage builds to create smaller, more efficient container images. + +4. **Automate with infrastructure as code** - Use AWS CloudFormation or AWS CDK to automate repository creation and configuration. + +For more information on production best practices, refer to: +- [Amazon ECR Best Practices Guide](https://docs.aws.amazon.com/AmazonECR/latest/userguide/best-practices.html) +- [AWS Well-Architected Framework](https://aws.amazon.com/architecture/well-architected/) +- [AWS Security Best Practices](https://aws.amazon.com/architecture/security-identity-compliance/) + +## Next steps + +Now that you've learned the basics of using Amazon ECR with the AWS CLI, you can explore more advanced features: + +- [Setting up image scanning](https://docs.aws.amazon.com/AmazonECR/latest/userguide/image-scanning.html) to identify software vulnerabilities in your container images +- [Configuring lifecycle policies](https://docs.aws.amazon.com/AmazonECR/latest/userguide/lifecycle_policy.html) to manage the lifecycle of images in your repositories +- [Setting up cross-region replication](https://docs.aws.amazon.com/AmazonECR/latest/userguide/replication.html) to copy images to repositories in different AWS Regions +- [Using Amazon ECR with Amazon ECS](https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_on_ECS.html) to deploy your container images on Amazon Elastic Container Service +- [Using Amazon ECR with Amazon EKS](https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_on_EKS.html) to deploy your container images on Amazon Elastic Kubernetes Service diff --git a/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.sh b/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.sh new file mode 100755 index 00000000..aad9bb9a --- /dev/null +++ b/tuts/078-amazon-elastic-container-registry-gs/amazon-elastic-container-registry-gs.sh @@ -0,0 +1,258 @@ +#!/bin/bash + +# Amazon ECR Getting Started Script +# This script demonstrates the lifecycle of a Docker image in Amazon ECR + +# Set up logging +LOG_FILE="ecr-tutorial.log" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "===================================================" +echo "Amazon ECR Getting Started Tutorial" +echo "===================================================" +echo "This script will:" +echo "1. Create a Docker image" +echo "2. Create an Amazon ECR repository" +echo "3. Authenticate to Amazon ECR" +echo "4. Push the image to Amazon ECR" +echo "5. Pull the image from Amazon ECR" +echo "6. Clean up resources (optional)" +echo "===================================================" + +# Check prerequisites +echo "Checking prerequisites..." + +# Check if AWS CLI is installed +if ! command -v aws &> /dev/null; then + echo "ERROR: AWS CLI is not installed. Please install it before running this script." + echo "Visit https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html for installation instructions." + exit 1 +fi + +# Check if AWS CLI is configured +if ! aws sts get-caller-identity &> /dev/null; then + echo "ERROR: AWS CLI is not configured properly. Please run 'aws configure' to set up your credentials." + exit 1 +fi + +# Check if Docker is installed +if ! command -v docker &> /dev/null; then + echo "ERROR: Docker is not installed. Please install Docker before running this script." + echo "Visit https://docs.docker.com/get-docker/ for installation instructions." + exit 1 +fi + +# Check if Docker daemon is running +if ! docker info &> /dev/null; then + echo "ERROR: Docker daemon is not running. Please start Docker and try again." + exit 1 +fi + +echo "All prerequisites met." + +# Initialize variables +REPO_URI="" +TIMEOUT_CMD="timeout 300" # 5-minute timeout for long-running commands + +# Function to handle errors +handle_error() { + echo "ERROR: $1" + echo "Check the log file for details: $LOG_FILE" + + echo "===================================================" + echo "Resources created:" + echo "- Docker image: hello-world (local)" + if [ -n "$REPO_URI" ]; then + echo "- ECR Repository: hello-repository" + echo "- ECR Image: $REPO_URI:latest" + fi + echo "===================================================" + + echo "Attempting to clean up resources..." + cleanup + exit 1 +} + +# Function to clean up resources +cleanup() { + echo "===================================================" + echo "Cleaning up resources..." + + # Delete the image from ECR if it exists + if [ -n "$REPO_URI" ]; then + echo "Deleting image from ECR repository..." + aws ecr batch-delete-image --repository-name hello-repository --image-ids imageTag=latest || echo "Failed to delete image, it may not exist or may have already been deleted." + fi + + # Delete the ECR repository if it exists + if [ -n "$REPO_URI" ]; then + echo "Deleting ECR repository..." + aws ecr delete-repository --repository-name hello-repository --force || echo "Failed to delete repository, it may not exist or may have already been deleted." + fi + + # Remove local Docker image + echo "Removing local Docker image..." + docker rmi hello-world:latest 2>/dev/null || echo "Failed to remove local image, it may not exist or may have already been deleted." + if [ -n "$REPO_URI" ]; then + docker rmi "$REPO_URI:latest" 2>/dev/null || echo "Failed to remove tagged image, it may not exist or may have already been deleted." + fi + + echo "Cleanup completed." + echo "===================================================" +} + +# Step 1: Create a Docker image +echo "Step 1: Creating a Docker image" + +# Create Dockerfile +echo "Creating Dockerfile..." +cat > Dockerfile << 'EOF' +FROM public.ecr.aws/amazonlinux/amazonlinux:latest + +# Install dependencies +RUN yum update -y && \ + yum install -y httpd + +# Install apache and write hello world message +RUN echo 'Hello World!' > /var/www/html/index.html + +# Configure apache +RUN echo 'mkdir -p /var/run/httpd' >> /root/run_apache.sh && \ + echo 'mkdir -p /var/lock/httpd' >> /root/run_apache.sh && \ + echo '/usr/sbin/httpd -D FOREGROUND' >> /root/run_apache.sh && \ + chmod 755 /root/run_apache.sh + +EXPOSE 80 + +CMD /root/run_apache.sh +EOF + +# Build Docker image +echo "Building Docker image..." +$TIMEOUT_CMD docker build -t hello-world . || handle_error "Failed to build Docker image or operation timed out after 5 minutes" + +# Verify image was created +echo "Verifying Docker image..." +docker images --filter reference=hello-world || handle_error "Failed to list Docker images" + +echo "Docker image created successfully." + +# Step 2: Create an Amazon ECR repository +echo "Step 2: Creating an Amazon ECR repository" + +# Get AWS account ID +echo "Getting AWS account ID..." +AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) +if [[ -z "$AWS_ACCOUNT_ID" || "$AWS_ACCOUNT_ID" == *"error"* ]]; then + handle_error "Failed to get AWS account ID. Make sure your AWS credentials are configured correctly." +fi +echo "AWS Account ID: $AWS_ACCOUNT_ID" + +# Get current region +AWS_REGION=$(aws configure get region) +if [[ -z "$AWS_REGION" ]]; then + AWS_REGION="us-east-1" # Default to us-east-1 if no region is configured + echo "No AWS region configured, defaulting to $AWS_REGION" +else + echo "Using AWS region: $AWS_REGION" +fi + +# Create ECR repository +echo "Creating ECR repository..." +REPO_RESULT=$(aws ecr create-repository --repository-name hello-repository) +if [[ -z "$REPO_RESULT" || "$REPO_RESULT" == *"error"* ]]; then + handle_error "Failed to create ECR repository" +fi + +# Extract repository URI +REPO_URI="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/hello-repository" +echo "Repository URI: $REPO_URI" + +# Step 3: Authenticate to Amazon ECR +echo "Step 3: Authenticating to Amazon ECR" + +echo "Getting ECR login password..." +aws ecr get-login-password --region "$AWS_REGION" | docker login --username AWS --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com" || handle_error "Failed to authenticate to ECR" + +echo "Successfully authenticated to ECR." + +# Step 4: Push the image to Amazon ECR +echo "Step 4: Pushing the image to Amazon ECR" + +# Tag the image +echo "Tagging Docker image..." +docker tag hello-world:latest "$REPO_URI:latest" || handle_error "Failed to tag Docker image" + +# Push the image with timeout +echo "Pushing image to ECR..." +$TIMEOUT_CMD docker push "$REPO_URI:latest" || handle_error "Failed to push image to ECR or operation timed out after 5 minutes" + +echo "Successfully pushed image to ECR." + +# Step 5: Pull the image from Amazon ECR +echo "Step 5: Pulling the image from Amazon ECR" + +# Remove local image to ensure we're pulling from ECR +echo "Removing local tagged image..." +docker rmi "$REPO_URI:latest" || echo "Warning: Failed to remove local tagged image" + +# Pull the image with timeout +echo "Pulling image from ECR..." +$TIMEOUT_CMD docker pull "$REPO_URI:latest" || handle_error "Failed to pull image from ECR or operation timed out after 5 minutes" + +echo "Successfully pulled image from ECR." + +# List resources created +echo "===================================================" +echo "Resources created:" +echo "- Docker image: hello-world (local)" +echo "- ECR Repository: hello-repository" +echo "- ECR Image: $REPO_URI:latest" +echo "===================================================" + +# Ask user if they want to clean up resources +echo "" +echo "===========================================" +echo "CLEANUP CONFIRMATION" +echo "===========================================" +echo "Do you want to clean up all created resources? (y/n): " +read -r CLEANUP_CHOICE + +if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + # Step 6: Delete the image from ECR + echo "Step 6: Deleting the image from ECR" + + DELETE_IMAGE_RESULT=$(aws ecr batch-delete-image --repository-name hello-repository --image-ids imageTag=latest) + if [[ -z "$DELETE_IMAGE_RESULT" || "$DELETE_IMAGE_RESULT" == *"error"* ]]; then + echo "Warning: Failed to delete image from ECR" + else + echo "Successfully deleted image from ECR." + fi + + # Step 7: Delete the ECR repository + echo "Step 7: Deleting the ECR repository" + + DELETE_REPO_RESULT=$(aws ecr delete-repository --repository-name hello-repository --force) + if [[ -z "$DELETE_REPO_RESULT" || "$DELETE_REPO_RESULT" == *"error"* ]]; then + echo "Warning: Failed to delete ECR repository" + else + echo "Successfully deleted ECR repository." + fi + + # Remove local Docker images + echo "Removing local Docker images..." + docker rmi hello-world:latest 2>/dev/null || echo "Warning: Failed to remove local image" + + echo "All resources have been cleaned up." +else + echo "Resources were not cleaned up. You can manually clean up later with:" + echo "aws ecr batch-delete-image --repository-name hello-repository --image-ids imageTag=latest" + echo "aws ecr delete-repository --repository-name hello-repository --force" + echo "docker rmi hello-world:latest" + echo "docker rmi $REPO_URI:latest" +fi + +echo "===================================================" +echo "Tutorial completed!" +echo "Log file: $LOG_FILE" +echo "===================================================" diff --git a/tuts/080-aws-step-functions-gs/README.md b/tuts/080-aws-step-functions-gs/README.md new file mode 100644 index 00000000..87917125 --- /dev/null +++ b/tuts/080-aws-step-functions-gs/README.md @@ -0,0 +1,5 @@ +# AWS Step Functions getting started + +This tutorial guides you through creating and running your first AWS Step Functions state machine using the AWS Command Line Interface (AWS CLI). You'll learn how to create a simple workflow, execute it with different inputs, and integrate with Amazon Comprehend for sentiment analysis. + +You can either run the automated shell script (`aws-step-functions-gs.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`aws-step-functions-gs.md`) to understand each component in detail. Both approaches will help you understand how to build serverless workflows using AWS Step Functions to coordinate multiple AWS services. diff --git a/tuts/080-aws-step-functions-gs/aws-step-functions-gs.md b/tuts/080-aws-step-functions-gs/aws-step-functions-gs.md new file mode 100644 index 00000000..97f5da54 --- /dev/null +++ b/tuts/080-aws-step-functions-gs/aws-step-functions-gs.md @@ -0,0 +1,682 @@ +# Getting started with AWS Step Functions using the AWS CLI + +This tutorial guides you through creating and running your first AWS Step Functions state machine using the AWS Command Line Interface (AWS CLI). You'll learn how to create a simple workflow, execute it with different inputs, and integrate with Amazon Comprehend for sentiment analysis. + +## Prerequisites + +Before you begin this tutorial, make sure you have the following: + +1. The AWS CLI installed and configured with appropriate credentials. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). +2. Basic understanding of JSON and state machines. +3. [Sufficient permissions](https://docs.aws.amazon.com/step-functions/latest/dg/security_iam_service-with-iam.html) to create and manage Step Functions resources in your AWS account. +4. Approximately 20-30 minutes to complete the tutorial. + +To run the tutorial successfully, set up the AWS CLI to use an AWS Region that Amazon Comprehend is available in. For more information, see [Amazon Comprehend endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/comprehend.html). + +**Cost information**: The resources created in this tutorial will incur minimal costs (less than $0.01) if you follow the cleanup instructions. Step Functions charges $0.025 per 1,000 state transitions for Standard workflows, and Amazon Comprehend charges $0.0001 per sentiment analysis request. + +## Create an IAM role for Step Functions + +First, you need to create an IAM role that allows Step Functions to execute. This role will be used by your state machine to access AWS services. + +**Create a trust policy document** + +Create a JSON file that defines the trust relationship for the Step Functions service: + +```bash +cat > step-functions-trust-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "states.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EOF +``` + +This trust policy allows the Step Functions service to assume this role. + +**Create the IAM role** + +Now, create the IAM role using the trust policy: + +```bash +aws iam create-role \ + --role-name StepFunctionsHelloWorldRole \ + --assume-role-policy-document file://step-functions-trust-policy.json +``` + +The output will include details about the newly created role, including its ARN (Amazon Resource Name). + +**Create a policy for Step Functions** + +Create a policy that grants permissions for Step Functions operations: + +```bash +cat > stepfunctions-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "states:*" + ], + "Resource": "*" + } + ] +} +EOF + +aws iam create-policy \ + --policy-name StepFunctionsPolicy \ + --policy-document file://stepfunctions-policy.json +``` + +**Attach the policy to the role** + +Attach the policy to the role you created: + +```bash +POLICY_ARN=$(aws iam list-policies \ + --query "Policies[?PolicyName=='StepFunctionsPolicy'].Arn" \ + --output text) + +aws iam attach-role-policy \ + --role-name StepFunctionsHelloWorldRole \ + --policy-arn $POLICY_ARN +``` + +This attaches the policy to the role, granting the necessary permissions. + +## Create your first state machine + +A state machine is a workflow defined using Amazon States Language (ASL), a JSON-based language. Let's create a simple "Hello World" state machine. + +**Define the state machine** + +Create a JSON file containing the state machine definition: + +```bash +cat > hello-world.json << 'EOF' +{ + "Comment": "A Hello World example of the Amazon States Language using a Pass state", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Result": { + "IsHelloWorldExample": true, + "ExecutionWaitTimeInSeconds": 10 + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "FailState" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + }, + "FailState": { + "Type": "Fail", + "Error": "NotHelloWorldExample", + "Cause": "The IsHelloWorldExample value was false" + } + } +} +EOF +``` + +This state machine includes several state types: +- A **Pass state** that sets initial variables +- A **Choice state** that makes a decision based on input +- A **Wait state** that pauses execution +- A **Parallel state** that runs tasks concurrently +- **Succeed** and **Fail** states that end the execution + +**Create the state machine** + +Now, create the state machine using the AWS CLI: + +```bash +ROLE_ARN=$(aws iam get-role \ + --role-name StepFunctionsHelloWorldRole \ + --query 'Role.Arn' \ + --output text) + +aws stepfunctions create-state-machine \ + --name MyFirstStateMachine \ + --definition file://hello-world.json \ + --role-arn $ROLE_ARN \ + --type STANDARD +``` + +The output will include the ARN of your new state machine. Save this ARN for later use. + +## Start your state machine execution + +Now that you've created a state machine, let's run it. + +**Start the execution** + +Start an execution of your state machine: + +```bash +STATE_MACHINE_ARN=$(aws stepfunctions list-state-machines \ + --query "stateMachines[?name=='MyFirstStateMachine'].stateMachineArn" \ + --output text) + +aws stepfunctions start-execution \ + --state-machine-arn $STATE_MACHINE_ARN \ + --name hello001 +``` + +The output will include the ARN of the execution and its start time. + +**Check the execution status** + +After waiting a few seconds for the execution to complete, check its status: + +```bash +EXECUTION_ARN=$(aws stepfunctions list-executions \ + --state-machine-arn $STATE_MACHINE_ARN \ + --query "executions[?name=='hello001'].executionArn" \ + --output text) + +aws stepfunctions describe-execution \ + --execution-arn $EXECUTION_ARN +``` + +The output will show details about the execution, including its status, input, and output. You should see that the execution has succeeded and the output contains the checkpoint message. + +## Process external input + +Let's modify the state machine to process external input instead of using hardcoded values. + +**Update the state machine definition** + +Create an updated state machine definition: + +```bash +cat > updated-hello-world.json << 'EOF' +{ + "Comment": "A Hello World example of the Amazon States Language using a Pass state", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Parameters": { + "IsHelloWorldExample.$": "$.hello_world", + "ExecutionWaitTimeInSeconds.$": "$.wait" + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "FailState" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + }, + "FailState": { + "Type": "Fail", + "Error": "NotHelloWorldExample", + "Cause": "The IsHelloWorldExample value was false" + } + } +} +EOF +``` + +The key difference in this updated definition is in the `SetVariables` state, which now uses the `Parameters` field with JSONPath references to pull values from the input. + +**Update the state machine** + +Update your state machine with the new definition: + +```bash +aws stepfunctions update-state-machine \ + --state-machine-arn $STATE_MACHINE_ARN \ + --definition file://updated-hello-world.json \ + --role-arn $ROLE_ARN +``` + +**Run the state machine with input** + +Create an input file: + +```bash +cat > input.json << 'EOF' +{ + "wait": 5, + "hello_world": true +} +EOF +``` + +Start an execution with this input: + +```bash +aws stepfunctions start-execution \ + --state-machine-arn $STATE_MACHINE_ARN \ + --name hello002 \ + --input file://input.json +``` + +After waiting a few seconds, check the execution status: + +```bash +EXECUTION2_ARN=$(aws stepfunctions list-executions \ + --state-machine-arn $STATE_MACHINE_ARN \ + --query "executions[?name=='hello002'].executionArn" \ + --output text) + +aws stepfunctions describe-execution \ + --execution-arn $EXECUTION2_ARN +``` + +You'll see that the execution succeeded and used the input values you provided. + +## Integrate Amazon Comprehend for sentiment analysis + +Now, let's enhance our state machine by integrating with Amazon Comprehend to perform sentiment analysis on text input. + +**Create a policy for Amazon Comprehend access** + +First, create a policy that allows access to Amazon Comprehend: + +```bash +cat > comprehend-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "comprehend:DetectSentiment" + ], + "Resource": "*" + } + ] +} +EOF + +aws iam create-policy \ + --policy-name DetectSentimentPolicy \ + --policy-document file://comprehend-policy.json +``` + +**Attach the policy to the role** + +Attach the Comprehend policy to your Step Functions role: + +```bash +COMPREHEND_POLICY_ARN=$(aws iam list-policies \ + --query "Policies[?PolicyName=='DetectSentimentPolicy'].Arn" \ + --output text) + +aws iam attach-role-policy \ + --role-name StepFunctionsHelloWorldRole \ + --policy-arn $COMPREHEND_POLICY_ARN +``` + +**Update the state machine with sentiment analysis** + +Create an updated state machine definition that includes sentiment analysis: + +```bash +cat > sentiment-hello-world.json << 'EOF' +{ + "Comment": "A Hello World example with sentiment analysis", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Parameters": { + "IsHelloWorldExample.$": "$.hello_world", + "ExecutionWaitTimeInSeconds.$": "$.wait", + "FeedbackComment.$": "$.feedback_comment" + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "DetectSentiment" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "DetectSentiment": { + "Type": "Task", + "Resource": "arn:aws:states:::aws-sdk:comprehend:detectSentiment", + "Parameters": { + "LanguageCode": "en", + "Text.$": "$.FeedbackComment" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + } + } +} +EOF +``` + +This updated definition adds a new `DetectSentiment` task state that uses the Amazon Comprehend service to analyze the sentiment of text provided in the input. + +**Update the state machine** + +Update your state machine with the new definition: + +```bash +aws stepfunctions update-state-machine \ + --state-machine-arn $STATE_MACHINE_ARN \ + --definition file://sentiment-hello-world.json \ + --role-arn $ROLE_ARN +``` + +**Run the state machine with sentiment analysis** + +Create an input file with a feedback comment: + +```bash +cat > sentiment-input.json << 'EOF' +{ + "hello_world": false, + "wait": 5, + "feedback_comment": "This getting started with Step Functions workshop is a challenge!" +} +EOF +``` + +Start an execution with this input: + +```bash +aws stepfunctions start-execution \ + --state-machine-arn $STATE_MACHINE_ARN \ + --name hello003 \ + --input file://sentiment-input.json +``` + +After waiting a few seconds, check the execution status: + +```bash +EXECUTION3_ARN=$(aws stepfunctions list-executions \ + --state-machine-arn $STATE_MACHINE_ARN \ + --query "executions[?name=='hello003'].executionArn" \ + --output text) + +aws stepfunctions describe-execution \ + --execution-arn $EXECUTION3_ARN +``` + +The output will show the sentiment analysis results, including the detected sentiment (POSITIVE, NEGATIVE, NEUTRAL, or MIXED) and confidence scores. + +## Clean up resources + +When you're finished with this tutorial, clean up the resources to avoid incurring additional charges. + +**Delete the state machine** + +Delete the state machine you created: + +```bash +aws stepfunctions delete-state-machine \ + --state-machine-arn $STATE_MACHINE_ARN +``` + +**Detach policies from the role** + +Detach the policies from the IAM role: + +```bash +aws iam detach-role-policy \ + --role-name StepFunctionsHelloWorldRole \ + --policy-arn $COMPREHEND_POLICY_ARN + +aws iam detach-role-policy \ + --role-name StepFunctionsHelloWorldRole \ + --policy-arn $POLICY_ARN +``` + +**Delete the policies** + +Delete the policies you created: + +```bash +aws iam delete-policy \ + --policy-arn $COMPREHEND_POLICY_ARN + +aws iam delete-policy \ + --policy-arn $POLICY_ARN +``` + +**Delete the role** + +Finally, delete the IAM role: + +```bash +aws iam delete-role \ + --role-name StepFunctionsHelloWorldRole +``` + +**Remove temporary files** + +Remove the temporary files created during this tutorial: + +```bash +rm -f hello-world.json updated-hello-world.json sentiment-hello-world.json step-functions-trust-policy.json comprehend-policy.json stepfunctions-policy.json input.json sentiment-input.json +``` +## Going to production + +This tutorial demonstrates basic Step Functions functionality for learning purposes. For production environments, consider these additional best practices: + +### Security best practices + +1. **Least privilege permissions**: The IAM policies in this tutorial use broad permissions. In production, scope permissions to specific resources and actions. + +2. **Encryption**: Configure KMS encryption for your state machine data: + ```bash + aws stepfunctions create-state-machine --encryption-configuration type=AWS_OWNED_KEY + ``` + +3. **Resource tagging**: Add tags to your resources for better organization and access control: + ```bash + aws stepfunctions create-state-machine --tags Key=Environment,Value=Production + ``` + +### Architecture best practices + +1. **Error handling**: Add Retry and Catch states to handle failures gracefully: + ```json + "DetectSentiment": { + "Type": "Task", + "Resource": "arn:aws:states:::aws-sdk:comprehend:detectSentiment", + "Retry": [ + { + "ErrorEquals": ["States.TaskFailed"], + "IntervalSeconds": 2, + "MaxAttempts": 3, + "BackoffRate": 2 + } + ], + "Catch": [ + { + "ErrorEquals": ["States.ALL"], + "Next": "ErrorHandler" + } + ], + "Next": "SuccessState" + } + ``` + +2. **Logging and monitoring**: Configure CloudWatch Logs and X-Ray tracing for observability. + +3. **Workflow type selection**: Choose between Standard and Express workflows based on your performance and cost requirements. + +For more information on production best practices, see: +- [AWS Step Functions Best Practices](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-best-practices.html) +- [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html) +- [Security Best Practices for AWS Step Functions](https://docs.aws.amazon.com/step-functions/latest/dg/security-best-practices.html) + +## Next steps + +Now that you've learned the basics of AWS Step Functions, explore these additional topics: + +1. [Creating workflows with AWS Step Functions](https://docs.aws.amazon.com/step-functions/latest/dg/creating-workflows.html) - Learn more about creating complex workflows. +2. [AWS Step Functions service integrations](https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html) - Discover how to integrate with other AWS services. +3. [Error handling in Step Functions](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html) - Learn about error handling strategies. +4. [Step Functions Express Workflows](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html) - Explore high-volume, event-processing workloads. +5. [Step Functions data processing patterns](https://docs.aws.amazon.com/step-functions/latest/dg/service-integration-patterns-data-processing.html) - Learn about common data processing patterns. diff --git a/tuts/080-aws-step-functions-gs/aws-step-functions-gs.sh b/tuts/080-aws-step-functions-gs/aws-step-functions-gs.sh new file mode 100755 index 00000000..a0e6abbd --- /dev/null +++ b/tuts/080-aws-step-functions-gs/aws-step-functions-gs.sh @@ -0,0 +1,752 @@ +#!/bin/bash + +# AWS Step Functions Getting Started Tutorial Script +# This script creates and runs a Step Functions state machine based on the AWS Step Functions Getting Started tutorial + +# Parse command line arguments +AUTO_CLEANUP=false +while [[ $# -gt 0 ]]; do + case $1 in + --auto-cleanup) + AUTO_CLEANUP=true + shift + ;; + -h|--help) + echo "Usage: $0 [--auto-cleanup] [--help]" + echo " --auto-cleanup: Automatically clean up resources without prompting" + echo " --help: Show this help message" + exit 0 + ;; + *) + echo "Unknown option: $1" + echo "Use --help for usage information" + exit 1 + ;; + esac +done + +# Set up logging +LOG_FILE="step-functions-tutorial.log" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "Starting AWS Step Functions Getting Started Tutorial..." +echo "Logging to $LOG_FILE" + +# Check if jq is available for better JSON parsing +if ! command -v jq &> /dev/null; then + echo "WARNING: jq is not installed. Using basic JSON parsing which may be less reliable." + echo "Consider installing jq for better error handling: brew install jq (macOS) or apt-get install jq (Ubuntu)" + USE_JQ=false +else + USE_JQ=true +fi + +# Use fixed region that supports Amazon Comprehend +CURRENT_REGION="us-west-2" +echo "Using fixed AWS region: $CURRENT_REGION (supports Amazon Comprehend)" + +# Set AWS CLI to use the fixed region for all commands +export AWS_DEFAULT_REGION="$CURRENT_REGION" + +# Amazon Comprehend is available in us-west-2, so we can always enable it +echo "Amazon Comprehend is available in region $CURRENT_REGION" +SKIP_COMPREHEND=false + +# Function to check for API errors in JSON response +check_api_error() { + local response="$1" + local operation="$2" + + if [[ "$USE_JQ" == "true" ]]; then + # Use jq for more reliable JSON parsing + if echo "$response" | jq -e '.Error' > /dev/null 2>&1; then + local error_message=$(echo "$response" | jq -r '.Error.Message // .Error.Code // "Unknown error"') + handle_error "$operation failed: $error_message" + fi + else + # Fallback to grep-based detection + if echo "$response" | grep -q '"Error":\|"error":'; then + handle_error "$operation failed: $response" + fi + fi +} + +# Function to wait for resource propagation with exponential backoff +wait_for_propagation() { + local resource_type="$1" + local wait_time="${2:-10}" + + echo "Waiting for $resource_type to propagate ($wait_time seconds)..." + sleep "$wait_time" +} + +# Function to handle errors +handle_error() { + echo "ERROR: $1" + echo "Resources created:" + if [ -n "$STATE_MACHINE_ARN" ]; then + echo "- State Machine: $STATE_MACHINE_ARN" + fi + if [ -n "$ROLE_NAME" ]; then + echo "- IAM Role: $ROLE_NAME" + fi + if [ -n "$POLICY_ARN" ]; then + echo "- IAM Policy: $POLICY_ARN" + fi + if [ -n "$STEPFUNCTIONS_POLICY_ARN" ]; then + echo "- Step Functions Policy: $STEPFUNCTIONS_POLICY_ARN" + fi + + echo "Attempting to clean up resources..." + cleanup + exit 1 +} + +# Function to clean up resources +cleanup() { + echo "Cleaning up resources..." + + # Delete state machine if it exists + if [ -n "$STATE_MACHINE_ARN" ]; then + echo "Deleting state machine: $STATE_MACHINE_ARN" + aws stepfunctions delete-state-machine --state-machine-arn "$STATE_MACHINE_ARN" || echo "Failed to delete state machine" + fi + + # Detach and delete policies if they exist + if [ -n "$POLICY_ARN" ] && [ -n "$ROLE_NAME" ]; then + echo "Detaching Comprehend policy $POLICY_ARN from role $ROLE_NAME" + aws iam detach-role-policy --role-name "$ROLE_NAME" --policy-arn "$POLICY_ARN" || echo "Failed to detach Comprehend policy" + fi + + if [ -n "$STEPFUNCTIONS_POLICY_ARN" ] && [ -n "$ROLE_NAME" ]; then + echo "Detaching Step Functions policy $STEPFUNCTIONS_POLICY_ARN from role $ROLE_NAME" + aws iam detach-role-policy --role-name "$ROLE_NAME" --policy-arn "$STEPFUNCTIONS_POLICY_ARN" || echo "Failed to detach Step Functions policy" + fi + + # Delete custom policies if they exist + if [ -n "$POLICY_ARN" ]; then + echo "Deleting Comprehend policy: $POLICY_ARN" + aws iam delete-policy --policy-arn "$POLICY_ARN" || echo "Failed to delete Comprehend policy" + fi + + if [ -n "$STEPFUNCTIONS_POLICY_ARN" ]; then + echo "Deleting Step Functions policy: $STEPFUNCTIONS_POLICY_ARN" + aws iam delete-policy --policy-arn "$STEPFUNCTIONS_POLICY_ARN" || echo "Failed to delete Step Functions policy" + fi + + # Delete role if it exists + if [ -n "$ROLE_NAME" ]; then + echo "Deleting role: $ROLE_NAME" + aws iam delete-role --role-name "$ROLE_NAME" || echo "Failed to delete role" + fi + + # Remove temporary files + echo "Removing temporary files" + rm -f hello-world.json updated-hello-world.json sentiment-hello-world.json step-functions-trust-policy.json comprehend-policy.json stepfunctions-policy.json input.json sentiment-input.json +} + +# Generate a random identifier for resource names +RANDOM_ID=$(openssl rand -hex 4) +ROLE_NAME="StepFunctionsHelloWorldRole-${RANDOM_ID}" +POLICY_NAME="DetectSentimentPolicy-${RANDOM_ID}" +STATE_MACHINE_NAME="MyFirstStateMachine-${RANDOM_ID}" + +echo "Using random identifier: $RANDOM_ID" +echo "Role name: $ROLE_NAME" +echo "Policy name: $POLICY_NAME" +echo "State machine name: $STATE_MACHINE_NAME" + +# Step 1: Create the state machine definition +echo "Creating state machine definition..." +cat > hello-world.json << 'EOF' +{ + "Comment": "A Hello World example of the Amazon States Language using a Pass state", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Result": { + "IsHelloWorldExample": true, + "ExecutionWaitTimeInSeconds": 10 + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "FailState" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + }, + "FailState": { + "Type": "Fail", + "Error": "NotHelloWorldExample", + "Cause": "The IsHelloWorldExample value was false" + } + } +} +EOF + +# Create IAM role trust policy +echo "Creating IAM role trust policy..." +cat > step-functions-trust-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "states.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EOF + +# Create IAM role +echo "Creating IAM role: $ROLE_NAME" +ROLE_RESULT=$(aws iam create-role \ + --role-name "$ROLE_NAME" \ + --assume-role-policy-document file://step-functions-trust-policy.json) + +check_api_error "$ROLE_RESULT" "Create IAM role" +echo "Role created successfully" + +# Get the role ARN +if [[ "$USE_JQ" == "true" ]]; then + ROLE_ARN=$(echo "$ROLE_RESULT" | jq -r '.Role.Arn') +else + ROLE_ARN=$(echo "$ROLE_RESULT" | grep "Arn" | cut -d'"' -f4) +fi + +if [ -z "$ROLE_ARN" ]; then + handle_error "Failed to extract role ARN" +fi +echo "Role ARN: $ROLE_ARN" + +# Create a custom policy for Step Functions +echo "Creating custom policy for Step Functions..." +cat > stepfunctions-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "states:*" + ], + "Resource": "*" + } + ] +} +EOF + +# Create the policy +echo "Creating Step Functions policy..." +STEPFUNCTIONS_POLICY_RESULT=$(aws iam create-policy \ + --policy-name "StepFunctionsPolicy-${RANDOM_ID}" \ + --policy-document file://stepfunctions-policy.json) + +check_api_error "$STEPFUNCTIONS_POLICY_RESULT" "Create Step Functions policy" +echo "Step Functions policy created successfully" + +# Get the policy ARN +if [[ "$USE_JQ" == "true" ]]; then + STEPFUNCTIONS_POLICY_ARN=$(echo "$STEPFUNCTIONS_POLICY_RESULT" | jq -r '.Policy.Arn') +else + STEPFUNCTIONS_POLICY_ARN=$(echo "$STEPFUNCTIONS_POLICY_RESULT" | grep "Arn" | cut -d'"' -f4) +fi + +if [ -z "$STEPFUNCTIONS_POLICY_ARN" ]; then + handle_error "Failed to extract Step Functions policy ARN" +fi +echo "Step Functions policy ARN: $STEPFUNCTIONS_POLICY_ARN" + +# Attach policy to the role +echo "Attaching Step Functions policy to role..." +ATTACH_RESULT=$(aws iam attach-role-policy \ + --role-name "$ROLE_NAME" \ + --policy-arn "$STEPFUNCTIONS_POLICY_ARN") + +if [ $? -ne 0 ]; then + handle_error "Failed to attach Step Functions policy to role" +fi + +# Wait for role to propagate (IAM changes can take time to propagate) +wait_for_propagation "IAM role" 10 + +# Create state machine +echo "Creating state machine: $STATE_MACHINE_NAME" +SM_RESULT=$(aws stepfunctions create-state-machine \ + --name "$STATE_MACHINE_NAME" \ + --definition file://hello-world.json \ + --role-arn "$ROLE_ARN" \ + --type STANDARD) + +check_api_error "$SM_RESULT" "Create state machine" +echo "State machine created successfully" + +# Get the state machine ARN +if [[ "$USE_JQ" == "true" ]]; then + STATE_MACHINE_ARN=$(echo "$SM_RESULT" | jq -r '.stateMachineArn') +else + STATE_MACHINE_ARN=$(echo "$SM_RESULT" | grep "stateMachineArn" | cut -d'"' -f4) +fi + +if [ -z "$STATE_MACHINE_ARN" ]; then + handle_error "Failed to extract state machine ARN" +fi +echo "State machine ARN: $STATE_MACHINE_ARN" + +# Step 2: Start the state machine execution +echo "Starting state machine execution..." +EXEC_RESULT=$(aws stepfunctions start-execution \ + --state-machine-arn "$STATE_MACHINE_ARN" \ + --name "hello001-${RANDOM_ID}") + +check_api_error "$EXEC_RESULT" "Start execution" +echo "Execution started successfully" + +# Get the execution ARN +if [[ "$USE_JQ" == "true" ]]; then + EXECUTION_ARN=$(echo "$EXEC_RESULT" | jq -r '.executionArn') +else + EXECUTION_ARN=$(echo "$EXEC_RESULT" | grep "executionArn" | cut -d'"' -f4) +fi + +if [ -z "$EXECUTION_ARN" ]; then + handle_error "Failed to extract execution ARN" +fi +echo "Execution ARN: $EXECUTION_ARN" + +# Wait for execution to complete (the workflow has a 10-second wait state) +echo "Waiting for execution to complete (15 seconds)..." +sleep 15 + +# Check execution status +echo "Checking execution status..." +EXEC_STATUS=$(aws stepfunctions describe-execution \ + --execution-arn "$EXECUTION_ARN") + +echo "Execution status: $EXEC_STATUS" + +# Step 3: Update state machine to process external input +echo "Updating state machine to process external input..." +cat > updated-hello-world.json << 'EOF' +{ + "Comment": "A Hello World example of the Amazon States Language using a Pass state", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Parameters": { + "IsHelloWorldExample.$": "$.hello_world", + "ExecutionWaitTimeInSeconds.$": "$.wait" + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "FailState" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + }, + "FailState": { + "Type": "Fail", + "Error": "NotHelloWorldExample", + "Cause": "The IsHelloWorldExample value was false" + } + } +} +EOF + +# Update state machine +echo "Updating state machine..." +UPDATE_RESULT=$(aws stepfunctions update-state-machine \ + --state-machine-arn "$STATE_MACHINE_ARN" \ + --definition file://updated-hello-world.json \ + --role-arn "$ROLE_ARN") + +check_api_error "$UPDATE_RESULT" "Update state machine" +echo "State machine updated successfully" + +# Create input file +echo "Creating input file..." +cat > input.json << 'EOF' +{ + "wait": 5, + "hello_world": true +} +EOF + +# Start execution with input +echo "Starting execution with input..." +EXEC2_RESULT=$(aws stepfunctions start-execution \ + --state-machine-arn "$STATE_MACHINE_ARN" \ + --name "hello002-${RANDOM_ID}" \ + --input file://input.json) + +check_api_error "$EXEC2_RESULT" "Start execution with input" +echo "Execution with input started successfully" + +# Get the execution ARN +if [[ "$USE_JQ" == "true" ]]; then + EXECUTION2_ARN=$(echo "$EXEC2_RESULT" | jq -r '.executionArn') +else + EXECUTION2_ARN=$(echo "$EXEC2_RESULT" | grep "executionArn" | cut -d'"' -f4) +fi + +if [ -z "$EXECUTION2_ARN" ]; then + handle_error "Failed to extract execution ARN" +fi +echo "Execution ARN: $EXECUTION2_ARN" + +# Wait for execution to complete (the workflow has a 5-second wait state) +echo "Waiting for execution to complete (10 seconds)..." +sleep 10 + +# Check execution status +echo "Checking execution status..." +EXEC2_STATUS=$(aws stepfunctions describe-execution \ + --execution-arn "$EXECUTION2_ARN") + +echo "Execution status: $EXEC2_STATUS" + +# Step 4: Integrate Amazon Comprehend for sentiment analysis (if available) +if [[ "$SKIP_COMPREHEND" == "false" ]]; then + echo "Creating policy for Amazon Comprehend access..." + cat > comprehend-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "comprehend:DetectSentiment" + ], + "Resource": "*" + } + ] +} +EOF + + # Create policy + echo "Creating IAM policy: $POLICY_NAME" + POLICY_RESULT=$(aws iam create-policy \ + --policy-name "$POLICY_NAME" \ + --policy-document file://comprehend-policy.json) + + check_api_error "$POLICY_RESULT" "Create Comprehend policy" + echo "Comprehend policy created successfully" + + # Get policy ARN + if [[ "$USE_JQ" == "true" ]]; then + POLICY_ARN=$(echo "$POLICY_RESULT" | jq -r '.Policy.Arn') + else + POLICY_ARN=$(echo "$POLICY_RESULT" | grep "Arn" | cut -d'"' -f4) + fi + + if [ -z "$POLICY_ARN" ]; then + handle_error "Failed to extract policy ARN" + fi + echo "Policy ARN: $POLICY_ARN" + + # Attach policy to role + echo "Attaching policy to role..." + ATTACH2_RESULT=$(aws iam attach-role-policy \ + --role-name "$ROLE_NAME" \ + --policy-arn "$POLICY_ARN") + + if [ $? -ne 0 ]; then + handle_error "Failed to attach policy to role" + fi + + # Create updated state machine definition with sentiment analysis + echo "Creating updated state machine definition with sentiment analysis..." + cat > sentiment-hello-world.json << 'EOF' +{ + "Comment": "A Hello World example with sentiment analysis", + "StartAt": "SetVariables", + "States": { + "SetVariables": { + "Type": "Pass", + "Parameters": { + "IsHelloWorldExample.$": "$.hello_world", + "ExecutionWaitTimeInSeconds.$": "$.wait", + "FeedbackComment.$": "$.feedback_comment" + }, + "Next": "IsHelloWorldExample" + }, + "IsHelloWorldExample": { + "Type": "Choice", + "Choices": [ + { + "Variable": "$.IsHelloWorldExample", + "BooleanEquals": true, + "Next": "WaitState" + } + ], + "Default": "DetectSentiment" + }, + "WaitState": { + "Type": "Wait", + "SecondsPath": "$.ExecutionWaitTimeInSeconds", + "Next": "ParallelProcessing" + }, + "ParallelProcessing": { + "Type": "Parallel", + "Branches": [ + { + "StartAt": "Process1", + "States": { + "Process1": { + "Type": "Pass", + "Result": { + "message": "Processing task 1" + }, + "End": true + } + } + }, + { + "StartAt": "Process2", + "States": { + "Process2": { + "Type": "Pass", + "Result": { + "message": "Processing task 2" + }, + "End": true + } + } + } + ], + "Next": "CheckpointState" + }, + "CheckpointState": { + "Type": "Pass", + "Result": { + "CheckpointMessage": "Workflow completed successfully!" + }, + "Next": "SuccessState" + }, + "DetectSentiment": { + "Type": "Task", + "Resource": "arn:aws:states:::aws-sdk:comprehend:detectSentiment", + "Parameters": { + "LanguageCode": "en", + "Text.$": "$.FeedbackComment" + }, + "Next": "SuccessState" + }, + "SuccessState": { + "Type": "Succeed" + } + } +} +EOF + + # Wait for IAM changes to propagate + wait_for_propagation "IAM changes" 10 + + # Update state machine + echo "Updating state machine with sentiment analysis..." + UPDATE2_RESULT=$(aws stepfunctions update-state-machine \ + --state-machine-arn "$STATE_MACHINE_ARN" \ + --definition file://sentiment-hello-world.json \ + --role-arn "$ROLE_ARN") + + check_api_error "$UPDATE2_RESULT" "Update state machine with sentiment analysis" + echo "State machine updated with sentiment analysis successfully" + + # Create input file with feedback comment + echo "Creating input file with feedback comment..." + cat > sentiment-input.json << 'EOF' +{ + "hello_world": false, + "wait": 5, + "feedback_comment": "This getting started with Step Functions workshop is a challenge!" +} +EOF + + # Start execution with sentiment analysis input + echo "Starting execution with sentiment analysis input..." + EXEC3_RESULT=$(aws stepfunctions start-execution \ + --state-machine-arn "$STATE_MACHINE_ARN" \ + --name "hello003-${RANDOM_ID}" \ + --input file://sentiment-input.json) + + check_api_error "$EXEC3_RESULT" "Start execution with sentiment analysis" + echo "Execution with sentiment analysis started successfully" + + # Get the execution ARN + if [[ "$USE_JQ" == "true" ]]; then + EXECUTION3_ARN=$(echo "$EXEC3_RESULT" | jq -r '.executionArn') + else + EXECUTION3_ARN=$(echo "$EXEC3_RESULT" | grep "executionArn" | cut -d'"' -f4) + fi + + if [ -z "$EXECUTION3_ARN" ]; then + handle_error "Failed to extract execution ARN" + fi + echo "Execution ARN: $EXECUTION3_ARN" + + # Wait for execution to complete + echo "Waiting for execution to complete (5 seconds)..." + sleep 5 + + # Check execution status + echo "Checking execution status..." + EXEC3_STATUS=$(aws stepfunctions describe-execution \ + --execution-arn "$EXECUTION3_ARN") + + echo "Execution status: $EXEC3_STATUS" +else + echo "Skipping Amazon Comprehend integration (not available in $CURRENT_REGION)" + EXECUTION3_ARN="" +fi + +# Display summary of resources created +echo "" +echo "===========================================" +echo "RESOURCES CREATED" +echo "===========================================" +echo "State Machine: $STATE_MACHINE_ARN" +echo "IAM Role: $ROLE_NAME" +echo "Step Functions Policy: StepFunctionsPolicy-${RANDOM_ID} ($STEPFUNCTIONS_POLICY_ARN)" +if [[ "$SKIP_COMPREHEND" == "false" ]]; then + echo "Comprehend Policy: $POLICY_NAME ($POLICY_ARN)" +fi +echo "Executions:" +echo " - hello001-${RANDOM_ID}: $EXECUTION_ARN" +echo " - hello002-${RANDOM_ID}: $EXECUTION2_ARN" +if [[ "$SKIP_COMPREHEND" == "false" ]]; then + echo " - hello003-${RANDOM_ID}: $EXECUTION3_ARN" +fi +echo "===========================================" + +# Prompt for cleanup +echo "" +echo "===========================================" +echo "CLEANUP CONFIRMATION" +echo "===========================================" + +if [[ "$AUTO_CLEANUP" == "true" ]]; then + echo "Auto-cleanup enabled. Cleaning up resources..." + cleanup + echo "All resources have been cleaned up." +else + echo "Do you want to clean up all created resources? (y/n): " + read -r CLEANUP_CHOICE + + if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + cleanup + echo "All resources have been cleaned up." + else + echo "Resources were not cleaned up. You can manually clean them up later." + echo "To view the state machine in the AWS console, visit:" + echo "https://console.aws.amazon.com/states/home?region=$CURRENT_REGION" + fi +fi + +echo "Script completed successfully!" diff --git a/tuts/085-amazon-ecs-service-connect/README.md b/tuts/085-amazon-ecs-service-connect/README.md new file mode 100644 index 00000000..d3c25cea --- /dev/null +++ b/tuts/085-amazon-ecs-service-connect/README.md @@ -0,0 +1,5 @@ +# Amazon ECS Service Connect + +This tutorial guides you through setting up Amazon Elastic Container Service (Amazon ECS) Service Connect using the AWS Command Line Interface (AWS CLI). You'll learn how to create an ECS cluster with Service Connect enabled, deploy a containerized application, and configure service discovery for inter-service communication. + +You can either run the automated shell script (`amazon-ecs-service-connect.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`amazon-ecs-service-connect.md`) to understand each component in detail. Both approaches will help you understand how to implement service-to-service communication in Amazon ECS using Service Connect. diff --git a/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.md b/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.md new file mode 100644 index 00000000..68269a58 --- /dev/null +++ b/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.md @@ -0,0 +1,711 @@ +# Configure Amazon ECS Service Connect with the AWS CLI + +This tutorial guides you through setting up Amazon ECS Service Connect using the AWS Command Line Interface (AWS CLI). You'll learn how to create an ECS cluster with Service Connect enabled, deploy a containerized application, and configure service discovery for inter-service communication. + +## Topics + +* [Prerequisites](#prerequisites) +* [Create the VPC infrastructure](#create-the-vpc-infrastructure) +* [Set up logging](#set-up-logging) +* [Create the ECS cluster](#create-the-ecs-cluster) +* [Configure IAM roles](#configure-iam-roles) +* [Register the task definition](#register-the-task-definition) +* [Create the service with Service Connect](#create-the-service-with-service-connect) +* [Verify the deployment](#verify-the-deployment) +* [Clean up resources](#clean-up-resources) +* [Going to production](#going-to-production) +* [Next steps](#next-steps) + +## Prerequisites + +Before you begin this tutorial, make sure you have the following. + +1. The AWS CLI. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). You can also [use AWS CloudShell](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_CLI.html), which includes the AWS CLI. +2. Configured your AWS CLI with appropriate credentials. Run `aws configure` if you haven't set up your credentials yet. +3. Basic familiarity with containerization concepts and Docker. +4. [Sufficient permissions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security-iam-awsmanpol.html#security-iam-awsmanpol-AmazonECS_FullAccess) to create and manage ECS resources, VPC resources, and IAM roles in your AWS account. + +### Cost considerations + +This tutorial creates AWS resources that incur charges. The estimated cost for running this tutorial is approximately **$0.017 per hour** (based on US East 1 pricing), primarily from the ECS Fargate task. If you complete the tutorial in 2-3 hours, the total cost will be approximately $0.035-$0.052. **Follow the cleanup instructions** at the end of this tutorial to avoid ongoing charges. + +Let's get started with creating the infrastructure needed for ECS Service Connect. + +## Create the VPC infrastructure + +Service Connect requires a VPC with proper networking configuration. In this section, you'll create a VPC with public subnets, an internet gateway, and security groups for your ECS tasks. + +**Create a VPC** + +The following command creates a new VPC with a CIDR block that provides enough IP addresses for your ECS tasks. + +``` +aws ec2 create-vpc --cidr-block 10.0.0.0/16 +``` + +The command returns details about the new VPC, including its ID. Note the `VpcId` value for use in subsequent commands. + +``` +{ + "Vpc": { + "CidrBlock": "10.0.0.0/16", + "DhcpOptionsId": "dopt-abcd1234", + "State": "pending", + "VpcId": "vpc-abcd1234", + "OwnerId": "123456789012", + "InstanceTenancy": "default", + "Ipv6CidrBlockAssociationSet": [], + "CidrBlockAssociationSet": [ + { + "AssociationId": "vpc-cidr-assoc-abcd1234", + "CidrBlock": "10.0.0.0/16", + "CidrBlockState": { + "State": "associated" + } + } + ], + "IsDefault": false + } +} +``` + +**Enable DNS support** + +ECS Service Connect requires DNS resolution within the VPC. Enable DNS support and DNS hostnames for your VPC. + +``` +aws ec2 modify-vpc-attribute --vpc-id vpc-abcd1234 --enable-dns-support +aws ec2 modify-vpc-attribute --vpc-id vpc-abcd1234 --enable-dns-hostnames +``` + +These commands enable DNS resolution and hostname assignment, which are required for Service Connect to function properly. + +**Create public subnets** + +Create two public subnets in different availability zones to provide high availability for your ECS tasks. + +``` +aws ec2 create-subnet --vpc-id vpc-abcd1234 --cidr-block 10.0.0.0/24 --availability-zone us-west-2a +aws ec2 create-subnet --vpc-id vpc-abcd1234 --cidr-block 10.0.1.0/24 --availability-zone us-west-2b +``` + +Each command creates a subnet in a different availability zone. Note the `SubnetId` values from the output for use in later steps. + +**Set up internet connectivity** + +Create an internet gateway and attach it to your VPC to provide internet access for your ECS tasks. + +``` +aws ec2 create-internet-gateway +``` + +The command returns an internet gateway ID. Use this ID to attach the gateway to your VPC. + +``` +aws ec2 attach-internet-gateway --internet-gateway-id igw-abcd1234 --vpc-id vpc-abcd1234 +``` + +**Configure routing** + +Create a route table and add a route to the internet gateway to enable outbound internet access. + +``` +aws ec2 create-route-table --vpc-id vpc-abcd1234 +``` + +Add a default route to the internet gateway and associate the route table with your subnets. + +``` +aws ec2 create-route --route-table-id rtb-abcd1234 --destination-cidr-block 0.0.0.0/0 --gateway-id igw-abcd1234 +aws ec2 associate-route-table --route-table-id rtb-abcd1234 --subnet-id subnet-abcd1234 +aws ec2 associate-route-table --route-table-id rtb-abcd1234 --subnet-id subnet-efgh5678 +``` + +These commands ensure that your ECS tasks can reach the internet to pull container images and communicate with AWS services. + +**Create a security group** + +Create a security group that allows HTTP traffic within the VPC for your ECS tasks. + +``` +aws ec2 create-security-group --group-name tutorial-ecs-sg --description "ECS Service Connect security group" --vpc-id vpc-abcd1234 +``` + +Add an inbound rule to allow HTTP traffic from within the VPC. + +``` +aws ec2 authorize-security-group-ingress --group-id sg-abcd1234 --protocol tcp --port 80 --cidr 10.0.0.0/16 +``` + +This security group configuration follows the principle of least privilege by only allowing HTTP traffic from within the VPC. + +## Set up logging + +CloudWatch Logs provides centralized logging for your ECS tasks and Service Connect proxy. Create log groups for both the application and the Service Connect proxy. + +**Create log groups** + +The following commands create log groups for the nginx application and the Service Connect proxy. + +``` +aws logs create-log-group --log-group-name /ecs/service-connect-nginx +aws logs create-log-group --log-group-name /ecs/service-connect-proxy +``` + +These log groups will store logs from your application container and the Service Connect proxy, making it easier to troubleshoot issues and monitor your services. + +## Create the ECS cluster + +An ECS cluster with Service Connect provides a logical grouping of tasks and services with built-in service discovery capabilities. + +**Create the cluster with Service Connect defaults** + +The following command creates an ECS cluster with Service Connect enabled and sets up a default namespace. + +``` +aws ecs create-cluster --cluster-name tutorial-cluster --service-connect-defaults namespace=service-connect +``` + +The command creates both the ECS cluster and a Service Connect namespace. The namespace provides service discovery within the cluster. + +``` +{ + "cluster": { + "clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/tutorial-cluster", + "clusterName": "tutorial-cluster", + "serviceConnectDefaults": { + "namespace": "arn:aws:servicediscovery:us-west-2:123456789012:namespace/ns-xmpl1234" + }, + "status": "PROVISIONING", + "registeredContainerInstancesCount": 0, + "runningTasksCount": 0, + "pendingTasksCount": 0, + "activeServicesCount": 0, + "statistics": [], + "tags": [], + "settings": [ + { + "name": "containerInsights", + "value": "disabled" + } + ], + "capacityProviders": [], + "defaultCapacityProviderStrategy": [], + "attachments": [ + { + "id": "a1b2c3d4-5678-90ab-cdef-xmpl12345678", + "type": "sc", + "status": "ATTACHING", + "details": [] + } + ], + "attachmentsStatus": "UPDATE_IN_PROGRESS" + } +} +``` + +**Verify cluster creation** + +Check that your cluster is active and ready to host services. + +``` +aws ecs describe-clusters --clusters tutorial-cluster +``` + +Wait for the cluster status to show "ACTIVE" before proceeding to the next step. + +## Configure IAM roles + +ECS tasks require IAM roles to interact with AWS services. You'll need both a task execution role and a task role for Service Connect functionality. + +**Create the task execution role** + +The task execution role allows ECS to pull container images and write logs to CloudWatch. + +``` +aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document '{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ecs-tasks.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +}' +``` + +Attach the managed policy that provides the necessary permissions. + +``` +aws iam attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy +``` + +**Create the task role** + +The task role provides permissions for the running task, including ECS Exec capabilities. + +``` +aws iam create-role --role-name ecsTaskRole --assume-role-policy-document '{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ecs-tasks.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +}' +``` + +Add an inline policy for ECS Exec functionality. + +``` +aws iam put-role-policy --role-name ecsTaskRole --policy-name ECSExecPolicy --policy-document '{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ssmmessages:CreateControlChannel", + "ssmmessages:CreateDataChannel", + "ssmmessages:OpenControlChannel", + "ssmmessages:OpenDataChannel" + ], + "Resource": "*" + } + ] +}' +``` + +These roles provide the minimum permissions required for ECS tasks with Service Connect and ECS Exec capabilities. + +## Register the task definition + +A task definition specifies the container configuration and Service Connect settings for your application. + +**Create the task definition** + +The following command registers a task definition for an nginx web server with Service Connect configuration. + +``` +aws ecs register-task-definition --family service-connect-nginx --execution-role-arn arn:aws:iam::123456789012:role/ecsTaskExecutionRole --task-role-arn arn:aws:iam::123456789012:role/ecsTaskRole --network-mode awsvpc --requires-compatibilities FARGATE --cpu 256 --memory 512 --container-definitions '[ + { + "name": "webserver", + "image": "public.ecr.aws/docker/library/nginx:latest", + "cpu": 100, + "portMappings": [ + { + "name": "nginx", + "containerPort": 80, + "protocol": "tcp", + "appProtocol": "http" + } + ], + "essential": true, + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-nginx", + "awslogs-region": "us-west-2", + "awslogs-stream-prefix": "nginx" + } + } + } +]' +``` + +The task definition includes a named port mapping with `appProtocol` specified, which is required for Service Connect to understand how to route traffic to your service. + +``` +{ + "taskDefinition": { + "taskDefinitionArn": "arn:aws:ecs:us-west-2:123456789012:task-definition/service-connect-nginx:1", + "family": "service-connect-nginx", + "taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole", + "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", + "networkMode": "awsvpc", + "revision": 1, + "volumes": [], + "status": "ACTIVE", + "requiresAttributes": [ + { + "name": "com.amazonaws.ecs.capability.logging-driver.awslogs" + }, + { + "name": "ecs.capability.execution-role-awslogs" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21" + }, + { + "name": "com.amazonaws.ecs.capability.task-iam-role" + }, + { + "name": "ecs.capability.execution-role-ecr-pull" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18" + }, + { + "name": "ecs.capability.task-eni" + } + ], + "placementConstraints": [], + "compatibilities": [ + "EC2", + "FARGATE" + ], + "requiresCompatibilities": [ + "FARGATE" + ], + "cpu": "256", + "memory": "512", + "containerDefinitions": [ + { + "name": "webserver", + "image": "public.ecr.aws/docker/library/nginx:latest", + "cpu": 100, + "memory": 0, + "portMappings": [ + { + "name": "nginx", + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp", + "appProtocol": "http" + } + ], + "essential": true, + "environment": [], + "mountPoints": [], + "volumesFrom": [], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-nginx", + "awslogs-region": "us-west-2", + "awslogs-stream-prefix": "nginx" + } + } + } + ] + } +} +``` + +## Create the service with Service Connect + +An ECS service manages the desired number of running tasks and integrates with Service Connect for service discovery. + +**Create the service** + +The following command creates an ECS service with Service Connect enabled. + +``` +aws ecs create-service --cluster tutorial-cluster --service-name service-connect-nginx-service --task-definition service-connect-nginx --desired-count 1 --launch-type FARGATE --platform-version LATEST --network-configuration 'awsvpcConfiguration={assignPublicIp=ENABLED,securityGroups=[sg-abcd1234],subnets=[subnet-abcd1234,subnet-efgh5678]}' --service-connect-configuration '{ + "enabled": true, + "services":[ + { + "portName": "nginx", + "clientAliases":[ + { + "port": 80 + } + ] + } + ], + "logConfiguration":{ + "logDriver": "awslogs", + "options":{ + "awslogs-group": "/ecs/service-connect-proxy", + "awslogs-region": "us-west-2", + "awslogs-stream-prefix": "service-connect-proxy" + } + } +}' --enable-execute-command +``` + +The Service Connect configuration enables service discovery and creates a client alias that other services can use to connect to this service. + +``` +{ + "service": { + "serviceArn": "arn:aws:ecs:us-west-2:123456789012:service/tutorial-cluster/service-connect-nginx-service", + "serviceName": "service-connect-nginx-service", + "clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/tutorial-cluster", + "loadBalancers": [], + "serviceRegistries": [], + "status": "ACTIVE", + "desiredCount": 1, + "runningCount": 0, + "pendingCount": 0, + "launchType": "FARGATE", + "platformVersion": "LATEST", + "platformFamily": "Linux", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/service-connect-nginx:1", + "deploymentConfiguration": { + "deploymentCircuitBreaker": { + "enable": false, + "rollback": false + }, + "maximumPercent": 200, + "minimumHealthyPercent": 100 + }, + "deployments": [ + { + "id": "ecs-svc/1234567890123456789", + "status": "PRIMARY", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/service-connect-nginx:1", + "desiredCount": 1, + "pendingCount": 0, + "runningCount": 0, + "failedTasks": 0, + "createdAt": "2025-01-13T12:00:00.000000+00:00", + "updatedAt": "2025-01-13T12:00:00.000000+00:00", + "launchType": "FARGATE", + "platformVersion": "1.4.0", + "platformFamily": "Linux", + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + }, + "rolloutState": "IN_PROGRESS", + "rolloutStateReason": "ECS deployment ecs-svc/1234567890123456789 in progress.", + "serviceConnectConfiguration": { + "enabled": true, + "namespace": "service-connect", + "services": [ + { + "portName": "nginx", + "clientAliases": [ + { + "port": 80 + } + ] + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-proxy", + "awslogs-region": "us-west-2", + "awslogs-stream-prefix": "service-connect-proxy" + } + } + }, + "serviceConnectResources": [ + { + "discoveryName": "nginx", + "discoveryArn": "arn:aws:servicediscovery:us-west-2:123456789012:service/srv-xmpl1234" + } + ] + } + ], + "roleArn": "arn:aws:iam::123456789012:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS", + "events": [], + "createdAt": "2025-01-13T12:00:00.000000+00:00", + "placementConstraints": [], + "placementStrategy": [], + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + }, + "healthCheckGracePeriodSeconds": 0, + "schedulingStrategy": "REPLICA", + "deploymentController": { + "type": "ECS" + }, + "createdBy": "arn:aws:iam::123456789012:user/tutorial-user", + "enableECSManagedTags": false, + "propagateTags": "NONE", + "enableExecuteCommand": true + } +} +``` + +**Wait for service stability** + +Wait for the service to reach a stable state before proceeding. + +``` +aws ecs wait services-stable --cluster tutorial-cluster --services service-connect-nginx-service +``` + +This command waits until the service deployment is complete and all tasks are running successfully. + +## Verify the deployment + +After creating your service, verify that Service Connect is working correctly and your application is accessible. + +**Check service status** + +Verify that your service is running and has the expected number of tasks. + +``` +aws ecs describe-services --cluster tutorial-cluster --services service-connect-nginx-service --query 'services[0].{Status:status,Running:runningCount,Desired:desiredCount}' +``` + +The output should show that the service is active with one running task. + +``` +{ + "Status": "ACTIVE", + "Running": 1, + "Desired": 1 +} +``` + +**View Service Connect configuration** + +Check the Service Connect configuration to confirm it's properly set up. + +``` +aws ecs describe-services --cluster tutorial-cluster --services service-connect-nginx-service --query 'services[0].deployments[0].serviceConnectConfiguration' +``` + +The output shows the Service Connect namespace, service discovery name, and client aliases that other services can use to connect to your nginx service. + +``` +{ + "enabled": true, + "namespace": "arn:aws:servicediscovery:us-west-2:123456789012:namespace/ns-xmpl1234", + "services": [ + { + "portName": "nginx", + "discoveryName": "nginx", + "clientAliases": [ + { + "port": 80, + "dnsName": "nginx.service-connect" + } + ] + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-proxy", + "awslogs-region": "us-west-2", + "awslogs-stream-prefix": "service-connect-proxy" + } + } +} +``` + +**Verify Service Connect namespace** + +Confirm that the Service Connect namespace is active in AWS Cloud Map. + +``` +aws servicediscovery list-namespaces --query "Namespaces[?Name=='service-connect']" +``` + +This command shows the namespace created by Service Connect for service discovery within your cluster. + +Your nginx service is now accessible to other services in the same namespace using the DNS name `nginx.service-connect` on port 80. Service Connect automatically handles load balancing and service discovery between your containerized applications. + +## Clean up resources + +To avoid incurring charges, delete the resources you created in this tutorial when you're finished experimenting. + +**Delete the ECS service and cluster** + +First, scale the service to zero tasks, then delete the service and cluster. + +``` +aws ecs update-service --cluster tutorial-cluster --service service-connect-nginx-service --desired-count 0 +aws ecs delete-service --cluster tutorial-cluster --service service-connect-nginx-service --force +aws ecs delete-cluster --cluster tutorial-cluster +``` + +**Delete networking resources** + +Remove the VPC and associated networking components. + +``` +aws ec2 delete-security-group --group-id sg-abcd1234 +aws ec2 disassociate-route-table --association-id rtbassoc-abcd1234 +aws ec2 delete-route-table --route-table-id rtb-abcd1234 +aws ec2 detach-internet-gateway --internet-gateway-id igw-abcd1234 --vpc-id vpc-abcd1234 +aws ec2 delete-internet-gateway --internet-gateway-id igw-abcd1234 +aws ec2 delete-subnet --subnet-id subnet-abcd1234 +aws ec2 delete-subnet --subnet-id subnet-efgh5678 +aws ec2 delete-vpc --vpc-id vpc-abcd1234 +``` + +**Delete log groups** + +Remove the CloudWatch log groups to stop incurring log storage charges. + +``` +aws logs delete-log-group --log-group-name /ecs/service-connect-nginx +aws logs delete-log-group --log-group-name /ecs/service-connect-proxy +``` + +**Clean up IAM roles (optional)** + +If you created the IAM roles specifically for this tutorial, you can delete them. + +``` +aws iam detach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy +aws iam delete-role-policy --role-name ecsTaskRole --policy-name ECSExecPolicy +aws iam delete-role --role-name ecsTaskExecutionRole +aws iam delete-role --role-name ecsTaskRole +``` + +## Going to production + +This tutorial is designed to help you learn how ECS Service Connect works in a simple, cost-effective environment. For production deployments, you should consider additional security, scalability, and operational requirements that are beyond the scope of this tutorial. + +### Security considerations + +* **Private subnets**: Move ECS tasks to private subnets and use a NAT Gateway for outbound internet access +* **Service Connect TLS**: Enable TLS encryption for service-to-service communication +* **Secrets management**: Use AWS Secrets Manager for sensitive configuration data +* **Network security**: Implement more restrictive security group rules and consider network ACLs +* **Container security**: Scan container images for vulnerabilities and use private ECR repositories + +### Architecture considerations + +* **Auto scaling**: Configure ECS Service Auto Scaling based on CloudWatch metrics +* **Load balancing**: Add an Application Load Balancer for external traffic +* **Multi-region deployment**: Implement cross-region deployment for disaster recovery +* **Monitoring and observability**: Add comprehensive monitoring, alerting, and distributed tracing +* **Database integration**: Add managed database services with proper scaling and backup strategies + +For comprehensive guidance on production-ready architectures, see the [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html) and [AWS Architecture Center](https://aws.amazon.com/architecture/). For security best practices, see the [AWS Security Best Practices](https://docs.aws.amazon.com/security/latest/userguide/security-best-practices.html). + +## Next steps + +Now that you've successfully configured ECS Service Connect, consider exploring these related topics: + +* [Service Connect concepts](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect-concepts.html) - Learn more about Service Connect architecture and components +* [Service Connect configuration](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect-configuration.html) - Explore advanced Service Connect configuration options +* [ECS Service Connect with Application Load Balancer](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect-alb.html) - Integrate Service Connect with load balancers for external traffic +* [Service Connect TLS encryption](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect-tls.html) - Secure inter-service communication with TLS +* [ECS Exec](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html) - Debug and troubleshoot your containers using ECS Exec +* [Amazon ECS monitoring](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch-metrics.html) - Monitor your ECS services with CloudWatch metrics and logs diff --git a/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.sh b/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.sh new file mode 100755 index 00000000..d0352a90 --- /dev/null +++ b/tuts/085-amazon-ecs-service-connect/amazon-ecs-service-connect.sh @@ -0,0 +1,563 @@ +#!/bin/bash + +# ECS Service Connect Tutorial Script v4 - Modified to use Default VPC +# This script creates an ECS cluster with Service Connect and deploys an nginx service +# Uses the default VPC to avoid VPC limits + +set -e # Exit on any error + +# Configuration +SCRIPT_NAME="ECS Service Connect Tutorial" +LOG_FILE="ecs-service-connect-tutorial-v4-default-vpc.log" +REGION=$(aws configure get region) +ENV_PREFIX="tutorial" +CLUSTER_NAME="${ENV_PREFIX}-cluster" +NAMESPACE_NAME="service-connect" + +# Generate random suffix for unique resource names +RANDOM_SUFFIX=$(openssl rand -hex 6) + +# Arrays to track created resources for cleanup +declare -a CREATED_RESOURCES=() + +# Logging function +log() { + echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE" +} + +# Error handling function +handle_error() { + log "ERROR: Script failed at line $1" + log "Attempting to clean up resources..." + cleanup_resources + exit 1 +} + +# Set up error handling +trap 'handle_error $LINENO' ERR + +# Function to add resource to tracking array +track_resource() { + CREATED_RESOURCES+=("$1") + log "Tracking resource: $1" +} + +# Function to check if command output contains actual errors +check_for_errors() { + local output="$1" + local command_name="$2" + + # Check for specific AWS CLI error patterns, not just any occurrence of "error" + if echo "$output" | grep -qi "An error occurred\|InvalidParameterException\|AccessDenied\|ValidationException\|ResourceNotFoundException"; then + log "ERROR in $command_name: $output" + return 1 + fi + return 0 +} + +# Function to get AWS account ID +get_account_id() { + ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) + log "Using AWS Account ID: $ACCOUNT_ID" +} + +# Function to wait for resources to be ready +wait_for_resource() { + local resource_type="$1" + local resource_id="$2" + + case "$resource_type" in + "cluster") + log "Waiting for cluster $resource_id to be active..." + local attempt=1 + local max_attempts=30 + while [ $attempt -le $max_attempts ]; do + local status=$(aws ecs describe-clusters --clusters "$resource_id" --query 'clusters[0].status' --output text) + if [ "$status" = "ACTIVE" ]; then + log "Cluster is now active" + return 0 + fi + log "Cluster status: $status (attempt $attempt/$max_attempts)" + sleep 10 + ((attempt++)) + done + log "ERROR: Cluster did not become active within expected time" + return 1 + ;; + "service") + log "Waiting for service $resource_id to be stable..." + aws ecs wait services-stable --cluster "$CLUSTER_NAME" --services "$resource_id" + ;; + "nat-gateway") + log "Waiting for NAT Gateway $resource_id to be available..." + aws ec2 wait nat-gateway-available --nat-gateway-ids "$resource_id" + ;; + esac +} + +# Function to use default VPC infrastructure +setup_default_vpc_infrastructure() { + log "Using default VPC infrastructure..." + + # Get default VPC + VPC_ID=$(aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" --query 'Vpcs[0].VpcId' --output text) + if [[ "$VPC_ID" == "None" || -z "$VPC_ID" ]]; then + log "ERROR: No default VPC found. Please create a default VPC first." + exit 1 + fi + log "Using default VPC: $VPC_ID" + + # Get default subnets + SUBNETS=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" "Name=default-for-az,Values=true" --query 'Subnets[].SubnetId' --output text) + SUBNET_ARRAY=($SUBNETS) + + if [ ${#SUBNET_ARRAY[@]} -lt 2 ]; then + log "ERROR: Need at least 2 subnets for ECS Service Connect. Found: ${#SUBNET_ARRAY[@]}" + exit 1 + fi + + PUBLIC_SUBNET1=${SUBNET_ARRAY[0]} + PUBLIC_SUBNET2=${SUBNET_ARRAY[1]} + + log "Using subnets: $PUBLIC_SUBNET1, $PUBLIC_SUBNET2" + + # Create security group for ECS tasks + SG_OUTPUT=$(aws ec2 create-security-group \ + --group-name "${ENV_PREFIX}-ecs-sg-${RANDOM_SUFFIX}" \ + --description "Security group for ECS Service Connect tutorial" \ + --vpc-id "$VPC_ID" 2>&1) + check_for_errors "$SG_OUTPUT" "create-security-group" + SECURITY_GROUP_ID=$(echo "$SG_OUTPUT" | grep -o '"GroupId": "[^"]*"' | cut -d'"' -f4) + track_resource "SG:$SECURITY_GROUP_ID" + log "Created security group: $SECURITY_GROUP_ID" + + # Add inbound rules to security group + aws ec2 authorize-security-group-ingress \ + --group-id "$SECURITY_GROUP_ID" \ + --protocol tcp \ + --port 80 \ + --cidr 0.0.0.0/0 >/dev/null 2>&1 || true + + aws ec2 authorize-security-group-ingress \ + --group-id "$SECURITY_GROUP_ID" \ + --protocol tcp \ + --port 443 \ + --cidr 0.0.0.0/0 >/dev/null 2>&1 || true + + log "Default VPC infrastructure setup completed" +} + +# Function to create CloudWatch log groups +create_log_groups() { + log "Creating CloudWatch log groups..." + + # Create log group for nginx container + aws logs create-log-group --log-group-name "/ecs/service-connect-nginx" 2>&1 | grep -v "ResourceAlreadyExistsException" || { + if [ ${PIPESTATUS[0]} -eq 0 ]; then + log "Log group /ecs/service-connect-nginx created" + track_resource "LOG_GROUP:/ecs/service-connect-nginx" + else + log "Log group /ecs/service-connect-nginx already exists" + fi + } + + # Create log group for service connect proxy + aws logs create-log-group --log-group-name "/ecs/service-connect-proxy" 2>&1 | grep -v "ResourceAlreadyExistsException" || { + if [ ${PIPESTATUS[0]} -eq 0 ]; then + log "Log group /ecs/service-connect-proxy created" + track_resource "LOG_GROUP:/ecs/service-connect-proxy" + else + log "Log group /ecs/service-connect-proxy already exists" + fi + } +} + +# Function to create ECS cluster with Service Connect +create_ecs_cluster() { + log "Creating ECS cluster with Service Connect..." + + CLUSTER_OUTPUT=$(aws ecs create-cluster \ + --cluster-name "$CLUSTER_NAME" \ + --service-connect-defaults namespace="$NAMESPACE_NAME" \ + --tags key=Environment,value=tutorial 2>&1) + check_for_errors "$CLUSTER_OUTPUT" "create-cluster" + + track_resource "CLUSTER:$CLUSTER_NAME" + log "Created ECS cluster: $CLUSTER_NAME" + + wait_for_resource "cluster" "$CLUSTER_NAME" + + # Track the Service Connect namespace that gets created + # Wait a moment for the namespace to be created + sleep 5 + NAMESPACE_ID=$(aws servicediscovery list-namespaces \ + --filters Name=TYPE,Values=HTTP \ + --query "Namespaces[?Name=='$NAMESPACE_NAME'].Id" --output text 2>/dev/null || echo "") + + if [[ -n "$NAMESPACE_ID" && "$NAMESPACE_ID" != "None" ]]; then + track_resource "NAMESPACE:$NAMESPACE_ID" + log "Service Connect namespace created: $NAMESPACE_ID" + fi +} + +# Function to create IAM roles +create_iam_roles() { + log "Creating IAM roles..." + + # Check if ecsTaskExecutionRole exists + if aws iam get-role --role-name ecsTaskExecutionRole >/dev/null 2>&1; then + log "IAM role ecsTaskExecutionRole exists" + else + log "ERROR: ecsTaskExecutionRole does not exist. Please create it first." + exit 1 + fi + + # Check if ecsTaskRole exists, create if not + if aws iam get-role --role-name ecsTaskRole >/dev/null 2>&1; then + log "IAM role ecsTaskRole exists" + else + log "IAM role ecsTaskRole does not exist, will create it" + + # Create trust policy for ECS tasks + cat > /tmp/ecs-task-trust-policy.json << EOF +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ecs-tasks.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EOF + + aws iam create-role \ + --role-name ecsTaskRole \ + --assume-role-policy-document file:///tmp/ecs-task-trust-policy.json >/dev/null + + track_resource "IAM_ROLE:ecsTaskRole" + log "Created ecsTaskRole" + + # Wait for role to be available + sleep 10 + fi +} + +# Function to create task definition +create_task_definition() { + log "Creating task definition..." + + # Create task definition JSON + cat > /tmp/task-definition.json << EOF +{ + "family": "service-connect-nginx", + "networkMode": "awsvpc", + "requiresCompatibilities": ["FARGATE"], + "cpu": "256", + "memory": "512", + "executionRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskExecutionRole", + "taskRoleArn": "arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskRole", + "containerDefinitions": [ + { + "name": "nginx", + "image": "nginx:latest", + "portMappings": [ + { + "containerPort": 80, + "protocol": "tcp", + "name": "nginx-port" + } + ], + "essential": true, + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-nginx", + "awslogs-region": "${REGION}", + "awslogs-stream-prefix": "ecs" + } + } + } + ] +} +EOF + + TASK_DEF_OUTPUT=$(aws ecs register-task-definition --cli-input-json file:///tmp/task-definition.json 2>&1) + check_for_errors "$TASK_DEF_OUTPUT" "register-task-definition" + + TASK_DEF_ARN=$(echo "$TASK_DEF_OUTPUT" | grep -o '"taskDefinitionArn": "[^"]*"' | cut -d'"' -f4) + track_resource "TASK_DEF:service-connect-nginx" + log "Created task definition: $TASK_DEF_ARN" + + # Clean up temporary file + rm -f /tmp/task-definition.json +} + +# Function to create ECS service with Service Connect +create_ecs_service() { + log "Creating ECS service with Service Connect..." + + # Create service definition JSON + cat > /tmp/service-definition.json << EOF +{ + "serviceName": "service-connect-nginx-service", + "cluster": "${CLUSTER_NAME}", + "taskDefinition": "service-connect-nginx", + "desiredCount": 1, + "launchType": "FARGATE", + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": ["${PUBLIC_SUBNET1}", "${PUBLIC_SUBNET2}"], + "securityGroups": ["${SECURITY_GROUP_ID}"], + "assignPublicIp": "ENABLED" + } + }, + "serviceConnectConfiguration": { + "enabled": true, + "namespace": "${NAMESPACE_NAME}", + "services": [ + { + "portName": "nginx-port", + "discoveryName": "nginx", + "clientAliases": [ + { + "port": 80, + "dnsName": "nginx" + } + ] + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": "/ecs/service-connect-proxy", + "awslogs-region": "${REGION}", + "awslogs-stream-prefix": "ecs-service-connect" + } + } + }, + "tags": [ + { + "key": "Environment", + "value": "tutorial" + } + ] +} +EOF + + SERVICE_OUTPUT=$(aws ecs create-service --cli-input-json file:///tmp/service-definition.json 2>&1) + check_for_errors "$SERVICE_OUTPUT" "create-service" + + track_resource "SERVICE:service-connect-nginx-service" + log "Created ECS service: service-connect-nginx-service" + + wait_for_resource "service" "service-connect-nginx-service" + + # Clean up temporary file + rm -f /tmp/service-definition.json +} + +# Function to verify deployment +verify_deployment() { + log "Verifying deployment..." + + # Check service status + SERVICE_STATUS=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "service-connect-nginx-service" \ + --query 'services[0].status' --output text) + log "Service status: $SERVICE_STATUS" + + # Check running tasks + RUNNING_COUNT=$(aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "service-connect-nginx-service" \ + --query 'services[0].runningCount' --output text) + log "Running tasks: $RUNNING_COUNT" + + # Get task ARN + TASK_ARN=$(aws ecs list-tasks \ + --cluster "$CLUSTER_NAME" \ + --service-name "service-connect-nginx-service" \ + --query 'taskArns[0]' --output text) + + if [[ "$TASK_ARN" != "None" && -n "$TASK_ARN" ]]; then + log "Task ARN: $TASK_ARN" + + # Try to get task IP address + TASK_IP=$(aws ecs describe-tasks \ + --cluster "$CLUSTER_NAME" \ + --tasks "$TASK_ARN" \ + --query 'tasks[0].attachments[0].details[?name==`privateIPv4Address`].value' \ + --output text 2>/dev/null || echo "") + + if [[ -n "$TASK_IP" && "$TASK_IP" != "None" ]]; then + log "Task IP address: $TASK_IP" + else + log "Could not retrieve task IP address" + fi + fi + + # Check Service Connect namespace + NAMESPACE_STATUS=$(aws servicediscovery list-namespaces \ + --filters Name=TYPE,Values=HTTP \ + --query "Namespaces[?Name=='$NAMESPACE_NAME'].Id" --output text 2>/dev/null || echo "") + + if [[ -n "$NAMESPACE_STATUS" && "$NAMESPACE_STATUS" != "None" ]]; then + log "Service Connect namespace '$NAMESPACE_NAME' is active" + else + log "Service Connect namespace '$NAMESPACE_NAME' not found or not active" + fi + + # Display Service Connect configuration + log "Service Connect configuration:" + aws ecs describe-services \ + --cluster "$CLUSTER_NAME" \ + --services "service-connect-nginx-service" \ + --query 'services[0].serviceConnectConfiguration' 2>/dev/null || true +} + +# Function to display created resources +display_resources() { + echo "" + echo "===========================================" + echo "CREATED RESOURCES" + echo "===========================================" + for resource in "${CREATED_RESOURCES[@]}"; do + echo "- $resource" + done + echo "===========================================" + echo "" +} + +# Function to cleanup resources +cleanup_resources() { + log "Starting cleanup process..." + + # Delete resources in reverse order of creation + for ((i=${#CREATED_RESOURCES[@]}-1; i>=0; i--)); do + resource="${CREATED_RESOURCES[i]}" + resource_type=$(echo "$resource" | cut -d':' -f1) + resource_id=$(echo "$resource" | cut -d':' -f2) + + log "Cleaning up $resource_type: $resource_id" + + case "$resource_type" in + "SERVICE") + # Scale service to 0 first + aws ecs update-service --cluster "$CLUSTER_NAME" --service "$resource_id" --desired-count 0 >/dev/null 2>&1 || true + # Wait for service to scale down + aws ecs wait services-stable --cluster "$CLUSTER_NAME" --services "$resource_id" 2>/dev/null || true + # Delete service + aws ecs delete-service --cluster "$CLUSTER_NAME" --service "$resource_id" >/dev/null 2>&1 || true + ;; + "TASK_DEF") + # Deregister all revisions of the task definition + TASK_DEF_ARNS=$(aws ecs list-task-definitions --family-prefix "$resource_id" --query 'taskDefinitionArns' --output text) + for arn in $TASK_DEF_ARNS; do + aws ecs deregister-task-definition --task-definition "$arn" >/dev/null 2>&1 || true + done + ;; + "IAM_ROLE") + # Detach policies and delete role + aws iam detach-role-policy --role-name "$resource_id" --policy-arn "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy" >/dev/null 2>&1 || true + aws iam delete-role --role-name "$resource_id" >/dev/null 2>&1 || true + ;; + "CLUSTER") + aws ecs delete-cluster --cluster "$resource_id" >/dev/null 2>&1 || true + ;; + "SG") + aws ec2 delete-security-group --group-id "$resource_id" >/dev/null 2>&1 || true + ;; + "LOG_GROUP") + aws logs delete-log-group --log-group-name "$resource_id" >/dev/null 2>&1 || true + ;; + "NAMESPACE") + # First, delete any services in the namespace + NAMESPACE_SERVICES=$(aws servicediscovery list-services \ + --filters Name=NAMESPACE_ID,Values="$resource_id" \ + --query 'Services[].Id' --output text 2>/dev/null || echo "") + + if [[ -n "$NAMESPACE_SERVICES" && "$NAMESPACE_SERVICES" != "None" ]]; then + for service_id in $NAMESPACE_SERVICES; do + aws servicediscovery delete-service --id "$service_id" >/dev/null 2>&1 || true + sleep 2 + done + fi + + # Then delete the namespace + aws servicediscovery delete-namespace --id "$resource_id" >/dev/null 2>&1 || true + ;; + esac + + sleep 2 # Brief pause between deletions + done + + # Clean up temporary files + rm -f /tmp/ecs-task-trust-policy.json + rm -f /tmp/task-definition.json + rm -f /tmp/service-definition.json + + log "Cleanup completed" +} + +# Main execution +main() { + log "Starting $SCRIPT_NAME v4 (Default VPC)" + log "Region: $REGION" + log "Log file: $LOG_FILE" + + # Get AWS account ID + get_account_id + + # Setup infrastructure using default VPC + setup_default_vpc_infrastructure + + # Create CloudWatch log groups + create_log_groups + + # Create ECS cluster + create_ecs_cluster + + # Create IAM roles + create_iam_roles + + # Create task definition + create_task_definition + + # Create ECS service + create_ecs_service + + # Verify deployment + verify_deployment + + log "Tutorial completed successfully!" + + # Display created resources + display_resources + + # Ask user if they want to clean up + echo "" + echo "===========================================" + echo "CLEANUP CONFIRMATION" + echo "===========================================" + echo "Do you want to clean up all created resources? (y/n): " + read -r CLEANUP_CHOICE + + if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + cleanup_resources + log "All resources have been cleaned up" + else + log "Resources left intact. You can clean them up later by running the cleanup function." + echo "" + echo "To clean up resources later, you can use the AWS CLI commands or the AWS Management Console." + echo "Remember to delete resources in the correct order to avoid dependency issues." + fi +} + +# Make script executable and run +chmod +x "$0" +main "$@" diff --git a/tuts/086-amazon-ecs-fargate-linux/README.md b/tuts/086-amazon-ecs-fargate-linux/README.md new file mode 100644 index 00000000..8cee0eeb --- /dev/null +++ b/tuts/086-amazon-ecs-fargate-linux/README.md @@ -0,0 +1,5 @@ +# Amazon ECS Fargate Linux + +This tutorial shows you how to create and run an Amazon Elastic Container Service (Amazon ECS) Linux task using Fargate with the AWS Command Line Interface (AWS CLI). You'll learn how to create an ECS cluster, register a task definition, create a service, and access your running application. + +You can either run the automated shell script (`amazon-ecs-fargate-linux.sh`) to quickly set up the entire environment, or follow the step-by-step instructions in the tutorial (`amazon-ecs-fargate-linux.md`) to understand each component in detail. Both approaches will help you understand how to deploy containerized applications on Amazon ECS using Fargate. diff --git a/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.md b/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.md new file mode 100644 index 00000000..889453be --- /dev/null +++ b/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.md @@ -0,0 +1,642 @@ +# Create an Amazon ECS Linux task for the Fargate launch type using the AWS CLI + +This tutorial shows you how to create and run an Amazon ECS Linux task using the Fargate launch type with the AWS CLI. You'll learn how to create an ECS cluster, register a task definition, create a service, and access your running application. + +## Topics + +* [Prerequisites](#prerequisites) +* [Create the cluster](#create-the-cluster) +* [Create a task definition](#create-a-task-definition) +* [Create the service](#create-the-service) +* [View your service](#view-your-service) +* [Clean up](#clean-up) +* [Going to production](#going-to-production) +* [Next steps](#next-steps) + +## Prerequisites + +Before you begin this tutorial, make sure you have the following. + +1. The AWS CLI. If you need to install it, follow the [AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). You can also [use AWS CloudShell](https://docs.aws.amazon.com/cloudshell/latest/userguide/welcome.html), which includes the AWS CLI. +2. Configured your AWS CLI with appropriate credentials. Run `aws configure` if you haven't set up your credentials yet. +3. [Sufficient permissions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security_iam_id-based-policy-examples.html) to create and manage Amazon ECS resources in your AWS account. +4. A default VPC in your chosen AWS Region. If you don't have a default VPC, you can [create one](https://docs.aws.amazon.com/vpc/latest/userguide/default-vpc.html). + +The AWS CLI attempts to automatically create the task execution IAM role, which is required for Fargate tasks. To ensure that the AWS CLI can create this IAM role, one of the following must be true: + +* Your user has administrator access. +* Your user has the IAM permissions to create a service role. +* A user with administrator access has manually created the task execution role so that it is available on the account to be used. + +### Cost considerations + +This tutorial creates AWS resources that incur charges. The estimated cost for completing this tutorial is approximately **$0.02 USD**, assuming you complete it within 30 minutes and clean up resources immediately afterward. The primary cost comes from AWS Fargate compute charges (approximately $0.045 per hour for the minimum configuration used in this tutorial). All other resources (VPC, security groups, IAM roles) are free of charge. For more information about AWS Fargate pricing, see [AWS Fargate Pricing](https://aws.amazon.com/fargate/pricing/). + +**Important**: The security group you create in this tutorial allows HTTP traffic from anywhere on the internet (0.0.0.0/0). This configuration is appropriate for this tutorial, but you should restrict access to specific IP ranges in production environments. + +## Create the cluster + +An Amazon ECS cluster is a logical grouping of tasks or services. In this section, you'll create a cluster to host your Fargate tasks. + +**Create an ECS cluster** + +The following command creates a new ECS cluster with a unique name. + +``` +$ aws ecs create-cluster --cluster-name my-fargate-cluster +``` + +Output: + +``` +{ + "cluster": { + "clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/my-fargate-cluster", + "clusterName": "my-fargate-cluster", + "status": "ACTIVE", + "registeredContainerInstancesCount": 0, + "runningTasksCount": 0, + "pendingTasksCount": 0, + "activeServicesCount": 0, + "statistics": [], + "tags": [], + "settings": [ + { + "name": "containerInsights", + "value": "disabled" + } + ], + "capacityProviders": [], + "defaultCapacityProviderStrategy": [] + } +} +``` + +The response shows that your cluster has been created successfully with an `ACTIVE` status. Note the cluster ARN, which uniquely identifies your cluster. + +## Create a task definition + +A task definition is like a blueprint for your application. It specifies which Docker image to use for containers, how many containers to use in the task, and the resource allocation for each container. + +**Register a task definition** + +First, create a JSON file that defines your task. The following command creates a task definition file for a simple web application. Replace the `executionRoleArn` with your own. + +``` +$ cat > task-definition.json << 'EOF' +{ + "family": "sample-fargate", + "networkMode": "awsvpc", + "requiresCompatibilities": ["FARGATE"], + "cpu": "256", + "memory": "512", + "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", + "containerDefinitions": [ + { + "name": "fargate-app", + "image": "public.ecr.aws/docker/library/httpd:latest", + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ], + "essential": true, + "entryPoint": ["sh", "-c"], + "command": [ + "/bin/sh -c \"echo ' Amazon ECS Sample App

Amazon ECS Sample App

Congratulations!

Your application is now running on a container in Amazon ECS.

' > /usr/local/apache2/htdocs/index.html && httpd-foreground\"" + ] + } + ] +} +EOF +``` + +This task definition specifies a Fargate-compatible task that runs a simple web server. The task uses 256 CPU units and 512 MB of memory, which are the minimum values for Fargate tasks. + +**Register the task definition** + +Now register the task definition with Amazon ECS using the following command. + +``` +$ aws ecs register-task-definition --cli-input-json file://task-definition.json +``` + +Output: + +``` +{ + "taskDefinition": { + "taskDefinitionArn": "arn:aws:ecs:us-west-2:123456789012:task-definition/sample-fargate:1", + "family": "sample-fargate", + "taskRoleArn": null, + "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole", + "networkMode": "awsvpc", + "revision": 1, + "volumes": [], + "status": "ACTIVE", + "requiresAttributes": [ + { + "name": "com.amazonaws.ecs.capability.logging-driver.awslogs" + }, + { + "name": "ecs.capability.execution-role-awslogs" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21" + }, + { + "name": "com.amazonaws.ecs.capability.task-iam-role" + }, + { + "name": "ecs.capability.execution-role-ecr-pull" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.18" + }, + { + "name": "ecs.capability.task-eni" + }, + { + "name": "com.amazonaws.ecs.capability.docker-remote-api.1.29" + } + ], + "placementConstraints": [], + "compatibilities": [ + "EC2", + "FARGATE" + ], + "requiresCompatibilities": [ + "FARGATE" + ], + "cpu": "256", + "memory": "512", + "containerDefinitions": [ + { + "name": "fargate-app", + "image": "public.ecr.aws/docker/library/httpd:latest", + "cpu": 0, + "memory": null, + "memoryReservation": null, + "links": null, + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ], + "essential": true, + "entryPoint": [ + "sh", + "-c" + ], + "command": [ + "/bin/sh -c \"echo ' Amazon ECS Sample App

Amazon ECS Sample App

Congratulations!

Your application is now running on a container in Amazon ECS.

' > /usr/local/apache2/htdocs/index.html && httpd-foreground\"" + ], + "environment": [], + "mountPoints": [], + "volumesFrom": [], + "logConfiguration": null + } + ] + } +} +``` + +The response confirms that your task definition has been registered successfully. Note the task definition ARN and revision number, which you'll use when creating the service. + +## Create the service + +A service runs and maintains a specified number of tasks simultaneously in an Amazon ECS cluster. In this section, you'll create a service that runs one instance of your task definition. + +**Set up networking** + +Before creating the service, you need to set up the networking components. First, get your default VPC ID and create a security group. + +``` +$ VPC_ID=$(aws ec2 describe-vpcs --filters "Name=is-default,Values=true" --query "Vpcs[0].VpcId" --output text) +$ echo "Using default VPC: $VPC_ID" +``` + +Output: + +``` +Using default VPC: vpc-abcd1234 +``` + +**Create a security group** + +Create a security group that allows HTTP traffic on port 80. + +``` +$ SECURITY_GROUP_ID=$(aws ec2 create-security-group \ + --group-name ecs-fargate-sg \ + --description "Security group for ECS Fargate tasks" \ + --vpc-id $VPC_ID \ + --query "GroupId" --output text) +$ echo "Created security group: $SECURITY_GROUP_ID" +``` + +Output: + +``` +Created security group: sg-abcd1234 +``` + +**Add an inbound rule** + +Add an inbound rule to allow HTTP traffic from anywhere on the internet. + +``` +$ aws ec2 authorize-security-group-ingress \ + --group-id $SECURITY_GROUP_ID \ + --protocol tcp \ + --port 80 \ + --cidr 0.0.0.0/0 +``` + +Output: + +``` +{ + "Return": true, + "SecurityGroupRules": [ + { + "SecurityGroupRuleId": "sgr-abcd1234", + "GroupId": "sg-abcd1234", + "GroupOwnerId": "123456789012", + "IsEgress": false, + "IpProtocol": "tcp", + "FromPort": 80, + "ToPort": 80, + "CidrIpv4": "0.0.0.0/0" + } + ] +} +``` + +**Get subnet information** + +Get the subnet IDs from your default VPC to use for the service network configuration. + +``` +$ SUBNET_IDS=$(aws ec2 describe-subnets \ + --filters "Name=vpc-id,Values=$VPC_ID" \ + --query "Subnets[*].SubnetId" \ + --output text | tr '\t' ',') +$ echo "Using subnets: $SUBNET_IDS" +``` + +Output: + +``` +Using subnets: subnet-abcd1234,subnet-efgh5678,subnet-ijkl9012,subnet-mnop3456 +``` + +**Create the ECS service** + +Now create the service using your task definition and network configuration. + +``` +$ aws ecs create-service \ + --cluster my-fargate-cluster \ + --service-name my-fargate-service \ + --task-definition sample-fargate \ + --desired-count 1 \ + --launch-type FARGATE \ + --network-configuration "awsvpcConfiguration={subnets=[$SUBNET_IDS],securityGroups=[$SECURITY_GROUP_ID],assignPublicIp=ENABLED}" +``` + +Output: + +``` +{ + "service": { + "serviceArn": "arn:aws:ecs:us-west-2:123456789012:service/my-fargate-cluster/my-fargate-service", + "serviceName": "my-fargate-service", + "clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/my-fargate-cluster", + "loadBalancers": [], + "serviceRegistries": [], + "status": "ACTIVE", + "desiredCount": 1, + "runningCount": 0, + "pendingCount": 0, + "launchType": "FARGATE", + "platformVersion": "LATEST", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/sample-fargate:1", + "deploymentConfiguration": { + "maximumPercent": 200, + "minimumHealthyPercent": 100 + }, + "deployments": [ + { + "id": "ecs-svc/1234567890123456789", + "status": "PRIMARY", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/sample-fargate:1", + "desiredCount": 1, + "pendingCount": 0, + "runningCount": 0, + "createdAt": 1673596800.000, + "updatedAt": 1673596800.000, + "launchType": "FARGATE", + "platformVersion": "1.4.0", + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678", + "subnet-ijkl9012", + "subnet-mnop3456" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + } + } + ], + "events": [], + "createdAt": 1673596800.000, + "placementConstraints": [], + "placementStrategy": [], + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678", + "subnet-ijkl9012", + "subnet-mnop3456" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + }, + "healthCheckGracePeriodSeconds": 0, + "schedulingStrategy": "REPLICA" + } +} +``` + +The response shows that your service has been created successfully. The service will start deploying your task, which may take a few minutes to become available. + +## View your service + +After creating the service, you can check its status and get the public IP address of your running task. + +**Wait for the service to stabilize** + +Use the following command to wait for your service to reach a stable state. + +``` +$ aws ecs wait services-stable --cluster my-fargate-cluster --services my-fargate-service +``` + +This command will wait until the service deployment is complete and the desired number of tasks are running. + +**Check service status** + +Verify that your service is running correctly. + +``` +$ aws ecs describe-services \ + --cluster my-fargate-cluster \ + --services my-fargate-service +``` +Output: + +``` +{ + "services": [ + { + "serviceArn": "arn:aws:ecs:us-west-2:123456789012:service/my-fargate-cluster/my-fargate-service", + "serviceName": "my-fargate-service", + "clusterArn": "arn:aws:ecs:us-west-2:123456789012:cluster/my-fargate-cluster", + "loadBalancers": [], + "serviceRegistries": [], + "status": "ACTIVE", + "desiredCount": 1, + "runningCount": 1, + "pendingCount": 0, + "launchType": "FARGATE", + "platformVersion": "LATEST", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/sample-fargate:1", + "deploymentConfiguration": { + "maximumPercent": 200, + "minimumHealthyPercent": 100 + }, + "deployments": [ + { + "id": "ecs-svc/1234567890123456789", + "status": "PRIMARY", + "taskDefinition": "arn:aws:ecs:us-west-2:123456789012:task-definition/sample-fargate:1", + "desiredCount": 1, + "pendingCount": 0, + "runningCount": 1, + "createdAt": 1673596800.000, + "updatedAt": 1673596800.000, + "launchType": "FARGATE", + "platformVersion": "1.4.0", + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678", + "subnet-ijkl9012", + "subnet-mnop3456" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + }, + "rolloutState": "COMPLETED" + } + ], + "events": [ + { + "id": "abcd1234-5678-90ab-cdef-1234567890ab", + "createdAt": 1673596800.000, + "message": "(service my-fargate-service) has reached a steady state." + } + ], + "createdAt": 1673596800.000, + "placementConstraints": [], + "placementStrategy": [], + "networkConfiguration": { + "awsvpcConfiguration": { + "subnets": [ + "subnet-abcd1234", + "subnet-efgh5678", + "subnet-ijkl9012", + "subnet-mnop3456" + ], + "securityGroups": [ + "sg-abcd1234" + ], + "assignPublicIp": "ENABLED" + } + }, + "healthCheckGracePeriodSeconds": 0, + "schedulingStrategy": "REPLICA" + } + ] +} +``` + +The output shows that your service is `ACTIVE` with 1 running task and has reached a steady state. + +**Get the public IP address** + +To access your application, you need to get the public IP address of the running task. + +``` +$ TASK_ARN=$(aws ecs list-tasks \ + --cluster my-fargate-cluster \ + --service-name my-fargate-service \ + --query "taskArns[0]" --output text) +$ echo "Task ARN: $TASK_ARN" +``` +Output: + +``` +Task ARN: arn:aws:ecs:us-west-2:123456789012:task/my-fargate-cluster/abcd1234567890abcdef1234567890ab +``` + +``` +$ ENI_ID=$(aws ecs describe-tasks \ + --cluster my-fargate-cluster \ + --tasks $TASK_ARN \ + --query "tasks[0].attachments[0].details[?name=='networkInterfaceId'].value" \ + --output text) +$ echo "Network Interface ID: $ENI_ID" +``` +Output: + +``` +Network Interface ID: eni-abcd1234 +``` + +``` +$ PUBLIC_IP=$(aws ec2 describe-network-interfaces \ + --network-interface-ids $ENI_ID \ + --query "NetworkInterfaces[0].Association.PublicIp" \ + --output text) +$ echo "Your application is available at: http://$PUBLIC_IP" +``` + +Output: + +``` +Your application is available at: http://203.0.113.75 +``` + +You can now open this URL in your web browser to see your running application. You should see a simple web page with the message "Amazon ECS Sample App" and "Congratulations! Your application is now running on a container in Amazon ECS." + +## Clean up + +When you're finished with this tutorial, clean up the resources to avoid incurring charges for resources you're not using. + +**Scale the service to zero tasks** + +First, scale your service down to zero tasks. + +``` +$ aws ecs update-service \ + --cluster my-fargate-cluster \ + --service my-fargate-service \ + --desired-count 0 +``` + +**Wait for the service to stabilize** + +Wait for the service to finish scaling down. + +``` +$ aws ecs wait services-stable \ + --cluster my-fargate-cluster \ + --services my-fargate-service +``` + +**Delete the service** + +Delete the service from your cluster. + +``` +$ aws ecs delete-service \ + --cluster my-fargate-cluster \ + --service my-fargate-service +``` + +**Delete the cluster** + +Delete the ECS cluster. + +``` +$ aws ecs delete-cluster --cluster my-fargate-cluster +``` + +**Delete the security group** + +Finally, delete the security group you created. + +``` +$ aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID +``` + +All resources created in this tutorial have been cleaned up. + +## Going to production + +This tutorial is designed to help you understand how Amazon ECS and AWS Fargate work. The configuration used here is suitable for learning and testing, but requires several modifications for production use. + +### Security considerations + +**Network security**: This tutorial creates a security group that allows HTTP traffic from anywhere on the internet (0.0.0.0/0). In production, you should restrict access to specific IP ranges or use an Application Load Balancer with more restrictive security groups. + +**VPC design**: The tutorial uses the default VPC for simplicity. Production applications should use a custom VPC with private subnets for tasks and public subnets for load balancers. + +**Container security**: Consider using private container registries, enabling image vulnerability scanning, and implementing least-privilege IAM policies for your tasks. + +### High availability and scalability + +**Multi-AZ deployment**: Deploy your service across multiple Availability Zones for fault tolerance. + +**Auto scaling**: Configure service auto scaling based on CPU utilization, memory utilization, or custom metrics. + +**Load balancing**: Use an Application Load Balancer to distribute traffic across multiple tasks and provide health checks. + +### Monitoring and logging + +**Container Insights**: Enable Amazon ECS Container Insights for detailed monitoring of your clusters, services, and tasks. + +**Application logging**: Configure centralized logging using Amazon CloudWatch Logs or other logging solutions. + +**Distributed tracing**: Implement AWS X-Ray for distributed tracing across your microservices. + +### Additional resources + +For comprehensive guidance on production-ready architectures and security best practices, see: + +* [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/framework/welcome.html) +* [Amazon ECS security best practices](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security.html) +* [AWS Architecture Center](https://aws.amazon.com/architecture/) +* [Amazon ECS Workshop](https://ecsworkshop.com/) + +## Next steps + +Now that you've successfully created and run an Amazon ECS task using Fargate, you can explore additional features: + +* [Amazon ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) - Learn more about configuring task definitions with advanced options like environment variables, volumes, and logging. +* [Amazon ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) - Discover how to configure load balancers, auto scaling, and service discovery for your services. +* [Amazon ECS clusters](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) - Explore cluster management, capacity providers, and container insights. +* [AWS Fargate](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Fargate.html) - Learn about Fargate platform versions, task networking, and storage options. +* [Amazon ECS security](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/security.html) - Understand security best practices for ECS tasks and services. diff --git a/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.sh b/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.sh new file mode 100755 index 00000000..0584eea8 --- /dev/null +++ b/tuts/086-amazon-ecs-fargate-linux/amazon-ecs-fargate-linux.sh @@ -0,0 +1,466 @@ +#!/bin/bash + +# Amazon ECS Fargate Tutorial Script - Version 5 +# This script creates an ECS cluster, task definition, and service using Fargate launch type +# Fixed version with proper resource dependency handling during cleanup + +set -e # Exit on any error + +# Initialize logging +LOG_FILE="ecs-fargate-tutorial-v5.log" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "Starting Amazon ECS Fargate tutorial at $(date)" +echo "Log file: $LOG_FILE" + +# Generate random identifier for unique resource names +RANDOM_ID=$(openssl rand -hex 6) +CLUSTER_NAME="fargate-cluster-${RANDOM_ID}" +SERVICE_NAME="fargate-service-${RANDOM_ID}" +TASK_FAMILY="sample-fargate-${RANDOM_ID}" +SECURITY_GROUP_NAME="ecs-fargate-sg-${RANDOM_ID}" + +# Array to track created resources for cleanup +CREATED_RESOURCES=() + +# Function to log and execute commands +execute_command() { + local cmd="$1" + local description="$2" + echo "" + echo "==========================================" + echo "EXECUTING: $description" + echo "COMMAND: $cmd" + echo "==========================================" + + local output + local exit_code + set +e # Temporarily disable exit on error + output=$(eval "$cmd" 2>&1) + exit_code=$? + set -e # Re-enable exit on error + + if [[ $exit_code -eq 0 ]]; then + echo "SUCCESS: $description" + echo "OUTPUT: $output" + return 0 + else + echo "FAILED: $description" + echo "EXIT CODE: $exit_code" + echo "OUTPUT: $output" + return 1 + fi +} + +# Function to check for actual AWS API errors in command output +check_for_aws_errors() { + local output="$1" + local description="$2" + + # Look for specific AWS error patterns, not just the word "error" + if echo "$output" | grep -qi "An error occurred\|InvalidParameter\|AccessDenied\|ResourceNotFound\|ValidationException"; then + echo "AWS API ERROR detected in output for: $description" + echo "Output: $output" + return 1 + fi + return 0 +} + +# Function to wait for network interfaces to be cleaned up +wait_for_network_interfaces_cleanup() { + local security_group_id="$1" + local max_attempts=30 + local attempt=1 + + echo "Waiting for network interfaces to be cleaned up..." + + while [[ $attempt -le $max_attempts ]]; do + echo "Attempt $attempt/$max_attempts: Checking for dependent network interfaces..." + + # Check if there are any network interfaces still using this security group + local eni_count + eni_count=$(aws ec2 describe-network-interfaces \ + --filters "Name=group-id,Values=$security_group_id" \ + --query "length(NetworkInterfaces)" \ + --output text 2>/dev/null || echo "0") + + if [[ "$eni_count" == "0" ]]; then + echo "No network interfaces found using security group $security_group_id" + return 0 + else + echo "Found $eni_count network interface(s) still using security group $security_group_id" + echo "Waiting 10 seconds before next check..." + sleep 10 + ((attempt++)) + fi + done + + echo "WARNING: Network interfaces may still be attached after $max_attempts attempts" + echo "This is normal and the security group deletion will be retried" + return 1 +} + +# Function to retry security group deletion with exponential backoff +retry_security_group_deletion() { + local security_group_id="$1" + local max_attempts=10 + local attempt=1 + local wait_time=5 + + while [[ $attempt -le $max_attempts ]]; do + echo "Attempt $attempt/$max_attempts: Trying to delete security group $security_group_id" + + if execute_command "aws ec2 delete-security-group --group-id $security_group_id" "Delete security group (attempt $attempt)"; then + echo "Successfully deleted security group $security_group_id" + return 0 + else + if [[ $attempt -eq $max_attempts ]]; then + echo "FAILED: Could not delete security group $security_group_id after $max_attempts attempts" + echo "This may be due to network interfaces that are still being cleaned up by AWS" + echo "You can manually delete it later using: aws ec2 delete-security-group --group-id $security_group_id" + return 1 + else + echo "Waiting $wait_time seconds before retry..." + sleep $wait_time + wait_time=$((wait_time * 2)) # Exponential backoff + ((attempt++)) + fi + fi + done +} + +# Function to cleanup resources with proper dependency handling +cleanup_resources() { + echo "" + echo "===========================================" + echo "CLEANUP PROCESS" + echo "===========================================" + echo "The following resources were created:" + for resource in "${CREATED_RESOURCES[@]}"; do + echo " - $resource" + done + echo "" + echo "Do you want to clean up all created resources? (y/n): " + read -r CLEANUP_CHOICE + + if [[ "$CLEANUP_CHOICE" =~ ^[Yy]$ ]]; then + echo "Starting cleanup process..." + + # Step 1: Scale service to 0 tasks first, then delete service + if [[ " ${CREATED_RESOURCES[*]} " =~ " ECS Service: $SERVICE_NAME " ]]; then + echo "" + echo "Step 1: Scaling service to 0 tasks..." + if execute_command "aws ecs update-service --cluster $CLUSTER_NAME --service $SERVICE_NAME --desired-count 0" "Scale service to 0 tasks"; then + echo "Waiting for service to stabilize after scaling to 0..." + execute_command "aws ecs wait services-stable --cluster $CLUSTER_NAME --services $SERVICE_NAME" "Wait for service to stabilize" + + echo "Deleting service..." + execute_command "aws ecs delete-service --cluster $CLUSTER_NAME --service $SERVICE_NAME" "Delete ECS service" + else + echo "WARNING: Failed to scale service. Attempting to delete anyway..." + execute_command "aws ecs delete-service --cluster $CLUSTER_NAME --service $SERVICE_NAME --force" "Force delete ECS service" + fi + fi + + # Step 2: Wait a bit for tasks to fully terminate + echo "" + echo "Step 2: Waiting for tasks to fully terminate..." + sleep 15 + + # Step 3: Delete cluster + if [[ " ${CREATED_RESOURCES[*]} " =~ " ECS Cluster: $CLUSTER_NAME " ]]; then + echo "" + echo "Step 3: Deleting cluster..." + execute_command "aws ecs delete-cluster --cluster $CLUSTER_NAME" "Delete ECS cluster" + fi + + # Step 4: Wait for network interfaces to be cleaned up, then delete security group + if [[ -n "$SECURITY_GROUP_ID" ]]; then + echo "" + echo "Step 4: Cleaning up security group..." + + # First, wait for network interfaces to be cleaned up + wait_for_network_interfaces_cleanup "$SECURITY_GROUP_ID" + + # Then retry security group deletion with backoff + retry_security_group_deletion "$SECURITY_GROUP_ID" + fi + + # Step 5: Clean up task definition (deregister all revisions) + if [[ " ${CREATED_RESOURCES[*]} " =~ " Task Definition: $TASK_FAMILY " ]]; then + echo "" + echo "Step 5: Deregistering task definition revisions..." + + # Get all revisions of the task definition + local revisions + revisions=$(aws ecs list-task-definitions --family-prefix "$TASK_FAMILY" --query "taskDefinitionArns" --output text 2>/dev/null || echo "") + + if [[ -n "$revisions" && "$revisions" != "None" ]]; then + for revision_arn in $revisions; do + echo "Deregistering task definition: $revision_arn" + execute_command "aws ecs deregister-task-definition --task-definition $revision_arn" "Deregister task definition $revision_arn" || true + done + else + echo "No task definition revisions found to deregister" + fi + fi + + echo "" + echo "===========================================" + echo "CLEANUP COMPLETED" + echo "===========================================" + echo "All resources have been cleaned up successfully!" + + else + echo "Cleanup skipped. Resources remain active." + echo "" + echo "To clean up manually later, use the following commands in order:" + echo "1. Scale service to 0: aws ecs update-service --cluster $CLUSTER_NAME --service $SERVICE_NAME --desired-count 0" + echo "2. Wait for stability: aws ecs wait services-stable --cluster $CLUSTER_NAME --services $SERVICE_NAME" + echo "3. Delete service: aws ecs delete-service --cluster $CLUSTER_NAME --service $SERVICE_NAME" + echo "4. Delete cluster: aws ecs delete-cluster --cluster $CLUSTER_NAME" + echo "5. Wait 2-3 minutes, then delete security group: aws ec2 delete-security-group --group-id $SECURITY_GROUP_ID" + if [[ " ${CREATED_RESOURCES[*]} " =~ " Task Definition: $TASK_FAMILY " ]]; then + echo "6. Deregister task definitions: aws ecs list-task-definitions --family-prefix $TASK_FAMILY" + echo " Then for each ARN: aws ecs deregister-task-definition --task-definition " + fi + fi +} + +# Trap to handle script interruption +trap cleanup_resources EXIT + +echo "Using random identifier: $RANDOM_ID" +echo "Cluster name: $CLUSTER_NAME" +echo "Service name: $SERVICE_NAME" +echo "Task family: $TASK_FAMILY" + +# Step 1: Ensure ECS task execution role exists +echo "" +echo "===========================================" +echo "STEP 1: VERIFY ECS TASK EXECUTION ROLE" +echo "===========================================" + +ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) +EXECUTION_ROLE_ARN="arn:aws:iam::${ACCOUNT_ID}:role/ecsTaskExecutionRole" + +# Check if role exists +if aws iam get-role --role-name ecsTaskExecutionRole >/dev/null 2>&1; then + echo "ECS task execution role already exists" +else + echo "Creating ECS task execution role..." + + # Create trust policy + cat > trust-policy.json << 'EOF' +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "ecs-tasks.amazonaws.com" + }, + "Action": "sts:AssumeRole" + } + ] +} +EOF + + execute_command "aws iam create-role --role-name ecsTaskExecutionRole --assume-role-policy-document file://trust-policy.json" "Create ECS task execution role" + + execute_command "aws iam attach-role-policy --role-name ecsTaskExecutionRole --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy" "Attach ECS task execution policy" + + # Clean up temporary file + rm -f trust-policy.json + + CREATED_RESOURCES+=("IAM Role: ecsTaskExecutionRole") +fi + +# Step 2: Create ECS cluster +echo "" +echo "===========================================" +echo "STEP 2: CREATE ECS CLUSTER" +echo "===========================================" + +CLUSTER_OUTPUT=$(execute_command "aws ecs create-cluster --cluster-name $CLUSTER_NAME" "Create ECS cluster") +check_for_aws_errors "$CLUSTER_OUTPUT" "Create ECS cluster" + +CREATED_RESOURCES+=("ECS Cluster: $CLUSTER_NAME") + +# Step 3: Create task definition +echo "" +echo "===========================================" +echo "STEP 3: CREATE TASK DEFINITION" +echo "===========================================" + +# Create task definition JSON +cat > task-definition.json << EOF +{ + "family": "$TASK_FAMILY", + "networkMode": "awsvpc", + "requiresCompatibilities": ["FARGATE"], + "cpu": "256", + "memory": "512", + "executionRoleArn": "$EXECUTION_ROLE_ARN", + "containerDefinitions": [ + { + "name": "fargate-app", + "image": "public.ecr.aws/docker/library/httpd:latest", + "portMappings": [ + { + "containerPort": 80, + "hostPort": 80, + "protocol": "tcp" + } + ], + "essential": true, + "entryPoint": ["sh", "-c"], + "command": [ + "/bin/sh -c \"echo ' Amazon ECS Sample App

Amazon ECS Sample App

Congratulations!

Your application is now running on a container in Amazon ECS.

' > /usr/local/apache2/htdocs/index.html && httpd-foreground\"" + ] + } + ] +} +EOF + +TASK_DEF_OUTPUT=$(execute_command "aws ecs register-task-definition --cli-input-json file://task-definition.json" "Register task definition") +check_for_aws_errors "$TASK_DEF_OUTPUT" "Register task definition" + +# Clean up temporary file +rm -f task-definition.json + +CREATED_RESOURCES+=("Task Definition: $TASK_FAMILY") + +# Step 4: Set up networking +echo "" +echo "===========================================" +echo "STEP 4: SET UP NETWORKING" +echo "===========================================" + +# Get default VPC ID +VPC_ID=$(aws ec2 describe-vpcs --filters "Name=is-default,Values=true" --query "Vpcs[0].VpcId" --output text) +if [[ "$VPC_ID" == "None" || -z "$VPC_ID" ]]; then + echo "ERROR: No default VPC found. Please create a default VPC or specify a custom VPC." + exit 1 +fi +echo "Using default VPC: $VPC_ID" + +# Create security group with restricted access +# Note: This allows HTTP access from anywhere for demo purposes +# In production, restrict source to specific IP ranges or security groups +SECURITY_GROUP_OUTPUT=$(execute_command "aws ec2 create-security-group --group-name $SECURITY_GROUP_NAME --description 'Security group for ECS Fargate tutorial - HTTP access' --vpc-id $VPC_ID" "Create security group") +check_for_aws_errors "$SECURITY_GROUP_OUTPUT" "Create security group" + +SECURITY_GROUP_ID=$(echo "$SECURITY_GROUP_OUTPUT" | grep -o '"GroupId": "[^"]*"' | cut -d'"' -f4) +if [[ -z "$SECURITY_GROUP_ID" ]]; then + SECURITY_GROUP_ID=$(aws ec2 describe-security-groups --group-names "$SECURITY_GROUP_NAME" --query "SecurityGroups[0].GroupId" --output text) +fi + +echo "Created security group: $SECURITY_GROUP_ID" +CREATED_RESOURCES+=("Security Group: $SECURITY_GROUP_ID") + +# Add HTTP inbound rule +# WARNING: This allows HTTP access from anywhere (0.0.0.0/0) +# In production environments, restrict this to specific IP ranges +execute_command "aws ec2 authorize-security-group-ingress --group-id $SECURITY_GROUP_ID --protocol tcp --port 80 --cidr 0.0.0.0/0" "Add HTTP inbound rule to security group" + +# Get subnet IDs from default VPC +echo "Getting subnet IDs from default VPC..." +SUBNET_IDS_RAW=$(aws ec2 describe-subnets --filters "Name=vpc-id,Values=$VPC_ID" --query "Subnets[*].SubnetId" --output text) +if [[ -z "$SUBNET_IDS_RAW" ]]; then + echo "ERROR: No subnets found in default VPC" + exit 1 +fi + +# Convert to proper comma-separated format, handling both spaces and tabs +SUBNET_IDS_COMMA=$(echo "$SUBNET_IDS_RAW" | tr -s '[:space:]' ',' | sed 's/,$//') +echo "Raw subnet IDs: $SUBNET_IDS_RAW" +echo "Formatted subnet IDs: $SUBNET_IDS_COMMA" + +# Validate subnet IDs format +if [[ ! "$SUBNET_IDS_COMMA" =~ ^subnet-[a-z0-9]+(,subnet-[a-z0-9]+)*$ ]]; then + echo "ERROR: Invalid subnet ID format: $SUBNET_IDS_COMMA" + exit 1 +fi + +# Step 5: Create ECS service +echo "" +echo "===========================================" +echo "STEP 5: CREATE ECS SERVICE" +echo "===========================================" + +# Create the service with proper JSON formatting for network configuration +SERVICE_CMD="aws ecs create-service --cluster $CLUSTER_NAME --service-name $SERVICE_NAME --task-definition $TASK_FAMILY --desired-count 1 --launch-type FARGATE --network-configuration '{\"awsvpcConfiguration\":{\"subnets\":[\"$(echo $SUBNET_IDS_COMMA | sed 's/,/","/g')\"],\"securityGroups\":[\"$SECURITY_GROUP_ID\"],\"assignPublicIp\":\"ENABLED\"}}'" + +echo "Service creation command: $SERVICE_CMD" + +SERVICE_OUTPUT=$(execute_command "$SERVICE_CMD" "Create ECS service") +check_for_aws_errors "$SERVICE_OUTPUT" "Create ECS service" + +CREATED_RESOURCES+=("ECS Service: $SERVICE_NAME") + +# Step 6: Wait for service to stabilize and get public IP +echo "" +echo "===========================================" +echo "STEP 6: WAIT FOR SERVICE AND GET PUBLIC IP" +echo "===========================================" + +echo "Waiting for service to stabilize (this may take a few minutes)..." +execute_command "aws ecs wait services-stable --cluster $CLUSTER_NAME --services $SERVICE_NAME" "Wait for service to stabilize" + +# Get task ARN +TASK_ARN=$(aws ecs list-tasks --cluster $CLUSTER_NAME --service-name $SERVICE_NAME --query "taskArns[0]" --output text) +if [[ "$TASK_ARN" == "None" || -z "$TASK_ARN" ]]; then + echo "ERROR: No running tasks found for service" + exit 1 +fi + +echo "Task ARN: $TASK_ARN" + +# Get network interface ID +ENI_ID=$(aws ecs describe-tasks --cluster $CLUSTER_NAME --tasks $TASK_ARN --query "tasks[0].attachments[0].details[?name=='networkInterfaceId'].value" --output text) +if [[ "$ENI_ID" == "None" || -z "$ENI_ID" ]]; then + echo "ERROR: Could not retrieve network interface ID" + exit 1 +fi + +echo "Network Interface ID: $ENI_ID" + +# Get public IP +PUBLIC_IP=$(aws ec2 describe-network-interfaces --network-interface-ids $ENI_ID --query "NetworkInterfaces[0].Association.PublicIp" --output text) +if [[ "$PUBLIC_IP" == "None" || -z "$PUBLIC_IP" ]]; then + echo "WARNING: No public IP assigned to the task" + echo "The task may be in a private subnet or public IP assignment failed" +else + echo "" + echo "===========================================" + echo "SUCCESS! APPLICATION IS RUNNING" + echo "===========================================" + echo "Your application is available at: http://$PUBLIC_IP" + echo "You can test it by opening this URL in your browser" + echo "" +fi + +# Display service information +echo "" +echo "===========================================" +echo "SERVICE INFORMATION" +echo "===========================================" +execute_command "aws ecs describe-services --cluster $CLUSTER_NAME --services $SERVICE_NAME" "Get service details" + +echo "" +echo "===========================================" +echo "TUTORIAL COMPLETED SUCCESSFULLY" +echo "===========================================" +echo "Resources created:" +for resource in "${CREATED_RESOURCES[@]}"; do + echo " - $resource" +done + +if [[ -n "$PUBLIC_IP" && "$PUBLIC_IP" != "None" ]]; then + echo "" + echo "Application URL: http://$PUBLIC_IP" +fi + +echo "" +echo "Script completed at $(date)"