add exec prerequisites to ecs-anywhere installation script #451

lydiafilipe · 2021-11-10T18:58:17Z

Summary

This pr makes changes to the ECS Anywhere installation script in order for exec to be supported on ECS Anywhere. Two changes are required - copying TLS certs and downloading ssm binaries. Both of these will be moved to a directory that will be bind-mounted into the customer's container at start, which is consistent with the current behavior for ECS exec on EC2.

For further context on exec, see:

Implementation details

The changes to installation script are:

Move certs. Looking for certs in the same places as Golang and moving them to the directory that will be bind-mounted
Add optional flag for the certs file. If host certs are not in any of searched locations, the customer can provide the path to the certs file
Download SSM binaries. Downloading from new public URL from the SSM team and moving to the directory that will be bind-mounted

Testing

Ran script on ubuntu, centos, and debian systems to register to ecs cluster
Ran tasks on these container instances with exec enabled and used exec
Passed cert location with the new flag both for an existing and non existing file and verified behavior was as expected

Description for the changelog

Add exec prerequisites to ecs-anywhere installation script

Licensing

This contribution is under the terms of the Apache 2.0 License: yes

ubhattacharjya · 2021-11-10T20:48:33Z

scripts/ecs-anywhere-install.sh

+    cd && mkdir -p ssm-binaries && cd ssm-binaries
+    case $ARCHITECTURE in
+        'x86_64')
+            curl https://s3.${REGION}.amazonaws.com/amazon-ssm-${REGION}/${BINARY_VERSION}/linux_amd64/amazon-ssm-agent-binaries.tar.gz -o amazon-ssm-agent.tar.gz


For CN partition, I think, this S3 url will be a bit different -> s3.${REGION}.amazonaws.com.cn

ubhattacharjya · 2021-11-10T20:51:52Z

scripts/ecs-anywhere-install.sh

+    fi
+
+    if [ -z "$certs" ]; then
+        echo "Could not find certificates. Please generate TLS certificates, or provide a valid file path to them"


Can this message ask the user to rerun the script with the --certs-file flag to provide the vaild certificate path?

sharanyad · 2021-11-11T18:14:18Z

scripts/ecs-anywhere-install.sh

@@ -35,6 +35,8 @@ usage() {
        (optional) Version of the ECS agent rpm/deb package to use. If not specified, default to latest.
  --skip-registration
        (optional) if this is enabled, SSM agent install and instance registration with SSM is skipped.
+  --certs-file
+        (optional) file for TLS certs. Defaults to predefined location for a given platform.


Could we add more details such as - TLS certs for Execute command feature
might also consider renaming flag to exec-certs-file

Realmonia · 2021-11-15T19:44:21Z

scripts/ecs-anywhere-install.sh

+}
+
+download-ssm-binaries-exec() {
+    BINARY_VERSION="3.1.338.0"


Is there a specific reason we use this version? Is there a latest that we can use?

There will be a stable version endpoint available 12/08. We're hardcoding version for now to access what's currently available, but around that date I'll update the script in this branch to use that

we're currently using 3.1.90.0 in the ecs ami: https://github.com/aws/amazon-ecs-ami/blob/main/variables.pkr.hcl#L59

we should probably use the same version, either by downgrading this version or following up with an ecs ami PR to update the version the AMI uses.

If we do update, might be worth bumping both to SSM's latest available version which is 3.1.501.0: https://github.com/aws/amazon-ssm-agent/releases

Thanks for pointing that out, it looks like there are a couple places where this might be inconsistent. For now at least I'll update to 3.1.501.0 in this script, though before releasing this we will be updating to a url that just points to the latest version. As part of that, I can look into how we can get that elsewhere and remove some of the hardcoded versions

angelcar · 2021-11-16T17:31:58Z

scripts/ecs-anywhere-install.sh

+        fail
+    else
+        echo "Using $certs"
+        mkdir -p $CERTS_PATH && cd $CERTS_PATH


not sure if cd would break other assumptions in the script now or in the future. Perhaps it's more convenient to not change the current dir and use absolute paths?

chienhanlin · 2021-11-16T23:45:10Z

scripts/ecs-anywhere-install.sh

+            echo "Provided certs file does not exist, looking for certs in known possible locations"
+        fi
+        for f in "${CERTS_FILES_LOCATIONS[@]}"
+        do


Q. Just a thought. Would it provide better UX if we printing out what certs have seen searched?

echo "Searching certs file `/etc/ssl/certs/ca-certificates.crt` " ... echo "Could not find `/etc/ssl/certs/ca-certificates.crt`"

Interesting point. I can see why some users might want to know these if the script doesn't find the certs. My thinking against doing that is that it would pollute the output a bit to log all of them, and that it's not super useful for the user to know given that these are often already present and easily able to be found just by looking at the script. Also, users can simply specify the location if the certs are not found by default. Thoughts?

I agree doing this will a kind of messing up output, and users can always provide a path if the certs cannot be found.

lydiafilipe · 2021-11-17T00:49:54Z

(accidentally removed mkdir $CERTS_PATH in last revision, added that back, which is the only difference)

ubhattacharjya reviewed Nov 10, 2021

View reviewed changes

lydiafilipe force-pushed the feature/ecs-anywhere-exec branch from 4cf7c11 to c368be7 Compare November 10, 2021 23:13

sharanyad reviewed Nov 11, 2021

View reviewed changes

lydiafilipe force-pushed the feature/ecs-anywhere-exec branch from c368be7 to b5275a9 Compare November 11, 2021 23:56

Realmonia reviewed Nov 15, 2021

View reviewed changes

Realmonia previously approved these changes Nov 15, 2021

View reviewed changes

angelcar reviewed Nov 16, 2021

View reviewed changes

lydiafilipe dismissed Realmonia’s stale review via ab54fe8 November 16, 2021 21:17

lydiafilipe force-pushed the feature/ecs-anywhere-exec branch from b5275a9 to ab54fe8 Compare November 16, 2021 21:17

angelcar previously approved these changes Nov 16, 2021

View reviewed changes

chienhanlin reviewed Nov 16, 2021

View reviewed changes

chienhanlin previously approved these changes Nov 17, 2021

View reviewed changes

add exec prerequisites to ecs-anywhere installation script

266badd

lydiafilipe dismissed stale reviews from chienhanlin and angelcar via 266badd November 17, 2021 00:48

lydiafilipe force-pushed the feature/ecs-anywhere-exec branch from ab54fe8 to 266badd Compare November 17, 2021 00:48

angelcar approved these changes Nov 17, 2021

View reviewed changes

Realmonia approved these changes Nov 17, 2021

View reviewed changes

lydiafilipe merged commit 132a402 into aws:feature/ecs-anywhere-exec Nov 17, 2021

lydiafilipe mentioned this pull request Jan 5, 2022

add exec prerequisites to ecs-anywhere installation script #466

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add exec prerequisites to ecs-anywhere installation script #451

add exec prerequisites to ecs-anywhere installation script #451

lydiafilipe commented Nov 10, 2021

ubhattacharjya Nov 10, 2021

ubhattacharjya Nov 10, 2021

sharanyad Nov 11, 2021 •

edited

Loading

Realmonia Nov 15, 2021

lydiafilipe Nov 15, 2021

sparrc Nov 16, 2021 •

edited

Loading

lydiafilipe Nov 16, 2021

angelcar Nov 16, 2021

chienhanlin Nov 16, 2021

lydiafilipe Nov 17, 2021 •

edited

Loading

chienhanlin Nov 17, 2021

lydiafilipe commented Nov 17, 2021 •

edited

Loading

add exec prerequisites to ecs-anywhere installation script #451

add exec prerequisites to ecs-anywhere installation script #451

Conversation

lydiafilipe commented Nov 10, 2021

Summary

Implementation details

Testing

Description for the changelog

Licensing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sharanyad Nov 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sparrc Nov 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lydiafilipe Nov 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lydiafilipe commented Nov 17, 2021 • edited Loading

sharanyad Nov 11, 2021 •

edited

Loading

sparrc Nov 16, 2021 •

edited

Loading

lydiafilipe Nov 17, 2021 •

edited

Loading

lydiafilipe commented Nov 17, 2021 •

edited

Loading