Add support for ECS exec

[arnaldo2792]

Issue number:

Closes #1649

Description of changes:
This includes the changes required to support ECS exec in Bottlerocket.

Under the hood, the ECS agent runs the Amazon SSM agent in each container configured to use EXEC. The ECS agent bind-mounts the statically-linked SSM agent's binaries (amazon-ssm-agent, ssm-agent-worker, ssm-session-worker) onto each container and uses docker to execute them within the container's namespaces.

In the ECS Optimized AMIs, the ECS agent looks at the path /managed-agents for the dependencies needed to use ECS exec. This directory is a bind-mount to /var/lib/ecs/deps/. The ECS agent checks that the bin, config and certs directories exist to mark the instances as “capable” to run ECS exec. The ECS agent requires the SSM binaries to be in a versioned directory within bin, and a CA bundle named tls-ca-bundle.pem to be available in certs. The agent uses config to store configurations created at runtime for each ECS exec session, so it needs to be writable by agent.

In this PR, a patch for the ECS agent updates the path used to check for dependencies from /managed-agents to /usr/libexec/amazon-ecs-agent/managed-agents. This is because in Bottlerocket the ECS agent runs as a system service instead of a docker container, and /usr/libexec is used to store binaries that are used by other programs.

The config directory for the SSM sessions is a symbolic link to /var/ecs/managed-agents/execute-command/config to allow the ECS agent generate the required configurations at runtime. This directory is created via tmpfiles.d. The certs directory contains a symbolic link to the CA Bundle of the host. The bin directory is a symbolic link to the directory that contains the versioned SSM binaries.

Testing done:

• SSM exec tests in the ECS conformance test suite pass.
• I validated the ECS exec sessions don't interfere with the SSM agent in the control host container
• I validated I can exec into a task

[ssm-user@control]$ apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "c76b9874",
    "pretty_name": "Bottlerocket OS 1.14.0 (aws-ecs-1)",
    "variant_id": "aws-ecs-1",
    "version_id": "1.14.0"
  }
}

#...

~ ❯ aws ecs execute-command --cluster bottlerocket \
    --task <task-arn> --container fedora \
    --interactive --command "cat /etc/os-release" | rg PRETTY_NAME
PRETTY_NAME="Fedora Linux 35 (Container Image)"

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

[arnaldo2792]

Builds will fail until I update the look aside cache

[bcressey]

You are confident this is another bind mount from the host, and not a CA cert bundle that the agent retrieves from elsewhere and manages?

[arnaldo2792]

Yes, this is another bind mount:

ecs-exec on  ecs-exec [$] ❯ aws ecs execute-command \
    --cluster bottlerocket --task <task-id> --container fedora \
    --interactive --command "cat /proc/self/mountinfo" | rg tls
466 439 0:30 /pki/tls/certs/ca-bundle.crt /ecs-execute-command-384f4d6e-39e3-41ee-9e00-609338445345/certs/amazon-ssm-agent.crt ro,noatime - tmpfs tmpfs rw,context=system_u:object_r:etc_t:s0,mode=755

The agent does not manage the file. Even in the ECS anywhere script, the host's CA bundle is copied to this location:

https://github.com/aws/amazon-ecs-init/blob/master/scripts/ecs-anywhere-install.sh#L595

[bcressey]

Can the patch just point the agent to /etc/pki/tls/certs/ca-bundle.crt then? Then we wouldn't need a symlink.

[arnaldo2792]

We could do this, however the patch will be bigger since there are other places in the agent that should be updated to reference this file, e.g.

https://github.com/aws/amazon-ecs-agent/blob/master/agent/engine/execcmd/manager_init_task_linux.go#L140

[arnaldo2792]

(Forced push addresses comments above)

[zmrow]

LGTM!

[yeazelm]

LGTM! 🚢