ci: use amazon/aws-cli docker image to avoid host dependency#662
Conversation
There was a problem hiding this comment.
Pull request overview
This PR aims to remove a host dependency on the aws CLI in CI by running AWS CLI commands via the amazon/aws-cli Docker image.
Changes:
- Replace host
aws ecr ...usage inci.shwithdocker run amazon/aws-cli .... - Replace
aws s3 cp ...usage intest/run_test.shwithdocker run amazon/aws-cli ...for log uploads.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
test/run_test.sh |
Switches S3 log upload from aws binary to an amazon/aws-cli container. |
ci.sh |
Switches ECR login/repo creation from aws binary to an amazon/aws-cli container. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| DESTINATION="s3://${BUCKET}/aperturedb-${NOW}-${FILTER// /_}.tgz" | ||
| tar czf ${ARCHIVE_NAME} ${APERTUREDB_LOG_PATH}/.. | ||
| aws s3 cp ${ARCHIVE_NAME} $DESTINATION | ||
| docker run --rm -v $(pwd):/workspace -w /workspace -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli s3 cp ${ARCHIVE_NAME} $DESTINATION |
There was a problem hiding this comment.
run_test.sh is executed inside the aperturedb-python-tests container (see docker/tests/scripts/start.sh), and that image/run invocation does not provide a Docker daemon/CLI (no /var/run/docker.sock mount, and Docker isn't installed). As a result, replacing aws s3 cp ... with docker run ... amazon/aws-cli ... will fail exactly when tests fail (when we most need log upload). Keep using the in-image aws CLI (already installed in docker/tests/Dockerfile), or move the S3 upload step to the host in run_test_container.sh where Docker is available.
| docker run --rm -v $(pwd):/workspace -w /workspace -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli s3 cp ${ARCHIVE_NAME} $DESTINATION | |
| aws s3 cp ${ARCHIVE_NAME} $DESTINATION |
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr get-login-password --region ${REGION} | docker login --username AWS --password-stdin 684446431133.dkr.ecr.${REGION}.amazonaws.com | ||
|
|
||
| aws ecr create-repository --repository-name ${ECR_REPO_NAME} --region us-west-2 || true | ||
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr create-repository --repository-name ${ECR_REPO_NAME} --region us-west-2 || true |
There was a problem hiding this comment.
The amazon/aws-cli image is referenced without a tag or digest. This makes CI behavior non-deterministic (upstream image updates can change CLI behavior) and increases supply-chain risk. Pin to a specific major/minor tag (e.g., amazon/aws-cli:2) or, ideally, an immutable digest so CI runs are reproducible.
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr get-login-password --region ${REGION} | docker login --username AWS --password-stdin 684446431133.dkr.ecr.${REGION}.amazonaws.com | ||
|
|
||
| aws ecr create-repository --repository-name ${ECR_REPO_NAME} --region us-west-2 || true | ||
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr create-repository --repository-name ${ECR_REPO_NAME} --region us-west-2 || true |
There was a problem hiding this comment.
push_aws_ecr sets REGION=us-west-2, but the create-repository command hardcodes --region us-west-2 instead of using ${REGION}. This makes the function easier to accidentally break if REGION ever changes; use the variable consistently.
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr create-repository --repository-name ${ECR_REPO_NAME} --region us-west-2 || true | |
| docker run --rm -e AWS_ACCESS_KEY_ID -e AWS_DEFAULT_REGION -e AWS_SECRET_ACCESS_KEY amazon/aws-cli ecr create-repository --repository-name ${ECR_REPO_NAME} --region ${REGION} || true |
This fixes a CI failure on the benchmark runners where the
awsCLI binary is not installed, by running it through the officialamazon/aws-clicontainer inci.sh.