Use AWS CLI v2 #214
Use AWS CLI v2 #214
Conversation
We were running into dependency issues with awscli v1, which was installed via Pip.¹ Update to AWS CLI v2 which is _not_ installed via Pip. Instead of working around the absolute path symlinks created by running `./aws/install`, we are "just installing the files ourselves"² and creating the symlink for `/final/bin/aws`. ¹ <#213> ² <#214 (comment)>
joverlee521
force-pushed
the
use-aws-cli-v2
branch
from
f06e64c
to
ef3577f
Compare
AWS CLI v2 runs into errors if the region is not defined¹ The AWS_DEFAULT_REGION variable is available in the GH Action CI after the aws-actions/configure-aws-credentials@v4 step. ¹ <actions/runner-images#2791 (comment)>
joverlee521
force-pushed
the
use-aws-cli-v2
branch
from
9b233f9
to
1591ad7
Compare
| @@ -24,7 +24,7 @@ Files are populated from NEXTSTRAIN_ENVD_URL. | |||
|
|
|||
| $ aws s3 cp --quiet env.d.zip "$NEXTSTRAIN_ENVD_URL" | |||
|
|
|||
| $ docker run --rm -e NEXTSTRAIN_ENVD_URL --env=AWS_{ACCESS_KEY_ID,SECRET_ACCESS_KEY,SESSION_TOKEN} "$IMAGE" \ | |||
| $ docker run --rm -e NEXTSTRAIN_ENVD_URL --env=AWS_{ACCESS_KEY_ID,SECRET_ACCESS_KEY,SESSION_TOKEN,DEFAULT_REGION} "$IMAGE" \ | |||
There was a problem hiding this comment.
AWS CLI v2 runs into errors if the region is not defined¹
Uh... so does this mean that all our usages of the Docker/AWS Batch/Singularity runtimes which call aws are going to have to ensure they have AWS_DEFAULT_REGION set? That seems like a unexpected breaking change. Hmm.
There was a problem hiding this comment.
If that's an issue we don't want to deal with now, we can also install AWS CLI v1 using the bundled installer
There was a problem hiding this comment.
Maybe? Or maybe we could provide a default AWS_DEFAULT_REGION that's backwards compat with v1? (What does it do if region isn't required?) Or maybe we bite the bullet and fix anything that breaks (although this means breaking other people's uses too).
Maybe we can talk thru this next week? I'm not really sure I have all the info at hand here to help make a decision (and it's Friday afternoon and my brain is fried).
There was a problem hiding this comment.
+1 (albeit from deep on the sidelines) for not making this type of decision on Friday afternoon, particularly one with lovely (distractingly) sunny weather...
Otherwise, the awscli → botocore dependency is broken by the subsequent installation of the nextstrain-cli → botocore dependency, which results in --no-sign-request not working as it should. This approach seemed like the option of least change. Alternatives to and/or additional improvements to this include a) installing Nextstrain CLI in its own venv instead/as well, b) upgrading to AWS CLI v2 (which is always isolated) or c) updating the nextstrain-cli → botocore dependency to not conflict. Resolves: <#213> Related-to: <#214>
|
Closing in favor of #215 since we don't want to deal with the fallout now while we are dealing with fallout from switch to OIDC in GH Action workflows. We will definitely want to revisit this later though! |
Description of proposed changes
Update AWS CLI to v2
We were running into dependency issues with awscli v1, which was
installed via Pip.¹
Update to AWS CLI v2 which is not installed via Pip.
Instead of working around the absolute path symlinks created by running
./aws/install, we are "just installing the files ourselves"² andcreating the symlink for
/final/bin/aws.¹ #213
² #214 (comment)
Related issue(s)
Resolves #213
Checklist