Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --no-sign-request flag to 'aws s3 cp' #59

Merged
merged 1 commit into from
Aug 26, 2021

Conversation

bsmith89
Copy link
Collaborator

@bsmith89 bsmith89 commented Aug 26, 2021

Here's the minimum-viable patch to iggtools that allows me to download with aws s3 cp --only-show-errors --no-sign-request s3://microbiome-igg/<SOME-FILE-PATH>.

When the new flag is excluded I get download failures like the following:

1629844368.0:  'aws s3 cp --only-show-errors s3://microbiome-igg/2.0/pangenomes/100022/centroids.ffn.lz4 - | lz4 -dc > ref/IGGdb2.0/pangenomes/100022/centroi
ds.ffn'
download failed: s3://microbiome-igg/2.0/pangenomes/100022/centroids.ffn.lz4 to - Unable to locate credentials
1629844532.5:  'rm -f ref/IGGdb2.0/pangenomes/100022/centroids.ffn'
1629844533.5:  Sleeping 54.31506110122879 seconds before retry 1 of <function download_reference at 0x7f1062e56d40> with ('s3://microbiome-igg/2.0/pangenomes
/100022/centroids.ffn.lz4', 'ref/IGGdb2.0/pangenomes/100022'), {}.

@bsmith89
Copy link
Collaborator Author

Just realized this was basically an exact duplicate of #31 ...

I'll leave it open for now, though, because that branch has conflicts with the current HEAD of the master branch.

@zhaoc1
Copy link
Contributor

zhaoc1 commented Aug 26, 2021

Thanks Byron! I haven't got time to tackle AWS related issues. I am gonna merge your changes!

@zhaoc1 zhaoc1 merged commit 9003faf into czbiohub-sf:master Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants