Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix v2 aws sync commands #296

Open
chrisjkuch opened this issue Dec 4, 2022 · 0 comments
Open

Fix v2 aws sync commands #296

chrisjkuch opened this issue Dec 4, 2022 · 0 comments
Assignees
Labels

Comments

@chrisjkuch
Copy link
Collaborator

chrisjkuch commented Dec 4, 2022

aws s3 sync s3://{{ cookiecutter.dataset_storage.s3.bucket }}/data/ data/\

In v2, sync_data_up incorrectly lists the bucket as the source directory, rather than the local data/ folder. The order should be filpped.

In addition, there is inconsistent use of environment variables. sync_data_down uses the templatized AWS profile from the cookiecutter form:

data/ {% if cookiecutter.dataset_storage.s3.aws_profile != 'default' %} --profile {{ cookiecutter.dataset_storage.s3.aws_profile }}{% endif %}

while sync_data_up uses the PROFILE environment variable which is not set in the Makefile.

{% if cookiecutter.dataset_storage.s3.aws_profile %} --profile $(PROFILE){% endif %}

@chrisjkuch chrisjkuch added the bug label Dec 4, 2022
@chrisjkuch chrisjkuch self-assigned this Dec 4, 2022
@chrisjkuch chrisjkuch changed the title Sync direction reversed in v2 aws sync Fix v2 aws sync commands Dec 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant